A concerning trend has emerged over the last few years, and it’s one that should make us sit up and think. “Retractions for all reasons, from honest error to plagiarism to the outright faking of data, are on the rise [1].”
Perhaps it’s the pull of the academic career path, the pressure to publicize or even lowered barriers to retraction that’s driving this change, but it presents us with some valid and valuable questions. What is good science? How do we insulate ourselves from the damage of retraction? It’s something Spinal Research takes seriously, because we have a cause to support, resource and back with solid evidence.
Concerns around the validity and reliability of scientific and research journals have existed for a number of years now, and some of the most vocal critics have occupied seats of influence in the industry. Two examples are Marcia Angell (MD) of the New England Journal of Medicine, and Richard Smith, editor of BMJ (previously known as the British Medical Journal). Their comments on the matter demand reflection.
“It is simply no longer possible to believe much of the clinical research that is published, or to rely on the judgment of trusted physicians or authoritative medical guidelines. I take no pleasure in this conclusion, which I reached slowly and reluctantly over my two decades as an editor of The New England Journal of Medicine (Marcia Angell MD).”
“…most of what appears in peer-reviewed journals is scientifically weak (Richard Smith, BMJ Editor).”
These two damaging conclusions, voiced by influential editors, could be somewhat disheartening for organisations and individuals who wish to seek out the truth rather than simply agreeable propaganda. Indeed, it is a consideration that foundations like ours hold close to heart.
We exist to facilitate research and disseminate knowledge that furthers the understanding, development and effectiveness of chiropractic care. This is our research culture statement [2].
This mandate is based around a desire to back the chiropractic profession with that which is solid, reliable and scientifically valid. According to Dr. Mark Uren*, head of the Spinal Research Clinical Advisory Panel, this serves a very important purpose. “It isn’t just something that needs to be done for our own benefit, but also for a greater population to be able to access chiropractic. If we want more funding bodies to get behind us, more government support, or more awareness in the greater population, then research is essential.”
It just has to be good research. The implication is twofold, demanding a robust process that honours both scientific integrity and the donor dollar.
So then – what distinguishes the scientifically stable from the shaky and unreliable? The starting point is undoubtedly an understanding of what makes science “good.” An untainted peer-review process, the ability to challenge every hypothesis via experiment, and the ability for results to stand up to scrutiny all factor high on the list.
It all comes from a catch-cry researchers will know all too well: “Trust, but verify [3].”
This is the simple idea that underpins science. Yet last year saw a peak number of retractions from giants like PubMed. Why is this so? Perhaps, in the race for publication, there was too much trust and not enough verification going on. This is a suspicion shared by a couple of voices in the industry [3]. The ability to replicate results may not always be possible, but there are some guiding rules that can indicate when replicability percentages just aren’t adding up.
“A rule of thumb among biotechnology venture-capitalists is that half of published research cannot be replicated. Even that may be optimistic. (In 2012), researchers at one biotech firm, Amgen, found they could reproduce just six of the 53 “landmark” studies in cancer research [3].”
Frankly, that number is low enough to demand more than a raised eyebrow or two. There’s a catch-22 here though: research doesn’t come cheap. Having to go back and check data has the power to greatly stymie progress, not to mention tie up some of the sharpest minds in biomedical science with backtracking rather than forward momentum.
Perhaps big-pharma can afford this cross check, but the fact is that chiropractic isn’t yet attracting the multi-billion dollar budgets that pharmacological and other such research is attracting. It may be years before (if ever) we do. We still do have a responsibility to advance our field though. Hence the scrutiny needs to come in way before the post-publication review.
Here is the interesting balance we need to strike: financial responsibility, scientific and research reliability, and relevance to chiropractic. For Spinal Research, this starts with a peer review process that is solid, in-depth and unsullied.
For Dr. Uren, this is a balance managed through the robustness of the Spinal Research application process. “We have a research culture statement. There’s a clinical component to it, and a scientific component. Each research application has to satisfy both. We want to be responsible with the Spinal Research budget and we don’t want to waste the time of researchers who might propose something that is scientifically valid but not relevant to chiropractors, or something that needs a bit more work before it is solid scientifically.”
“That is why we have the expression of interest early on in the process. This goes to the Clinical Advisory Panel – these are practicing chiropractors. Their purpose is to check the expressions of interest for their relevance to the field of chiropractic. This ensures we are staying true to the research culture statement.”
In order to pass the initial Expression of Interest, applications then go to the Scientific Review Panel. “These are researchers, academics and scientists working in the field, not necessarily chiropractors. Their purpose is to look at the scientific merit of each expression of interest.”
It’s a blinded process, ensuring the reviewers don’t know who the applicants are or what institution they are from. This is a purposefully maintained safeguard against bias, that allows solid critiques and feedback to be given early and through-out the process. This aspect is important, as research is to be encouraged, even if it isn’t solid enough to be funded just yet.
We don’t want to waste precious research time. Hence the early communication. “Sometimes its valuable research, but it can be funded somewhere else as its not that relevant to chiropractic. Research on medications for example. Its good research, its just not what we want to fund. For others, there might be merit in what they want to look at, but it requires fine-tuning. We encourage them to look at certain areas of the application and come back to us” says Dr. Uren. “Everything is filtered through two lenses: the lens of our research culture statement, and the sense of scientific validity and reliability.”
Based on the score given by each of these areas, some applicants will be invited to move through to the full application process. Here, the applications face both panels again, but with far more detail worked in.
Yes, the process is in-depth and detailed, but it is so on purpose. Putting the right footwork in at the beginning is integral to maintaining the healthy track record Spinal Research’s projects have enjoyed. The transparent and robust nature of the process is paying off too: the nature of work that has come out of Spinal Research grants is something of which we are truly proud.
There has been an increase in the amount of applications that have come to us in recent years, and not just from Australia. It’s an encouraging trend, especially when there is so much left to explore. According to Dr. Uren, this includes more research on the mechanisms behind how and why chiropractic works, on the effects of subluxations on human health, neurology and neural plasticity.
Supporting good science pertaining to the field we believe in continues to be both a mandate and a passion. We are proud to get behind the work that has to potential to widen the reach of chiropractic into a population that could benefit so much from what we have to offer.
Note *refers to the Australian Qualification, Doctor of Chiropractic
References
[1] Marcus, A and Oransky I (2014), “Retractions are coming thick and fast: Its time for publishers to act,” The Guardian, http://www.theguardian.com/science/blog/2014/jul/14/retractions-journal-publishers-scientific-papers-peer-review retrieved 7 May 2015
[2] Staff Writer, (2013), “Research” Spinal Research, http://spinalresearch.com.au/Research/tabid/148/language/en-US/Default.aspx retrieved 7 May 2015
[3] Staff Writer, (2013), “How Science Goes Wrong: Scientific Research Has Changed The World, Now It Needs To Change Itself,” The Economist, https://www.economist.com/news/leaders/21588069-scientific-research-has-changed-the-world-now-it-needs-to-change-itself-how-science-goes-wrong retrieved 7 May 2015
