Statistics and the detection of scientific misconduct
Citation Manager Formats
Make Comment
See Comments

The paper by Bolland et al.1 in this issue had a longer “gestation” than usual at Neurology®; it was received on December 4, 2015. We usually review on a tight timeline and publish important work quickly. But this manuscript has a different focus as it presents a statistical analysis that demonstrates probable scientific misconduct (fraud) on a large scale. It is also unique in that its analysis uses complicated methods that may be beyond what most readers are willing to tackle. However, the bottom line is simple: there are statistical methods that, when properly applied, can detect fraudulent behavior by investigators. Clinical trial data can thereby be validated or called into question. In this instance, analysis of a collection of studies by a large research group suggested that at least some of the studies were likely to be fraudulent.
Why did our review process take nearly a year? The findings in this report had implications for the authors and their institutions, for other journals and editors who published work by these authors, and for clinicians, guideline committees, and policymakers who rely on the validity of these findings. In the interests of all involved, we sought input from other editors, coauthors, and institutions. We asked 3 statisticians who regularly review for Neurology® to provide feedback on the manuscript. They validated the work and suggested important changes (as did the editorial team) that resulted in revisions of the article and the final version that is now published.
The present report analyzes 33 studies, 3 of which were published by Neurology. In the midst of our investigation and during the review process of the Bolland et al. manuscript, we heard from the main author of the examined studies (Sato), who admitted that the work reported in Neurology was fraudulent, relieved those he listed as coauthors of any wrongdoing, and requested retraction of the studies.
Fraud in an individual paper may be difficult to detect. Peer review, being a human process, can never be perfect; furthermore, the detection of likely fraudulent activity in the Bolland et al. paper required analysis of many trials. As such, it is unlikely that detection of fraud could have occurred during the review of single papers. Furthermore, simply because this group analysis suggested the likelihood of fraudulent activity, one cannot conclude that any one study—among those in the analysis—is, or is not, fraudulent. The authors were careful to avoid any deductions about individual studies, especially as there were multiple authors within the research group.
As part of our due process, we have notified other editors whose journals published papers by Sato et al., communicated with Sato's institution, and published retractions of the 3 papers and a letter published in Neurology.2–4 Each step involved our scientific integrity advisor, former editor-in-chief Robert Daroff. The final outcome is the publication of the Bolland et al. article. Our retractions help to correct the literature and any meta-analyses or guidelines that relied on those data, while publication of this paper shows the method and approach that led to the retractions. We also alert readers and authors that we will continue to use rigorous statistical review to detect fraud with the goal of maintaining the highest possible standards in publishing.
STUDY FUNDING
No targeted funding reported.
DISCLOSURE
Robert A. Gross is supported for educational endeavors from the University of Rochester Medical Center's Clinical and Translational Science Award from the NIH. Since his appointment as Editor-in-Chief in 2009, Dr. Gross has ceased participation in industry-sponsored clinical trials and speakers' bureaus. He receives an honorarium from AAN as Editor-in-Chief of Neurology.
Footnotes
See page 2391
- © 2016 American Academy of Neurology
REFERENCE
- 1.↵
- Bolland MJ,
- Avenell A,
- Gamble GD,
- Grey A
- 2.↵Retraction: Risedronate therapy for prevention of hip fracture after stroke in elderly women; Risedronate therapy for prevention of hip fracture after stroke in elderly women. Reply from the Authors. Neurology 2016;87:239. doi: 10.1212/WNL.0000000000002788.
- 3.↵Retraction: Risedronate and ergocalciferol prevent hip fracture in elderly men with Parkinson disease. Neurology 2016;87:239. doi: 10.1212/WNL.0000000000003021.
- 4.↵Retraction: Amelioration of osteoporosis and hypovitaminosis D by sunlight exposure in stroke patients. Neurology 2016;87:239. doi: 10.1212/WNL.0000000000003022.
Letters: Rapid online correspondence
REQUIREMENTS
If you are uploading a letter concerning an article:
You must have updated your disclosures within six months: http://submit.neurology.org
Your co-authors must send a completed Publishing Agreement Form to Neurology Staff (not necessary for the lead/corresponding author as the form below will suffice) before you upload your comment.
If you are responding to a comment that was written about an article you originally authored:
You (and co-authors) do not need to fill out forms or check disclosures as author forms are still valid
and apply to letter.
Submission specifications:
- Submissions must be < 200 words with < 5 references. Reference 1 must be the article on which you are commenting.
- Submissions should not have more than 5 authors. (Exception: original author replies can include all original authors of the article)
- Submit only on articles published within 6 months of issue date.
- Do not be redundant. Read any comments already posted on the article prior to submission.
- Submitted comments are subject to editing and editor review prior to posting.
You May Also be Interested in
Hemiplegic Migraine Associated With PRRT2 Variations A Clinical and Genetic Study
Dr. Robert Shapiro and Dr. Amynah Pradhan