Technical rigour, exaggeration, and peer reviewing in the publishing of medical research

Abstract

To say that accuracy is of paramount importance in academic publishing is little short of a platitude. Nevertheless, the question of how accuracy can be ensured is not a trivial one in the real world, given the landscape of competing interests and a plethora of practical constraints and limitations. Though the primary responsibility lies with authors themselves, the framework of peer review, managed and overseen by editors, was developed in part to ensure robustness and resilience of the system as a whole. Analysing how often and why the system fails is inherently difficult as in most instances the specific processes are not visible to parties not directly involved in the handling of a prospective paper. For this reason it is important to encourage the reporting of case studies which should inform avenues for potential improvement. The present manuscript describes an example which illustrates well a number of flaws of the peer review framework as it exists today. In particular, I detail a number of serious errors in an article recently published in a leading journal, which include conclusions not supported by evidence, methodological flaws, and ill-conceived statistical analysis, alarmist and exaggerated manner in which the findings are communicated in the media, and the poor handling of these issues by the journal’s editorial board. In conclusion, the research community should exert a concerted effort to report, discuss, and document instances of criticisms of published scientific work being silenced.