Reporting bias undermines the integrity of the evidence base by inflating apparent drug efficacy and minimizing drug harms, thus skewing their risk-benefit ratio. This talk will review the topic of reporting bias with a focus on drugs prescribed for psychiatric conditions, especially depression, schizophrenia, bipolar disorder, and autism. Reporting bias is pervasive—although psychiatry/psychology may be the most seriously afflicted field, it occurs throughout medicine and science.
Responsibility lies with various parties—authors as well as journals, academia as well as industry—thus the motives appear to extend beyond the financial interests of drug companies. The desire for success, in combination with cognitive biases, can also influence academic authors and journals. Amid the flood of new medical information coming out each day, the attention of the news media and academic community is more likely to be captured by studies whose results are positive or newsworthy.
In the peer review system, a fundamental flaw arises from the fact that authors usually write manuscripts after they know the results. This allows hindsight and other biases to come into play, so data can be "tortured until they confess" (a detailed example is given). If a "publishable" result cannot be achieved, non-publication remains an option.
To address reporting bias, various measures have been undertaken, including registries. Drug regulatory agencies may play a more prominent role. It is suggested that journals borrow from the FDA review model. Because the significance of study results biases reviewers, results should be excluded from review until after a preliminary judgment of study scientific quality has been rendered, based on the original study protocol. Protocol publication can further enhance the credibility of the published literature.
Erick Turner, MD, is an Associate Professor in the Department of Psychiatry and the Department of Physiology & Pharmacology at Oregon Health & Science University (OHSU). Dr. Turner’s clinical affiliation is with the Portland VA Medical Center as a Staff Psychiatrist, in which capacity he treats adult outpatients with depression, bipolar disorder, PTSD, psychotic disorders, and other psychiatric diagnoses.
His early publications result from his research on mood disorders and circadian rhythms while a research fellow at the NIMH. His interest in reporting bias grew out of his work as an FDA Medical Officer, acting as gatekeeper for new psychotropic drugs seeking to enter the US market. In this capacity, he saw first hand how vast amounts of data relevant to drug efficacy and safety never see the light of day in publication. Since coming to OHSU, he and his colleagues have shown how reporting bias inflates drug efficacy by conducting a conventional meta-analysis with the published literature and a “control” meta-analysis with data from FDA Drug Approval Packages. His first such paper, published in the NEJM in 2008, dealt with drugs for depression and has been widely cited (>900 citations according to Scopus as of early 2015). A follow-up paper dealt with drugs for psychotic disorders, and a third paper, which deals with drugs for anxiety disorders, is currently in press.