Publication bias

Poetry of reality Science

We must know. We will know.

Physics Chemistry Biology Astronomy Geology

A view from the shoulders of giants.

Monkey typewriter theory Scientific objectivity Machine learning FAQ on radioactivity and nuclear technology Gravity Kurzgesagt Hippocrates Dentistry v - t - e

Publication bias, or the file-drawer effect, refers to the possibility of a systemic bias in published research in a field due to the over-reporting of positive results. The reasons for this can be:

• A positive result is usually more interesting than a study finding no effect. Thus, positive results tend to get published more often, even if they may be spurious. Many peer-reviewed publications have to reject many otherwise high-quality studies and need to rank publications not only on scientific merit but on scientific importance; positive results may be one contributor to the perceived importance of a study.
• The researcher, having found nothing, may simply be uninterested in writing up and reporting the results.
• The researcher may be self-interested against publishing negative results; for example, a curriculum vitae of published studies with many negative results is likely less promotion-worthy or grant-worthy than one with many positive results. A researcher might also have a financial interest in positive results, e.g., a pharmaceutical study in which the researcher is financially invested.

A study of published and unpublished clinical trials found that positive results in the sample were three times more likely to be published.^[1] Publication bias is known to be an issue with research funded by "Big Pharma" as pharmaceutical companies are often required to register their clinical trials in a public database but not to report all the results, allowing them to inflate the perceived efficacy of treatments. Some journals have been launched to report negative results to combat publication bias.^[2] John Ioannidis, a medical researcher, has provoked substantial academic debate with his claims that most published research is ultimately false.^[3]

A large research project involved seventy-three independent research teams analyzing the same data to test the same given hypothesis. The teams were free to use whatever additional data and any statistical methodologies they thought appropriate. There was a surprising diversity of results: some teams found a large negative independent variable effect, some a large positive, and some concluded there was not enough data to test the hypothesis. The authors descibed their findings as revealing a "hidden universe of uncertainty."^[4] Although the authors didn't specifically discuss publication bias, researchers who find significant results relative to others using the same or similar data would have better chances of getting published and cited.

A secondary impact of publication bias is on review articles and meta-analyses. Both types of publications summarize the existing literature and, thus, rely on either the inclusion of all studies or, at the very least, for unpublished studies to be otherwise similar to published studies. If unpublished studies are negative, publication bias further biases reviews and meta-analyses.

See also[edit]

• Peer review
• Reproducibility

External links[edit]

• Overviews of publication bias:
  • Cochrane Reviews
  • Comprehensive Meta-Analysis
  • Number Watch
• Entries in The Skeptic's Dictionary:
  • Positive Outcome (Publication) Bias
  • File Drawer Effect
• Steven Novella:
  • Are Most Medical Studies Wrong?
  • The Effect of Publication Bias
• Journals dedicated to publishing negative results:
  • All Results Journals
  • Journal of Articles in Support of the Null Hypothesis
  • Journal of Interesting Negative Results in Natural Language Processing and Machine Learning
  • Journal of Negative Results in Biomedicine
  • Psych File Drawer (archive)

References[edit]