Eliminating bias/living with bias?

Jens Chapman

The scientific method and elements of bias are fundamentally at odds with one another. While the prior seeks to identify truth through objective analysis, the latter wishes to prove one’s preconceptions through artificially created or filtered observations. Both the scientific method and bias are purpose-driven notions, both serve an intrinsic value. Arguably the more general and truly-lasting societal value is derived from our attempts at approximating truth through scientific methodology, compared with the more self-serving notions of reaffirming our preconceptions through bias. But what if we cannot really separate the two entities—the scientific method and bias?

We have explored the theme of bias in previous EBSJ editorials from the perspectives of investigator, commercial, and even publisher bias [1]. We have also learned that characteristics, which favor publication in major scientific medical journals, are United States–based articles that report good outcomes and are commercially funded [2]. Efforts to eliminate bias for the greater good of medicine are still far from complete but the movement to ‘rate’ research based on its Class of Evidence (CoE) is important, as it is basically a rating of the presence of bias. Please look through the Science in Spine article in this EBSJ issue on “Class or level of evidence” by Dettori (page 9–12). Without oversimplifying its content, the entire article underscores the value of a ‘high-class’ study compared with a ‘lower- class evidence’ by enumerating the incessant awareness it takes to eliminate bias with every step of the way of a research project and why the rating of the evidence is so important. Ultimately it is bias-free evidence that reveals the truth behind a hypothesis, shows previously unknown cross-connections, and allows us to rationally formulate new and more effective treatments.

The thought of improving upon nature through our medical interventions is perhaps one of the core motivators that has led us into the healthcare field in the first place. As physicians and healthcare providers, we by nature tend to be more optimists, we hope for improvement with our patients, and value positive achievements. However, this disposition may also lead us to displace negative experiences. Perhaps this is a helpful protective phenomenon as it provides us with capacity to move onward. But what if this core proclivity toward optimism is a problem in itself by introducing bias not only in our clinical medicine practices but also in our approach toward research? This unsettling thought was raised by Emerson et al (full disclosure: some of the authors are fellow employees at the University of Washington) in their recent article titled “Testing for a positive outcome bias in peer review: a randomized controlled trial.” With permission from the editors-in-chief of two leading orthopaedic publications, JBJS and CORR, the authors submitted a fabricated sham article either with positive outcomes or the very same sham article with ‘no-difference’ findings to 210 regular reviewers. A statistically significant number of reviewers recommended publication of the positive outcomes version of the sham paper compared with the negative version (97.3% versus 80.0%, P < .005) [3].

Of course this publication created a storm of protest as readers found fault with the study premise and details of the study [4]. Nevertheless, this publication should help us become more aware of the many obvious and surreptitious ways in which our preconceptions seep into our daily life and tend to influence our thinking. The power of positive is a great energizer but it is only through continuous testing and raising of our standards that we can increase our methodological awareness and ultimately advance in our field.

Jens Chapman

Editor-in-chief

• References

• 1 Chapman J. BMP debate: evidence in the name of science?. EBSJ 2011; 2 (2) 2-7
• 2 Lynch JR, Cunningham MR, Warme WJ et al. Commercially funded and United States-based research is more likely to be published: good-quality studies with negative outcomes are not. J Bone Joint Surg Am 2007; 89 (5) 1010-1018
• 3 Emerson GB, Warme WJ, Wolf FM et al. Testing for the presence of positive-outcome bias in peer review: a randomized controlled trial. Arch Intern Med 2010; 170 (21) 1934-1939
• 4 Mirkin JN, Bach PB. Outcome-blinded peer review. Arch Intern Med 2011; 171 (13) 1213-1214 author reply 1214