How much spin is there with nonsignificant results?

January 01, 0001

How much spin is there with nonsignificant results?

Previous reports suggest that study data are sometimes distorted by authors’ interpretation. These British and French authors sought to characterize the nature and frequency of this distortion, which they call "spin" and define as "specific reporting strategies, whatever their motive, to highlight that the experimental treatment is beneficial, despite a statistically nonsignificant difference for the primary outcome, or to distract the reader from statistically nonsignificant results) in published reports of randomized controlled trials (RCTs) with statistically nonsignificant results for primary outcomes." They performed a MEDLINE using the Cochrane Highly Sensitive Search Strategy to identify reports of RCTs published in December 2006. Parallel-group RCTs with clearly identified primary outcomes but having no statistically significant.

The authors found: "From the 616 published reports of RCTs examined, 72 were eligible and appraised. The title was reported with spin in 13 articles (18.0 percent). Spin was identified in the Results and Conclusions sections of the abstracts of 27 (37.5) and 42 (58.3 percent) reports, respectively, with the conclusions of 17 (23.6 percent) focusing only on treatment effectiveness. Spin was identified in the main-text Results, Discussion, and Conclusions sections of 21 (29.2 percent), 31 (43.1 percent), and 36 (50.0 percent) reports, respectively. More than 40 percent of the reports had spin in at least 2 of these sections in the main text."

The authors concluded: "In this representative sample of RCTs published in 2006 with statistically nonsignificant primary outcomes, the reporting and interpretation of findings was frequently inconsistent with the results."

This study suggests we must be cautious when using authors interpretations of studies with negative results.

For the full abstract, click here.

JAMA 303(20):2058-2064, 26 May 2010 © 2010 to the American Medical Association
Reporting and Interpretation of Randomized Controlled Trials With Statistically Nonsignificant Results for Primary Outcomes. Isabelle Boutron, Susan Dutton, Philippe Ravaud, and Douglas G. Altman.

Category: A. General/Unspecified. Keywords: spin, non-significant results, data interpretation, randomized controlled trials, published data, systematic review, journal watch.
Synopsis edited by Dr Paul Schaefer, Toledo, Ohio. Posted on Global Family Doctor 2 July 2010

Pearls are an independent product of the Cochrane primary care group and are meant for educational use and not to guide clinical care.