Methodological shortcomings predicted lower harm estimates in one of two sets of studies of clinical interventions

Roger Chou, Rongwei Fu, Susan Carson, Somnath Saha, Mark Helfand

Research output: Contribution to journalArticlepeer-review

27 Scopus citations


Objectives: High quality harms data are necessary to appropriately assess the balance between benefits and harms of interventions. Little is known, however, about whether perceived methodological shortcomings are associated with lower estimates of harms. Study Design and Setting: Studies reporting harms associated with carotid endarterectomy (CEA) and rofecoxib were identified using published systematic reviews. A standardized abstraction form, including eight predefined criteria for assessing the quality of harms reporting, was used to extract data. Univariate and multivariate analyses were performed to empirically evaluate the association between quality criteria and estimates of harms. Results: In 111 studies of CEA, meeting five of the eight-quality criteria was associated with significantly higher adverse event rates. A quality-rating instrument with four criteria predicted adverse events (5.7% in studies rated "adequate," compared to 3.9% in studies rated "inadequate" [P = 0.0003]). In multivariate analyses, the quality-rating assignment remained significant when controlling for other clinical and study-related variables. Different quality criteria, however, predicted estimates of risk for myocardial infarction in 16 trials of rofecoxib. Conclusion: The presence of methodological shortcomings can predict lower estimates of serious harms. Clinicians and researchers should consider methodological shortcomings when evaluating estimates of harms associated with clinical interventions.

Original languageEnglish
Pages (from-to)18-28
Number of pages11
JournalJournal of Clinical Epidemiology
Issue number1
StatePublished - Jan 2007
Externally publishedYes


  • Carotid
  • Cerebrovascular accident
  • Cyclooxygenase-2 inhibitors
  • Endarterectomy
  • Meta-analysis
  • Postoperative complications
  • Regression analysis


Dive into the research topics of 'Methodological shortcomings predicted lower harm estimates in one of two sets of studies of clinical interventions'. Together they form a unique fingerprint.

Cite this