Model Validity and Risk of Bias in Randomised, Placebo-Controlled, Trials of Non-individualised Homeopathic Treatment: Impact on Meta-Analysis Findings

Background: Randomised controlled trials (RCTs) of non-individualised homeopathic treatment (NIHT) apply a pre-selected medicine to typical symptoms of a medical condition. Meta-analysis of such RCTs revealed a small, statistically significant, effect greater than placebo. We have also assessed these RCTs for the risk of bias (RoB; extent of reliable evidence) and for model validity (MV; evidence of best therapeutic practice). Three RCTs were identified ‘reliable evidence’, based on RoB. When meta-analysis was restricted to these three RCTs, statistical significance was not maintained (pooled odds ratio [OR]: 1.39; 95% confidence interval [CI]: 0.84–2.33; p = 0.20), consistent with a conclusion that NIHT is not distinguishable from placebo. Nine trials were rated as having ‘acceptable MV’.

Objectives: To merge the RoB and MV findings, creating an overall quality rating for each RCT. To examine the impact of this quality rating on the meta-analysis results.

Methods: RCTs with uncertain RoB or low RoB were eligible for inclusion. A study was rated ‘high quality’ (reliable evidence and acceptable MV) or ‘moderate quality’ (uncertain RoB and/or uncertain MV) or ‘low quality’ (uncertain RoB and inadequate MV). One outcome measure per RCT was identified and used in sensitivity analysis based on overall quality rating.

Results: Twenty-six RCTs of NIHT were eligible; their meta-analysis yielded a statistically significant pooled OR. Only one RCT (on patients with menopausal syndrome) was rated overall ‘high quality’. Restricting analysis to that singular trial restored the statistical significance of NIHT compared with placebo (OR: 2.18; 95% CI: 1.06–4.47; p = 0.03).

Conclusion: Accommodating MV into an overall quality rating has an important impact on meta-analysis findings for RCTs of NIHT. Though the statistically significant finding from a solitary high-quality RCT is consistent with a conclusion that NIHT is distinguishable from placebo, more decisive interpretation will require results from considerably more high-quality RCTs.

Keywords: Meta-analysis, model validity, non-individualised homeopathy, randomised controlled trials, risk of bias, sensitivity analysis