Recommendations for Designing, Conducting and Reporting Observational Studies in Homeopathy

Abstract

Background Randomized placebo-controlled trials are considered to be the gold standard in clinical research and have the highest importance in the hierarchical system of evidence-based medicine. However, from the viewpoint of decision makers, due to lower external validity, practical results of efficacy research are often not in line with the huge investments made over decades.

Method We conducted a narrative review. With a special focus on homeopathy, we give an overview on cohort, comparative cohort, case-control and cross-sectional study designs and explain guidelines and tools that help to improve the quality of observational studies, such as the STROBE Statement, RECORD, GRACE and ENCePP Guide.

Results Within the conventional medical research field, two types of arguments have been employed in favor of observational studies. First, observational studies allow for a more generalizable and robust estimation of effects in clinical practice, and if cohorts are large enough, there is no over-estimation of effect sizes, as is often feared. We argue that observational research is needed to balance the current over-emphasis on internal validity at the expense of external validity. Thus, observational research can be considered an important research tool to describe “real-world” care settings and can assist with the design and inform the results of randomised controlled trails.

Conclusions We present recommendations for designing, conducting and reporting observational studies in homeopathy and provide recommendations to complement the STROBE Statement for homeopathic observational studies.

Keywords

observational studies - homeopathy - complementary and alternative medicine - STROBE guidelines - cohort studies - comparative cohort studies - case-control studies - cross-sectional studies

Highlights

• Observational research is needed to balance the current over-emphasis on internal validity at the expense of external validity, is an important research tool to describe real-world care settings, and can assist with the design and inform the results of RCTs.

• We give an overview on cohort, comparative cohort, case-control and cross-sectional study designs and explain guidelines relevant for homeopathy that help to improve the quality of observational studies, such as the STROBE Statement, RECORD, GRACE and the ENCePP Guide.

• We add information to the existing statements and guidelines that are specifically meant for homeopathy studies.

• References

• 1 Witt C, Linde K. Clinical Research in Complementary and Integrative Medicine. Munich: Elsevier; 2011
• 2 OCEBM Levels of Evidence Working Group. The Oxford 2011 levels of evidence. Available at: http://www.cebm.net/index.aspx?o=5653
• 3 Wittmann WW, Walach H. Evaluating complementary medicine: lessons to be learned from evaluation research. In: Lewith G, Jonas WB, Walach H. , eds. Clinical Research in Complementary Therapies: Principles, Problems, and Solutions. London: Churchill Livingston; 2002: 93-108
• 4 Rossi PH, Freeman HE. Evaluation. A Systematic Approach. Beverly Hills: Sage; 1982
• 5 Concato J, Shah N, Horwitz RI. Randomized, controlled trials, observational studies, and the hierarchy of research designs. N Engl J Med 2000; 342: 1887-1892
• 6 Concato J. Is it time for medicine-based evidence?. JAMA 2012; 307: 1641-1643
• 7 Concato J, Horwitz RI. Beyond randomised versus observational studies. Lancet 2004; 363: 1660-1661
• 8 Concato J, Lawler EV, Lew RA, Gaziano JM, Aslan M, Huang GD. Observational methods in comparative effectiveness research. Am J Med 2010; 123: e16-e23
• 9 Walach H, Loef M. Using a matrix-analytical approach to synthesizing evidence solved incompatibility problem in the hierarchy of evidence. J Clin Epidemiol 2015; 68: 1251-1260
• 10 Fønnebø V, Grimsgaard S, Walach H. , et al. Researching complementary and alternative treatments—the gatekeepers are not at home. BMC Med Res Methodol 2007; 7: 7
• 11 Walach H, Jonas WB, Lewith G. The role of outcomes research in evaluating complementary and alternative medicine. In: Lewith G, Jonas WB, Walach H. , eds. Clinical Research in Complementary Therapies: Principles, Problems, and Solutions. London: Churchill Livingston; 2002: 29-45
• 12 Witt CM, Lüdtke R, Baur R, Willich SN. Homeopathic medical practice: long-term results of a cohort study with 3981 patients. BMC Public Health 2005; 5: 115
• 13 Witt CM, Lüdtke R, Mengler N, Willich SN. How healthy are chronically ill patients after eight years of homeopathic treatment?--Results from a long term observational study. BMC Public Health 2008; 8: 413
• 14 Witt CM, Lüdtke R, Willich SN. Homeopathic treatment of patients with dysmenorrhea: a prospective observational study with 2 years follow-up. Arch Gynecol Obstet 2009; 280: 603-611
• 15 Witt CM, Lüdtke R, Willich SN. Homeopathic treatment of patients with psoriasis—a prospective observational study with 2 years follow-up. J Eur Acad Dermatol Venereol 2009; 23: 538-543
• 16 Teut M, Lüdtke R, Schnabel K, Willich SN, Witt CM. Homeopathic treatment of elderly patients--a prospective observational study with follow-up over a two year period. BMC Geriatr 2010; 10: 10
• 17 Thompson EA, Reillly D. The homeopathic approach to symptom control in the cancer patient: a prospective observational study. Palliat Med 2002; 16: 227-233
• 18 Thompson EA, Reilly D. The homeopathic approach to the treatment of symptoms of oestrogen withdrawal in breast cancer patients. A prospective observational study. Homeopathy 2003; 92: 131-134
• 19 Thompson EA, Montgomery A, Douglas D, Reilly D. A pilot, randomized, double-blinded, placebo-controlled trial of individualized homeopathy for symptoms of estrogen withdrawal in breast-cancer survivors. J Altern Complement Med 2005; 11: 13-20
• 20 Grimaldi-Bensouda L, Begaud B, Lert F. , et al; EPI3-LA-SER Group. Benchmarking the burden of 100 diseases: results of a nationwide representative survey within general practices. BMJ Open 2011; 1: e000215
• 21 Grimaldi-Bensouda L, Bégaud B, Rossignol M. , et al. Management of upper respiratory tract infections by different medical practices, including homeopathy, and consumption of antibiotics in primary care: the EPI3 cohort study in France 2007-2008. PLoS One 2014; 9: e89990
• 22 Grimaldi-Bensouda L, Abenhaim L, Massol J. , et al; EPI3-LA-SER Group. Utilization of psychotropic drugs by patients consulting for sleeping disorders in homeopathic and conventional primary care settings: the EPI3 cohort study. Homeopathy 2015; 104: 170-175
• 23 Grimaldi-Bensouda L, Abenhaim L, Massol J. , et al; EPI3-LA-SER group. Homeopathic medical practice for anxiety and depression in primary care: the EPI3 cohort study. BMC Complement Altern Med 2016; 16: 125
• 24 Danno K, Joubert C, Duru G, Vetel JM. Physician practicing preferences for conventional or homeopathic medicines in elderly subjects with musculoskeletal disorders in the EPI3-MSD cohort. Clin Epidemiol 2014; 6: 333-341
• 25 Rossignol M, Bégaud B, Avouac B. , et al. Who seeks primary care for musculoskeletal disorders (MSDs) with physicians prescribing homeopathic and other complementary medicine? Results from the EPI3-LASER survey in France. BMC Musculoskelet Disord 2011; 12: 21
• 26 Rossignol M, Begaud B, Engel P. , et al; EPI3-LA-SER group. Impact of physician preferences for homeopathic or conventional medicines on patients with musculoskeletal disorders: results from the EPI3-MSD cohort. Pharmacoepidemiol Drug Saf 2012; 21: 1093-1101
• 27 Kaur H, Chalia DS, Manchanda RK. Homeopathy in Public Health in India. Homeopathy 2019; 108: 76-87
• 28 Nayak C, Singh V, Singh K. , et al. Management of distress during climacteric years by homeopathic therapy. J Altern Complement Med 2011; 17: 1037-1042
• 29 Nayak C, Singh V, Singh VP. , et al. Homeopathy in chronic sinusitis: a prospective multi-centric observational study. Homeopathy 2012; 101: 84-91
• 30 Nayak C, Oberai P, Varanasi R. , et al. A prospective multi-centric open clinical trial of homeopathy in diabetic distal symmetric polyneuropathy. Homeopathy 2013; 102: 130-138
• 31 Oberai P, Balachandran I, Nair KJ. , et al. Homoeopathic management in depressive episodes: a prospective, unicentric, non-comparative, open-label observational study. Indian J Res Homoeopathy 2013; 7: 116-125
• 32 Oberai P, Gopinadhan S, Sharma A, Nayak C, Gautam K. Homoeopathic management of schizophrenia: a prospective, non-comparative, open-label observational study. Indian J Res Homoeopathy 2016; 10: 108-118
• 33 Rosenbaum PR, Rubin DB. The central role of the propensity score in observational studies for causal effects. Biometrika 1983; 70: 41-55
• 34 Stürmer T, Wyss R, Glynn RJ, Brookhart MA. Propensity scores for confounder adjustment when assessing the effects of medical interventions using nonexperimental study designs. J Intern Med 2014; 275: 570-580
• 35 Witt CM, Brinkhaus B, Pach D. , et al. Homoeopathic versus conventional therapy for atopic eczema in children: medical and economic results. Dermatology 2009; 219: 329-340
• 36 Roll S, Reinhold T, Pach D. , et al. Comparative effectiveness of homeopathic vs. conventional therapy in usual care of atopic eczema in children: long-term medical and economic outcomes. PLoS One 2013; 8: e54973
• 37 Nayak D, Chadha V, Jain S. , et al. Effect of adjuvant homeopathy with usual care in management of thrombocytopenia due to dengue: a comparative cohort study. Homeopathy 2019; 108: 150-157
• 38 Manchanda RK, Oberai P, Roja V. , et al. Evaluation of homeopathic medicines as add-on to institutional management protocol in acute encephalitis syndrome: An exploratory observational comparative study. Indian J Res Homoeopathy 2015; 9: 34-41
• 39 Ernster VL. Nested case-control studies. Prev Med 1994; 23: 587-590
• 40 Løhre A, Rise MB, Steinsbekk A. Characteristics of visitors to practitioners of homeopathy in a large adult Norwegian population (the HUNT 3 study). Homeopathy 2012; 101: 175-181
• 41 Stub T, Kristoffersen AE, Alræk T, Musial F, Steinsbekk A. Risk in homeopathy: Classification of adverse events and homeopathic aggravations--a cross sectional study among Norwegian homeopath patients. Complement Ther Med 2015; 23: 535-543
• 42 Bikker AP, Mercer SW, Reilly D. A pilot prospective study on the consultation and relational empathy, patient enablement, and health changes over 12 months in patients going to the Glasgow Homoeopathic Hospital. J Altern Complement Med 2005; 11: 591-600
• 43 Paterson C. Measuring outcomes in primary care: a patient generated measure, MYMOP, compared with the SF-36 health survey. BMJ 1996; 312: 1016-1020
• 44 Polus BI, Kimpton AJ, Walsh MJ. Use of the measure your medical outcome profile (MYMOP2) and W-BQ12 (Well-Being) outcomes measures to evaluate chiropractic treatment: an observational study. Chiropr Man Therap 2011; 19: 7
• 45 Thompson E, Viksveen P, Barron S. A patient reported outcome measure in homeopathic clinical practice for long-term conditions. Homeopathy 2016; 105: 309-317
• 46 Thompson EA, Mathie RT, Baitson ES. , et al. Towards standard setting for patient-reported outcomes in the NHS homeopathic hospitals. Homeopathy 2008; 97: 114-121
• 47 von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP. ; STROBE Initiative. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Lancet 2007; 370: 1453-1457
• 48 Vandenbroucke JP, von Elm E, Altman DG. , et al; STROBE Initiative. Strengthening the reporting of observational studies in epidemiology (STROBE): explanation and elaboration. Epidemiology 2007; 18: 805-835
• 49 Langan SM, Schmidt SAJ, Wing K. , et al. The reporting of studies conducted using observational routinely collected health data statement for Pharmaco-epidemiology (RECORD-PE). BMJ 2018; 363: k3532
• 50 Benchimol EI, Smeeth L, Guttmann A. , et al; RECORD Working Committee. The REporting of studies Conducted using Observational Routinely-collected health Data (RECORD) statement. PLoS Med 2015; 12: e1001885
• 51 Dreyer NA, Bryant A, Velentgas P. The GRACE checklist: a validated assessment tool for high quality observational studies of comparative effectiveness. J Manag Care Spec Pharm 2016; 22: 1107-1113
• 52 Dreyer NA, Schneeweiss S, McNeil BJ. , et al; GRACE Initiative. GRACE principles: recognizing high-quality observational studies of comparative effectiveness. Am J Manag Care 2010; 16: 467-471
• 53 Des Jarlais DC, Lyles C, Crepaz N. ; TREND Group. Improving the reporting quality of non-randomized evaluations of behavioral and public health interventions: the TREND statement. Am J Public Health 2004; 94: 361-366
• 54 van Haselen RA. Homeopathic clinical case reports: development of a supplement (HOM-CASE) to the CARE clinical case reporting guideline. Complement Ther Med 2016; 25: 78-85
• 55 Witt CM, Rafferty Withers S, Grant S, Lauer MS, Tunis S, Berman BM. What can comparative effectiveness research contribute to integrative health in international perspective?. J Altern Complement Med 2014; 20: 874-880
• 56 Viksveen P, Dymitr Z, Simoens S. Economic evaluations of homeopathy: a review. Eur J Health Econ 2014; 15: 157-174
• 57 Last J, Spasoff R, Harris S. A Dictionary of Epidemiology. Oxford: Oxford University Press; 2001
• 58 Comparative Effectiveness Research Initiative. Available at: https://www.hsph.harvard.edu/comparative-effectiveness-research-initiative/definition/ . Accessed December 15, 2019
• 59 Chang TI, Winkelmayer WC. Comparative effectiveness research: what is it and why do we need it in nephrology?. Nephrol Dial Transplant 2012; 27: 2156-2161
• 60 Witt CM, Chesney M, Gliklich R. , et al. Building a strategic framework for comparative effectiveness research in complementary and integrative medicine. Evid Based Complement Alternat Med 2012; 2012: 531096