Recommendations for Designing, Conducting and Reporting Observational Studies in Homeopathy

Abstract

Background Randomized placebo-controlled trials are considered to be the gold standard in clinical research and have the highest importance in the hierarchical system of evidence-based medicine. However, from the viewpoint of decision makers, due to lower external validity, practical results of efficacy research are often not in line with the huge investments made over decades.

Method We conducted a narrative review. With a special focus on homeopathy, we give an overview on cohort, comparative cohort, case-control and cross-sectional study designs and explain guidelines and tools that help to improve the quality of observational studies, such as the STROBE Statement, RECORD, GRACE and ENCePP Guide.

Results Within the conventional medical research field, two types of arguments have been employed in favor of observational studies. First, observational studies allow for a more generalizable and robust estimation of effects in clinical practice, and if cohorts are large enough, there is no over-estimation of effect sizes, as is often feared. We argue that observational research is needed to balance the current over-emphasis on internal validity at the expense of external validity. Thus, observational research can be considered an important research tool to describe “real-world” care settings and can assist with the design and inform the results of randomised controlled trails.

Conclusions We present recommendations for designing, conducting and reporting observational studies in homeopathy and provide recommendations to complement the STROBE Statement for homeopathic observational studies.

Keywords

observational studies - homeopathy - complementary and alternative medicine - STROBE guidelines - cohort studies - comparative cohort studies - case-control studies - cross-sectional studies

Introduction

In observational studies on the effects of a risk factor, diagnostic tests, treatments or other interventions are investigated in a natural environment without any experimental manipulation.

The three classical types of observational studies in epidemiological research are cohort, case-control and cross-sectional studies. Prospective and retrospective observational studies have to be distinguished. ‘Prospective’ means that first, the aims and methods are defined, and then patients are included, treated and observed. ‘Retrospective’ means that data collected in the past are analyzed.[1]

Within the hierarchical system applied in evidence-based medicine, the evidence of observational studies is usually placed below that of meta-analyses, systematic reviews and randomised controlled trials (RCTs) but above that of case studies and expert opinion.[2]

There is a long-standing debate, both within conventional research methodology and CAM (complementary and alternative medicine) research, on whether naturalistic or experimental designs are better to investigate treatment effects. In the area of evaluation studies in social science, there was a consensus in the late 1980s that experimental studies alone do not describe true effects in the natural world. Hence, observational research has to be employed alongside experimental studies.[3] [4] Within the conventional medical research field, two types of arguments have been employed in favor of observational studies. First, observational studies allow for a more generalizable and robust estimation of effects in clinical practice, and if cohorts are large enough, there is no over-estimation of effect sizes, as is often feared.[5] [6] [7] [8] It has been argued that internal and external validity are incompatible concepts.[9] Internal and external validity are identified in RCTs via an explanatory/pragmatic continuum. Therefore, no single study can maximize both. While explanatory randomized trials and experimental research maximize internal validity, observational studies may help to maximize external validity. Both are needed. Currently, the hierarchical concept of evidence in EBM (evidence-based medicine) leads to an over-emphasis of internal over external validity, and as a community we are in danger of over-emphasizing internal valid evidence that may turn out to be unhelpful as its external validity may be low. Observational studies can balance this situation, not as a replacement but as a complement to evidence from randomised and experimental studies.

This has been understood by researchers in the CAM field, where often the research logic is reversed. Interventions have been around and are often already legally regulated such that it is not mandatory, and often difficult, to conduct randomized studies.[10] Moreover, in CAM, often patients are treated who have long-standing, chronic diseases with a well-known baseline. In such situations, observational studies can give us valuable hints whether a further investigation by randomized trials is worthwhile.[11] Any good randomized study needs robust outcome documentation for power estimation. Hence, observational studies are useful from various angles.

Data from observational studies are obtained from patients in medical practice and therefore considered to represent and describe real-life settings. In most observational studies, there are no or only a few pre-defined inclusion and exclusion criteria and researchers intend to describe whatever is truly happening, as far as the quality of data allows this to be done. Observational studies also allow examination of multiple treatment comparisons. Very importantly for small research budgets, they are also usually much cheaper to realize than explanatory trials.

The aim of this article is to present and discuss recommendations for designing, conducting and reporting observational studies in homeopathy.

Types of Observational Studies

Cohort Studies

In cohort studies, a cohort, i.e., a group of people with some similarity, e.g., in treatment, diagnosis or location, is followed up and systematically observed within a given timeframe. The term “cohort” is derived from the Latin word “cohors”, which was, at the time of the Roman Empire, “a group of soldiers”. The classical research question in cohort studies is what happens to members of the cohort that have been exposed to a particular variable in comparison to members who have not been exposed or in comparison to a pre-defined baseline state. A classical cohort could be a “birth cohort”. All members of this group may have been born in a given year and are systematically followed up for the next decades by measuring certain outcomes which are of interest for the research question, e.g., onset of certain diseases, intelligence, height and weight. A classical exposure could be social status, living in the city versus living in the countryside and many other types of exposures. At the time of initiation of the study, some of the participants may have been exposed and others were not. The cohort is then followed up over time to evaluate the occurrence of the outcome of interest. The most often used type of cohort study is a prospective cohort study, which means that the study begins with a baseline assessment at the starting time point and progresses into the future with follow-up assessments of data. In a retrospective cohort study, the data are collected only from already extant records. The outcome has already occurred and is assessed retrospectively. In some cases, retrospective and prospective designs are mixed: the data at baseline assessment are gathered from records and then prospectively assessed from the sample.

Cohort studies have certain limitations. First, they describe the progress of the cohort under real-life conditions without experimentally intervening. Therefore, they are not suited to investigate therapeutic causality or specific drug effects (efficacy).

An example for a classical cohort study in homeopathy is the German cohort study of Witt et al[12] [13] which included nearly 4000 new patients undergoing a homeopathic case-taking and followed up this cohort for 8 years. This study allowed observation of who consulted a homeopathic physician, how frequently the consultations took place, which remedies and potencies were prescribed and how the outcomes, intensity of complaints and quality of life changed over time. This design does not allow us to prove that homeopathy is effective, because other reasons may explain the effects (such as natural course of disease, regression to the mean, other not assessed treatments that the patients may have used without documentation, and others). However, the results are in line with the narrative observations on the course of treatments from homeopaths and patients and may provide data for planning further studies on effectiveness of homeopathic treatment or efficacy of homeopathic drugs. This cohort was then analyzed and reported according to certain disease groups.[14] [15] [16]

Another example of a homeopathic observational study project used to design better RCTs is the studies conducted in the Glasgow Homeopathic Hospital on women with breast cancer: Thompson and Reilly[17] observed 100 patients attending a research clinic in what was then the Glasgow Homeopathic Hospital. This study demonstrated that the most frequent attendees were women who had undergone breast cancer treatments and were struggling with side effects and that, based on evaluation tools before and after five visits, improvements were seen for these problematic symptoms. The next step was to observe those women with breast cancer and symptoms of estrogen withdrawal, which allowed clarification that hot flushes, sleep and mood disturbance and joint pain were a constellation of symptoms that led them to seek help using non-pharmaceutical approaches, and attending a homeopathic clinic was associated with improvements in those symptoms.[18] Based on those results of the observational studies, the next step was to carry out a placebo-controlled RCT investigating the specific effects of homeopathy in the same clinical setting. Those data from observational studies were vital to allow a useful research setting and hypothesis to emerge.[19]

It is also possible to compare sub-groups within a prospective cohort study design. An example is the French EPI3 cohort.[20] [21] [22] [23] [24] [25] [26] In France, homeopathy is widely used by family physicians. In this cohort, 8,559 patients attending 825 family physician practices were followed up over time. Within the observational period patients were exposed to treatments by homeopathic physicians, conventional physicians and physicians practising both methods (mixed therapies). After a follow-up period, the patient groups were analyzed and sub-groups were compared according to a defined disease category, e.g., upper respiratory tract infections, musculoskeletal disease, sleep, anxiety and depressive disorders. Accordingly, clinical outcomes and the use of medications were compared. In most EPI3 evaluations so far, the patients of homeopathic physicians, mixed therapies and conventional physicians reported similar improvements, but homeopaths prescribed fewer conventional medications than conventional physicians.

In the case of the EPI3 cohort study, it is not possible to conclude that homeopathic medicines were responsible for the outcome. There may be other reasons that explain the outcomes. For example, patients chose their physician by preference. It could simply be possible that those patients who were more sceptical about conventional medication consulted homeopathic physicians, providing a care model based on less conventional medication.

An additional advantage of large observational or cohort studies is that a high number of included participants may allow us to investigate safety. In RCTs, the numbers of patients are usually too small to observe a reliable number of rare adverse drug reactions. In cohort studies including many thousand individuals, this is more likely. However, severe adverse events are more often reported in case reports and may be summarized in case series.

Observational studies may also be of high interest for health policy makers, because these are based on “real-world” data. Homeopathy is contributing substantially to Indian health care delivery. Homeopathic practitioners are an important resource in sustaining community health development in India.[27] The Indian Central Council for Research in Homoeopathy under Ministry of AYUSH (Ayurveda, Yoga & Naturopathy, Unani, Siddha and Homoeopathy), Govt. of India, has conducted many observational studies in primary health care settings to describe and investigate homeopathic health care strategies under primary care conditions and to get data to evaluate the usefulness of the therapy (e.g. for climacteric symptoms,[28] chronic sinusitis,[29] diabetic neuropathy,[30] depression,[31] schizophrenia[32] and many other disease conditions).

Comparative Cohort Studies

Although randomization is considered to be the optimal tool to reduce bias and achieve similar and comparable groups between the treatment arms, randomization is not always possible or appropriate. One important reason against randomization is patient preference. Patients may not be willing to participate in a trial because of the fear they may get a placebo or less effective treatment. A possible alternative to RCTs is a comparative observational or cohort study design. Patients may follow their treatment preferences and the treatment settings, e.g., homeopathic physician versus conventional physician, are authentic and represent “real-world” conditions. The disadvantage of these designs is that, in most cases, the intervention and control groups show differences in their characteristics and outcomes at baseline, which may confound the results of the study. Techniques exist to statistically control those baseline differences, such as propensity score matching.[33] However, they require large patient numbers. Thus, if they are small, comparative cohort studies have a low internal validity, but due to their real-world approach, they usually have a high external validity. Nevertheless, baseline differences can be statistically adjusted and balanced to a certain degree for potentially confounding factors, making the groups more comparable by multivariate statistics or propensity score.[34] The goal of an adjustment procedure is to minimise differences in confounding factors between the groups.

Adjusting for baseline differences should be done by an experienced statistician. The first step is identifying potentially confounding factors at baseline that may have influenced the outcomes. Examples for such confounders could be age, socioeconomic status, expectation, and differences in disease status.

An example of a classical comparative cohort study design in homeopathy is the study by Witt et al[35] and Roll et al[36] on atopic eczema in children. In this prospective multi-center comparative observational non-randomized rater-blinded study, 135 children (48 in the homeopathic and 87 in the conventional medicine group) with mild-to-moderate atopic eczema were included by their respective physicians. Depending on the specialization of the physician, the primary treatment was either standard conventional treatment or individualized homeopathy as delivered in routine medical care. In this study the effects of homeopathic treatment, after statistical adjustment for baseline differences, were similar to conventional treatment for children with mild-to-moderate atopic eczema, but involved higher costs for homeopathic treatment.

An Indian comparative observational study on thrombocytopenia due to dengue fever, conducted at an allopathic hospital set-up, resulted in better results of adjuvant homeopathy compared with usual care alone in increasing the platelet counts.[37] Another Indian comparative study on patients with acute encephalitis syndrome showed reduction in mortality and morbidity in the group administered adjuvant homeopathy to standard institutional management protocols compared with standard therapy alone.[38]

Another method to minimize confounding and bias before starting a study (at baseline) is to control using a matching technique.[1] The aim of a matching procedure is to find for every treated patient one (or more) control patients with similar observable characteristics against whom the effect of the treatment can be assessed. For a good matching, one should identify potential confounding factors at baseline by reflecting on the target population, discussing with experts and doing a literature search on comparable cohorts and populations. The more matching criteria that can be identified, the more comparable the groups become, but matching pairs also becomes less likely and more difficult to find. If too many criteria are applied, it is likely that in the end, matching will not be possible. Therefore, it is advisable to test a matching strategy in advance for feasibility.

Case-Control Studies

Another classical type of observational study is the case-control study. Here, researchers identify people with an existing health problem (“cases”) and a similar group without the problem (“controls”) and then compare them retrospectively with respect to exposure.[1] The term “exposure” can be applied to any factor of interest that may be associated with an outcome of interest. Retrospectively means that we are looking back in time, e.g., by comparing existing case records of patients with a certain exposure with “matched” case records from patients without the exposure. Retrospective studies have more problems with data quality in general because at the time of data recording, the data were usually never systematically recorded for the purpose of later being evaluated and compared. However, the method is useful for investigating rare diseases, initiating research, getting an overview, or generating hypotheses on a certain topic. It can be difficult to find a suitable group for comparison. There may also be a high risk of confounding when an exposure and an outcome are both strongly associated with a third (unknown) variable. They usually take less time to complete because the condition or disease has already occurred (e.g. is already documented in a case record). They can also take into account multiple risk factors.

It is also possible to include a case-control study within a much larger cohort study, called a nested case-control study.[39] This may allow control for rare cases that would otherwise be difficult to identify and include in the research.

Cross-Sectional Studies

In cross-sectional studies, measurements are only made at one time point. Thus, they provide a “snapshot” at one point in time. Therefore, they are mainly used to determine prevalence or associations for further studies. Compared with longitudinal cohort or case-control studies, within cross-sectional studies the included persons are usually not deliberately exposed or treated and not followed up, and it is not possible to differentiate between cause and effect or the sequence of events as in longitudinal studies. It is possible to test for associations, such as the prevalence of a disease with potential risk factors, but it is not possible to know whether the exposure preceded the effect, unless the cross-sectional study is broadened to a study with two time points. This would then be a panel-design cross-sectional study that could theoretically distinguish cause and effect. Because there is no exposure or treatment over time in a classical cross-sectional study, ethical issues are usually not critical. Most cross-sectional studies use questionnaires to gather information. Multiple outcomes can be studied. A crucial factor is the response rate, which clarifies the ratio between subjects responding and not responding to the call to participate in the study. With low numbers of responders, the result could be more biased. Another crucial factor to consider is the representativeness of the sample chosen. Ideally, the sample should be randomly selected from the population. An advantage of cross-sectional studies is that they are usually cheaper and easier to organize than longitudinal studies. They may be useful to generate hypotheses.

A few classical examples for cross-sectional studies in homeopathy are:

In the Norwegian HUNT 3 study,[40] the aim was to investigate characteristics of female and male visitors to practitioners of homeopathy in the large adult population in Norway. In total 50,827 participated (54% of the total population). The prevalence of visits to practitioners of homeopathy was 1.3%, a decline from 4.3% 10 years earlier. Both female and male visitors were four to five times more likely to experience recent somatic complaints. Further, female visitors were characterized by higher education, non-smoking, more chronic complaints and visiting a physician or a chiropractor in the past year, whereas male visitors were characterised by seeking help for psychiatric complaints and visiting a chiropractor. There were no associations between age, marital status, physical activity, perceived global health, respiratory, skin or musculoskeletal diseases and visiting practitioners of homeopathy.

In a cross-sectional survey reported by Stub et al,[41] patients were asked to register any reactions they had experienced 14 days after taking homeopathic remedies. A total of 26% of the participants reported worsening of symptoms. One third was classified as adverse events. Half of these were graded as minor and the other half as moderate according to the Common Terminology Criteria for Adverse Events. Two thirds were classified as homeopathic aggravations; of these, 73% were classified as minor and 27% as moderate, giving a tendency toward milder severity for those classified as homeopathic aggravations.

Practical Steps to Plan and Conduct an Observational Study

To plan and conduct an observational study in homeopathy, the steps outlined in [Table 1] may provide help.

Table 1 Practical steps to plan and conduct an observational study

Item	Description
Aims and objectives	What exactly is your research question? What setting do you want to describe? Take some time to write down your aims and research questions.
Background information	Find out how other researchers have investigated your field of interest and what they have found out. A systematic review on your topic would be a possible first step which helps you to understand your research field and later on helps you to design your observational study and discuss results. You may also contact experts in this field very early to include their knowledge and expertise. Based on the background information, is it necessary to change your research questions?
Design	Is an observational study the appropriate design for your research question? If you are aiming to investigate the efficacy of a drug, you will need an RCT design instead. If an observational design is appropriate, which study design will fit best to your research questions? Do you need a control group? Do you already have good data available and could use this for a retrospective case-control design? Or do you plan to investigate prospectively with a cohort or comparative cohort study design? Would a “snapshot” perspective derived from a cross sectional study design be an option?
Patients	Which setting, group of patients or sample do you want to investigate? In observational studies, your sample should represent “real-world” patients. Therefore, try to minimize inclusion and exclusion criteria. How do you get access to your patients? For prospective studies: Would patients be willing to participate in your study? For retrospective studies: Are you authorized to use and analyze the data, and in which way? Do you get access to the necessary data? Find out about data protection rules and regulations being relevant to your research question and data.
Therapy and setting	What is the therapy and the setting you are observing in your sample? Please observe and describe it. In retrospective studies, you may study a (random) small sample of medical records to determine if the documentation includes the necessary information you are looking for.
Control/Comparator	When using a control group, is this group/sample appropriate? If necessary, you may include a small field study and observe the control in an authentic medical setting. As for Therapy and Setting, in retrospective studies, you may study a (random) small sample of medical records to determine if the documentation includes the necessary information.
Medication	To organize your study and the data collection consider in advance which type of medication is used in your research sample/setting. For homeopathy, there are many possibilities like individualized, classical, constitutional, clinical, standardized, single constituent or complex (multi-constituent), isopathy. Also think about dose, potency, frequency/repetition. Similarly, for comparator, if applicable.
Outcomes	For prospective studies, identify potential outcomes that fit to your sample and reflect your research questions. Start with identifying outcomes in past studies and reflect on how appropriate those would be for your study. Take some time to take qualitative interviews with patients from your sample to determine about perceived outcomes that matter to the patients. You may also interview other clinical or research experts in your field. Another possibility for finding better outcomes is focus groups. A focus group is a form of qualitative research. A small, but relevant group of patients and/or experts is studied in guided or open discussions about the potential and relevant outcomes which can be expected from a larger population. This may help you to find out more about the changes perceived under your therapeutic setting and how to measure them. When choosing outcomes, try to include validated outcome measures as much as possible. Outcomes usually include disease-specific measures (such as SCORAD for atopic eczema, Diabetic distal symmetric polyneuropathy symptom score, or Acne-Specific Quality of Life questionnaire). With general measures, it is possible to observe general changes across different types of disease conditions and patients. Examples of general outcome measures used in homeopathic outcome studies are Quality of Life (SF36, SF12),[12] [13] the Glasgow Homeopathic Hospital Outcome Scale (GHHOS),[42] the Measure Yourself Medical Outcome Profile(MYMOP),[43] [44] [45] the intensity of symptoms on a numeric rating scale,[12] [13] and Outcome Related to Impact on Daily Living (ORIDL, formerly referred to as GHHOS).[46] In retrospective outcome studies, your random sample of medical records will enable you to get an overview of documented outcomes and quality of documentation before planning in more detail. Once you have decided upon outcomes, you have to consider using online or paper questionnaires, depending on your budget and logistical possibilities.
Statistics	Try to include an experienced statistician or biometrician as early as possible. This will help you to avoid mistakes, improve quality and lessen the burden of your work. The statistician will also assist and guide your team in building up a database for your outcome data documentation. The typical work of a statistician includes reflecting on design and methods, calculating a sample size (if necessary), writing down the statistical information and the statistical analysis plan for your study protocol, assisting in establishing a database for your data, analyzing and discussing your data, assisting in your publication and others.
Budget and funding	The budget you may use for your research is a crucial resource. Please reflect early on how much money you have for your study. Also calculate how much unpaid help you may receive. Many studies in homeopathy have been realized with low budgets and much unpaid help. However, funding and resources help to increase the quality of your research. Consider applying for funding or a grant with your study idea and protocol.
Team	Consider early what kind of expertise you may need and what kind of expertise and service you may have to recruit and pay for.
Study protocol	Draft your study protocol, and be sure to improve your draft by receiving critical reviews from peers and experts. Before including patients or data, register your study protocol online. Most peer-reviewed journals will want to check your manuscripts with your registered study protocols. Check your national study registry or use https://clinicaltrials.gov/.
Data and quality management	Consider using a quality- and data-management system/plan. Pre-defined guidelines and standard operating procedures (SOP) help to improve the quality of your research.
Ethics and legal situation	Find out which ethics board is responsible for dealing with your observational study protocol. Prepare the necessary documents and obtain an approval, if necessary. Find out about the legal situation regarding your planned study and follow the laws and regulations.
Other	Before conducting a large (and possibly expensive) observational study it may be beneficial to realize a small-scale pilot study as first step to test your approach and learn from any mistakes.

Guidelines and Resources That May Help in Designing and Reporting Observational Studies

There are international guidelines that can help to increase the quality of your work in planning, conducting and publishing research.

STROBE Statement

The most important guideline for reporting observational studies is the STROBE Statement, which stands for Strengthening The Reporting of Observational studies in Epidemiology and was developed by an international, collaborative initiative of epidemiologists, methodologists, statisticians, researchers and journal editors involved in the conduct and dissemination of observational studies.[47] [48] The website is: www.strobe-statement.org. The initiative established a short checklist of items that should be included in articles reporting outcome studies. The recommendations are restricted to the three main designs that are used in observational research: cohort, case-control, and cross-sectional studies. STROBE guides authors on how to report observational research well. One should be aware that the recommendations are not aimed at helping with designing or conducting studies. Additionally, the checklist is not an instrument to evaluate the quality of observational research. However, studying the checklist at the stage of planning a study will help to improve quality. At the STROBE website, there are many links to relevant publications and examples on how to report your results.

RECORD

RECORD stands for Reporting of studies Conducted using Observational Routinely collected Data.[49] This initiative, consisting of different stakeholders, developed guidelines for reporting for studies conducted using routinely collected health data (such as in health administration, electronic medical records, primary care surveillance and disease registries). The website is http://www.record-statement.org/. RECORD is an extension of the STROBE Statement and includes checklists and links to relevant publications. The RECORD-PE Guideline extension focuses on pharmaco-epidemiological research.[50]

GRACE

GRACE stands for Good Research for Comparative Effectiveness. The GRACE Initiative developed a set of key principles to be used to assess the quality of observational research for studying treatment effectiveness.[51] The website is www.graceprinciples.org. The initiative is working in collaboration with the International Society of Pharmaco-epidemiology and with some funding from the National Pharmaceutical Council. GRACE includes a set of principles and a checklist. The GRACE principles lay out the elements of good practice for the design, conduct, analysis and reporting of observational comparative effectiveness research (CER) studies.[52] The GRACE checklist is designed to guide the assessment of observational studies of comparative effectiveness in terms of their quality and usefulness for decision-making.

ENCePP Guide on Methodological Standards in Pharmaco-epidemiology

The European Network of Centres for Pharmaco-epidemiology and Pharmacovigilance (ENCePP) Guide on Methodological Standards in Pharmaco-epidemiology, published by the European Medicines Agency (EMA), offers a free and publicly available web resource for methodological English-language guidance in pharmaco-epidemiology. It provides links to selected published articles and guidelines that illustrate important principles of pharmaco-epidemiological research. It also includes a chapter on comparative effectiveness research. Website: http://www.encepp.eu/standards_and_guidances/methodologicalGuide.shtml

EQUATOR

An overarching provider of reporting guidelines and related resources is the Equator Network website. Here up-to-date links to health research reporting guidelines and extensions, such as STROBE, RECORD and others, can be found: http://www.equator-network.org/.

Other Resources

Depending on your research questions and design, you may also consider taking a look at other reporting guidelines:

If the intervention you are investigating in a study is more interventional than observational, you may benefit from the TREND Statement.[53] The mission of Transparent Reporting of Evaluations with Non-randomized Designs (TREND) is to improve the completeness and reporting of evaluations with non-randomized designs. The website with a checklist is freely available at https://www.cdc.gov/trendstatement/index.html

If your research has more the character of a case series you may take a look at the Hom-Case Extension to CARE Guideline.[54]

Additional Recommendations for Reporting Observational Studies in Homeopathy

As a supplement to the STROBE guidelines, we recommend that you consider assessing and reporting homeopathy-specific information in observational studies as described in [Table 2].

Table 2 STROBE Statement—checklist of items that should be included in reports of observational studies (cohort, case-control and cross-sectional studies) with additional recommendations for homeopathy studies

	Item no.	Recommendation	Additional recommendation for homeopathy studies
Title and abstract	1	(a) Indicate the study's design with a commonly used term in the title or the abstract
		(b) Provide in the abstract an informative and balanced summary of what was done and what was found
Introduction
Background/rationale	2	Explain the scientific background and rationale for the investigation being reported	Explain why homeopathy is investigated in your study
Objectives	3	State specific objectives, including any pre-specified hypotheses
Methods
Study design	4	Present key elements of study design early in the paper	Describe the rationale of comparators /control groups
Setting	5	Describe the setting, locations and relevant dates, including periods of recruitment, exposure, follow-up and data collection	Describe the medical setting, e.g., primary care, secondary care, hospital, public or private (also in control group) Describe the qualifications and experience of included homeopathic care providers (and providers in control groups)
Participants	6	(a) Cohort study—Give the eligibility criteria, and the sources and methods of selection of participants. Describe methods of follow-up Case-control study—Give the eligibility criteria, and the sources and methods of case ascertainment and control selection. Give the rationale for the choice of cases and controls Cross-sectional study—Give the eligibility criteria, and the sources and methods of selection of participants
		(b) Cohort study—For matched studies, give matching criteria and number of exposed and unexposed Case-control study—For matched studies, give matching criteria and the number of controls per case
Variables	7	Clearly define all outcomes, exposures, predictors, potential confounders, and effect modifiers. Give diagnostic criteria, if applicable	Define in advance on how to classify adverse events, adverse drug reactions and homeopathic aggravations
Data sources/measurement	8[a]	For each variable of interest, give sources of data and details of methods of assessment (measurement). Describe comparability of assessment methods if there is more than one group
Bias	9	Describe any efforts to address potential sources of bias
Study size	10	Explain how the study size was arrived at
Quantitative variables	11	Explain how quantitative variables were handled in the analyses. If applicable, describe which groupings were chosen and why
Statistical methods	12	(a) Describe all statistical methods, including those used to control for confounding
		(b) Describe any methods used to examine sub-groups and interactions
		(c) Explain how missing data were addressed
		(d) Cohort study—If applicable, explain how loss to follow-up was addressed Case-control study—If applicable, explain how matching of cases and controls was addressed Cross-sectional study—If applicable, describe analytical methods taking account of sampling strategy
		(e) Describe any sensitivity analyses
Results
Participants	13[a]	(a) Report numbers of individuals at each stage of study—e.g. numbers potentially eligible, examined for eligibility, confirmed eligible, included in the study, completing follow-up and analyzed
		(b) Give reasons for non-participation at each stage
		(c) Consider use of a flow diagram
Descriptive data	14[a]	(a) Give characteristics of study participants (e.g. demographic, clinical, social) and information on exposures and potential confounders	Describe the homeopathic patient (and control) sample. Report types of homeopathy used: individualized, formula, isopathy, mixed conventional + homeopathy (and therapies used in control groups). If possible, report on homeopathic analysis strategies applied (types of repertorisation). Report homeopathic remedies prescribed (frequencies, potencies, timing, form of medication), conventional medications prescribed (frequencies, dosages), and others (also in control). Also report on prescription criteria or treatment concepts being applied. Describe concomitant therapies and co-therapies (e.g. lifestyle advice) (also in control). Describe number and duration of consultations and follow-ups (also in control group).
		(b) Indicate number of participants with missing data for each variable of interest
		(c) Cohort study—Summarize follow-up time (e.g. average and total amount)
Outcome data	15[a]	Cohort study—Report numbers of outcome events or summary measures over time	Report adverse events, adverse drug reactions and homeopathic aggravations. Report on treatment adherence (optional, if assessed).
		Case-control study—Report numbers in each exposure category, or summary measures of exposure
		Cross-sectional study—Report numbers of outcome events or summary measures
Main results	16	(a) Give unadjusted estimates and, if applicable, confounder-adjusted estimates and their precision (e.g. 95% confidence interval). Make clear which confounders were adjusted for and why they were included
		(b) Report category boundaries when continuous variables were categorized
		(c) If relevant, consider translating estimates of relative risk into absolute risk for a meaningful time period
Other analyses	17	Report other analyses done—e.g., analyses of sub-groups and interactions, and sensitivity analyses
Discussion
Key results	18	Summarize key results with reference to study objectives
Limitations	19	Discuss limitations of the study, taking into account sources of potential bias or imprecision. Discuss both direction and magnitude of any potential bias
Interpretation	20	Give a cautious overall interpretation of results considering objectives, limitations, multiplicity of analyses, results from similar studies, and other relevant evidence
Generalisability	21	Discuss the generalisability (external validity) of the study results
Other information
Funding	22	Give the source of funding and the role of the funders for the present study and, if applicable, for the original study on which the present article is based

^a Give information separately for cases and controls in case-control studies and, if applicable, for exposed and unexposed groups in cohort and cross-sectional studies.

Discussion

Randomized placebo-controlled trials are still considered to be the gold standard in clinical research and have the highest importance in the hierarchical system of evidence-based medicine. However, from the viewpoint of decision makers, the practical results of efficacy research are not in line with the huge investments made over decades. They have yielded relatively little ‘‘actionable intelligence”, to quote Witt et al.[55] When an intervention has a specific effect and has been shown in an experimental study to have a relevant effect beyond the placebo, this does not necessarily mean that the therapy is effective under real-life conditions. Often, direct comparisons of effectiveness between different therapeutic strategies are missing. Such data would be helpful for decision makers. Pragmatic trials, including observational studies, may close this gap by focusing more on external instead of internal validity by observing and comparing real-world medical care settings.

In the last decades, a main focus of homeopathy research initiatives was to investigate specific effects of homeopathic drugs compared with placebo (efficacy research). Possible beneficial therapeutic strategies of the homeopathic therapeutic setting as a “whole medical system”, such as therapeutic relationship, empathy, triggering expectations, hope, use of suggestions and metaphors, resource activation, lifestyle advice, changing conventional medication and others, were reduced to experimental comparators in the classical RCT approach. Observational studies under real-world medical conditions have shown that homeopaths and other CAM health practitioners yield effects comparable to conventional therapists but with less conventional drug exposure and often at less cost.[56] Those “non-specific effects” are still a blind spot in researchers’ eyes, but may in fact be highly beneficial for patients and a resource for the health care system.

A new perspective also on observational research is offered by CER. Comparative effectiveness research strategies have been developed in the last decade to measure the extent to which an intervention, when deployed in the field in routine circumstances, does what it is intended to do for a specific population. CER is applied to investigate and observe “real world effects” (effectiveness, pragmatic and observational studies), in contrast to more experimental laboratory-like settings (efficacy) which often include highly pre-screened and selected patients defined by inclusion and exclusion criteria. CER is therefore more focused on the effectiveness and comparison of pragmatic clinical practice than investigating the specific effects of drugs in an experimental setting. CER is aimed at identifying what clinical and public health interventions and therapies work best for improving health under real-world conditions. The conceptual background of CER is that well-designed experimental randomized blinded trials, such as drug trials, are considered to have a high internal validity and a low risk of bias, but often exclude patients with co-morbidities, co-medications and special groups such as older adults or children. Thus, they represent only a limited part of the clinical care reality and may have a low external validity. CER instead favors pragmatic approaches—pragmatic RCTs and also observational studies—that focus on “effectiveness”, which is defined as “a measure of the extent to which a specific intervention, when deployed in the field in routine circumstances, does what it is intended to do for a specific population”.[57] Study methods may include randomized trials with at least two active (non-placebo) intervention arms, database studies, observational studies, model-based studies and decision analysis. Research projects that develop methods or infrastructure for CER would also be classifiable as CER.[58] Comparative effectiveness research is meant to be applied to different clinical strategies that may involve medical or non-medical preventive strategies, diagnostic testing (single tests or several in sequence or combination) and different treatments, including medications, devices, surgery or rehabilitative techniques.[59] Results of CER should also be useful for patients and providers in their shared decision-making and be broadly generalizable to the overall population or specific sub-groups. CER has been identified as a very important research strategy to investigate CAM treatments, because there is a lack of effectiveness data for decision makers.[60] Within the CER method, observational studies offer a toolbox to answer a wide range of research questions regarding the observation and description of real-world settings. In addition, participatory stakeholder involvement is used to develop more practically relevant research questions, designs and outcomes. CER is a research strategy that would help homeopaths to better investigate and present its benefits and advantages under real-life conditions, including the contribution of “non-specific effects”.

Observational research is needed to balance the current over-emphasis on internal validity at the expense of external validity. It is important to realize three areas where observational research can be valuable. For one, as already mentioned, it can be valuable as a preparatory type of research for designing good randomized studies. Second, it can be valuable as a stand-alone type of research, where pragmatic or ethical reasons stand against conducting a randomized study. Additionally, it can be valuable as the only adequate method where choices are involved: for instance, in any type of lifestyle research or where patients have very strong preferences, such as in homeopathy and other CAM. This might also lead to a diversification of research efforts and a broader, more realistic, picture of the effects of therapeutic interventions.

Conclusion

Observational studies can be considered an important research tool to describe real-world care settings and can assist with the design and inform the results of RCTs.

This article may assist homeopathic researchers to better design, conduct and report homeopathic observational studies.

Conflict of Interest

All authors except HW practice homeopathy as physicians besides their work as researchers.

Acknowledgments

We thank Homöopathiestiftung des Deutschen Zentralvereins homöopathischer Ärzte, Robert-Bosch Stiftung and WissHom, Germany, for supporting this project, the discussion process and the open-access publication of this article. We thank all participants of the 2019 HRI Workshop in London on “Guidelines for designing and reporting homeopathic research” for reviewing and discussing our manuscript.

Highlights

• Observational research is needed to balance the current over-emphasis on internal validity at the expense of external validity, is an important research tool to describe real-world care settings, and can assist with the design and inform the results of RCTs.

• We give an overview on cohort, comparative cohort, case-control and cross-sectional study designs and explain guidelines relevant for homeopathy that help to improve the quality of observational studies, such as the STROBE Statement, RECORD, GRACE and the ENCePP Guide.

• We add information to the existing statements and guidelines that are specifically meant for homeopathy studies.

• References

• 1 Witt C, Linde K. Clinical Research in Complementary and Integrative Medicine. Munich: Elsevier; 2011
• 2 OCEBM Levels of Evidence Working Group. The Oxford 2011 levels of evidence. Available at: http://www.cebm.net/index.aspx?o=5653
• 3 Wittmann WW, Walach H. Evaluating complementary medicine: lessons to be learned from evaluation research. In: Lewith G, Jonas WB, Walach H. , eds. Clinical Research in Complementary Therapies: Principles, Problems, and Solutions. London: Churchill Livingston; 2002: 93-108
• 4 Rossi PH, Freeman HE. Evaluation. A Systematic Approach. Beverly Hills: Sage; 1982
• 5 Concato J, Shah N, Horwitz RI. Randomized, controlled trials, observational studies, and the hierarchy of research designs. N Engl J Med 2000; 342: 1887-1892
• 6 Concato J. Is it time for medicine-based evidence?. JAMA 2012; 307: 1641-1643
• 7 Concato J, Horwitz RI. Beyond randomised versus observational studies. Lancet 2004; 363: 1660-1661
• 8 Concato J, Lawler EV, Lew RA, Gaziano JM, Aslan M, Huang GD. Observational methods in comparative effectiveness research. Am J Med 2010; 123: e16-e23
• 9 Walach H, Loef M. Using a matrix-analytical approach to synthesizing evidence solved incompatibility problem in the hierarchy of evidence. J Clin Epidemiol 2015; 68: 1251-1260
• 10 Fønnebø V, Grimsgaard S, Walach H. , et al. Researching complementary and alternative treatments—the gatekeepers are not at home. BMC Med Res Methodol 2007; 7: 7
• 11 Walach H, Jonas WB, Lewith G. The role of outcomes research in evaluating complementary and alternative medicine. In: Lewith G, Jonas WB, Walach H. , eds. Clinical Research in Complementary Therapies: Principles, Problems, and Solutions. London: Churchill Livingston; 2002: 29-45
• 12 Witt CM, Lüdtke R, Baur R, Willich SN. Homeopathic medical practice: long-term results of a cohort study with 3981 patients. BMC Public Health 2005; 5: 115
• 13 Witt CM, Lüdtke R, Mengler N, Willich SN. How healthy are chronically ill patients after eight years of homeopathic treatment?--Results from a long term observational study. BMC Public Health 2008; 8: 413
• 14 Witt CM, Lüdtke R, Willich SN. Homeopathic treatment of patients with dysmenorrhea: a prospective observational study with 2 years follow-up. Arch Gynecol Obstet 2009; 280: 603-611
• 15 Witt CM, Lüdtke R, Willich SN. Homeopathic treatment of patients with psoriasis—a prospective observational study with 2 years follow-up. J Eur Acad Dermatol Venereol 2009; 23: 538-543
• 16 Teut M, Lüdtke R, Schnabel K, Willich SN, Witt CM. Homeopathic treatment of elderly patients--a prospective observational study with follow-up over a two year period. BMC Geriatr 2010; 10: 10
• 17 Thompson EA, Reillly D. The homeopathic approach to symptom control in the cancer patient: a prospective observational study. Palliat Med 2002; 16: 227-233
• 18 Thompson EA, Reilly D. The homeopathic approach to the treatment of symptoms of oestrogen withdrawal in breast cancer patients. A prospective observational study. Homeopathy 2003; 92: 131-134
• 19 Thompson EA, Montgomery A, Douglas D, Reilly D. A pilot, randomized, double-blinded, placebo-controlled trial of individualized homeopathy for symptoms of estrogen withdrawal in breast-cancer survivors. J Altern Complement Med 2005; 11: 13-20
• 20 Grimaldi-Bensouda L, Begaud B, Lert F. , et al; EPI3-LA-SER Group. Benchmarking the burden of 100 diseases: results of a nationwide representative survey within general practices. BMJ Open 2011; 1: e000215
• 21 Grimaldi-Bensouda L, Bégaud B, Rossignol M. , et al. Management of upper respiratory tract infections by different medical practices, including homeopathy, and consumption of antibiotics in primary care: the EPI3 cohort study in France 2007-2008. PLoS One 2014; 9: e89990
• 22 Grimaldi-Bensouda L, Abenhaim L, Massol J. , et al; EPI3-LA-SER Group. Utilization of psychotropic drugs by patients consulting for sleeping disorders in homeopathic and conventional primary care settings: the EPI3 cohort study. Homeopathy 2015; 104: 170-175
• 23 Grimaldi-Bensouda L, Abenhaim L, Massol J. , et al; EPI3-LA-SER group. Homeopathic medical practice for anxiety and depression in primary care: the EPI3 cohort study. BMC Complement Altern Med 2016; 16: 125
• 24 Danno K, Joubert C, Duru G, Vetel JM. Physician practicing preferences for conventional or homeopathic medicines in elderly subjects with musculoskeletal disorders in the EPI3-MSD cohort. Clin Epidemiol 2014; 6: 333-341
• 25 Rossignol M, Bégaud B, Avouac B. , et al. Who seeks primary care for musculoskeletal disorders (MSDs) with physicians prescribing homeopathic and other complementary medicine? Results from the EPI3-LASER survey in France. BMC Musculoskelet Disord 2011; 12: 21
• 26 Rossignol M, Begaud B, Engel P. , et al; EPI3-LA-SER group. Impact of physician preferences for homeopathic or conventional medicines on patients with musculoskeletal disorders: results from the EPI3-MSD cohort. Pharmacoepidemiol Drug Saf 2012; 21: 1093-1101
• 27 Kaur H, Chalia DS, Manchanda RK. Homeopathy in Public Health in India. Homeopathy 2019; 108: 76-87
• 28 Nayak C, Singh V, Singh K. , et al. Management of distress during climacteric years by homeopathic therapy. J Altern Complement Med 2011; 17: 1037-1042
• 29 Nayak C, Singh V, Singh VP. , et al. Homeopathy in chronic sinusitis: a prospective multi-centric observational study. Homeopathy 2012; 101: 84-91
• 30 Nayak C, Oberai P, Varanasi R. , et al. A prospective multi-centric open clinical trial of homeopathy in diabetic distal symmetric polyneuropathy. Homeopathy 2013; 102: 130-138
• 31 Oberai P, Balachandran I, Nair KJ. , et al. Homoeopathic management in depressive episodes: a prospective, unicentric, non-comparative, open-label observational study. Indian J Res Homoeopathy 2013; 7: 116-125
• 32 Oberai P, Gopinadhan S, Sharma A, Nayak C, Gautam K. Homoeopathic management of schizophrenia: a prospective, non-comparative, open-label observational study. Indian J Res Homoeopathy 2016; 10: 108-118
• 33 Rosenbaum PR, Rubin DB. The central role of the propensity score in observational studies for causal effects. Biometrika 1983; 70: 41-55
• 34 Stürmer T, Wyss R, Glynn RJ, Brookhart MA. Propensity scores for confounder adjustment when assessing the effects of medical interventions using nonexperimental study designs. J Intern Med 2014; 275: 570-580
• 35 Witt CM, Brinkhaus B, Pach D. , et al. Homoeopathic versus conventional therapy for atopic eczema in children: medical and economic results. Dermatology 2009; 219: 329-340
• 36 Roll S, Reinhold T, Pach D. , et al. Comparative effectiveness of homeopathic vs. conventional therapy in usual care of atopic eczema in children: long-term medical and economic outcomes. PLoS One 2013; 8: e54973
• 37 Nayak D, Chadha V, Jain S. , et al. Effect of adjuvant homeopathy with usual care in management of thrombocytopenia due to dengue: a comparative cohort study. Homeopathy 2019; 108: 150-157
• 38 Manchanda RK, Oberai P, Roja V. , et al. Evaluation of homeopathic medicines as add-on to institutional management protocol in acute encephalitis syndrome: An exploratory observational comparative study. Indian J Res Homoeopathy 2015; 9: 34-41
• 39 Ernster VL. Nested case-control studies. Prev Med 1994; 23: 587-590
• 40 Løhre A, Rise MB, Steinsbekk A. Characteristics of visitors to practitioners of homeopathy in a large adult Norwegian population (the HUNT 3 study). Homeopathy 2012; 101: 175-181
• 41 Stub T, Kristoffersen AE, Alræk T, Musial F, Steinsbekk A. Risk in homeopathy: Classification of adverse events and homeopathic aggravations--a cross sectional study among Norwegian homeopath patients. Complement Ther Med 2015; 23: 535-543
• 42 Bikker AP, Mercer SW, Reilly D. A pilot prospective study on the consultation and relational empathy, patient enablement, and health changes over 12 months in patients going to the Glasgow Homoeopathic Hospital. J Altern Complement Med 2005; 11: 591-600
• 43 Paterson C. Measuring outcomes in primary care: a patient generated measure, MYMOP, compared with the SF-36 health survey. BMJ 1996; 312: 1016-1020
• 44 Polus BI, Kimpton AJ, Walsh MJ. Use of the measure your medical outcome profile (MYMOP2) and W-BQ12 (Well-Being) outcomes measures to evaluate chiropractic treatment: an observational study. Chiropr Man Therap 2011; 19: 7
• 45 Thompson E, Viksveen P, Barron S. A patient reported outcome measure in homeopathic clinical practice for long-term conditions. Homeopathy 2016; 105: 309-317
• 46 Thompson EA, Mathie RT, Baitson ES. , et al. Towards standard setting for patient-reported outcomes in the NHS homeopathic hospitals. Homeopathy 2008; 97: 114-121
• 47 von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP. ; STROBE Initiative. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Lancet 2007; 370: 1453-1457
• 48 Vandenbroucke JP, von Elm E, Altman DG. , et al; STROBE Initiative. Strengthening the reporting of observational studies in epidemiology (STROBE): explanation and elaboration. Epidemiology 2007; 18: 805-835
• 49 Langan SM, Schmidt SAJ, Wing K. , et al. The reporting of studies conducted using observational routinely collected health data statement for Pharmaco-epidemiology (RECORD-PE). BMJ 2018; 363: k3532
• 50 Benchimol EI, Smeeth L, Guttmann A. , et al; RECORD Working Committee. The REporting of studies Conducted using Observational Routinely-collected health Data (RECORD) statement. PLoS Med 2015; 12: e1001885
• 51 Dreyer NA, Bryant A, Velentgas P. The GRACE checklist: a validated assessment tool for high quality observational studies of comparative effectiveness. J Manag Care Spec Pharm 2016; 22: 1107-1113
• 52 Dreyer NA, Schneeweiss S, McNeil BJ. , et al; GRACE Initiative. GRACE principles: recognizing high-quality observational studies of comparative effectiveness. Am J Manag Care 2010; 16: 467-471
• 53 Des Jarlais DC, Lyles C, Crepaz N. ; TREND Group. Improving the reporting quality of non-randomized evaluations of behavioral and public health interventions: the TREND statement. Am J Public Health 2004; 94: 361-366
• 54 van Haselen RA. Homeopathic clinical case reports: development of a supplement (HOM-CASE) to the CARE clinical case reporting guideline. Complement Ther Med 2016; 25: 78-85
• 55 Witt CM, Rafferty Withers S, Grant S, Lauer MS, Tunis S, Berman BM. What can comparative effectiveness research contribute to integrative health in international perspective?. J Altern Complement Med 2014; 20: 874-880
• 56 Viksveen P, Dymitr Z, Simoens S. Economic evaluations of homeopathy: a review. Eur J Health Econ 2014; 15: 157-174
• 57 Last J, Spasoff R, Harris S. A Dictionary of Epidemiology. Oxford: Oxford University Press; 2001
• 58 Comparative Effectiveness Research Initiative. Available at: https://www.hsph.harvard.edu/comparative-effectiveness-research-initiative/definition/ . Accessed December 15, 2019
• 59 Chang TI, Winkelmayer WC. Comparative effectiveness research: what is it and why do we need it in nephrology?. Nephrol Dial Transplant 2012; 27: 2156-2161
• 60 Witt CM, Chesney M, Gliklich R. , et al. Building a strategic framework for comparative effectiveness research in complementary and integrative medicine. Evid Based Complement Alternat Med 2012; 2012: 531096