By Steve Marsden, DVM ND MSOM Lac Dipl.CH CVA AHG

Interest in holistic medicine has swept through the entire population of North America, yet holistic practitioners are still an often stigmatized minority within the veterinary profession. What keeps them on the outside looking in, according to their critics, is a lack of evidence for their methods. In this context, evidence-based medicine (EBM) is becoming viewed as a political sword used to hamstring holistic practitioners, by denying CE credits, accreditation and recognition by regulatory bodies.

Is this perception of Evidence-based medicine accurate, however? Does holistic medicine lack an evidence base? Are the “anecdotal reports” of efficacy that each holistic practitioner has acquired meaningless in the pursuit of evidence-based practice? To begin to answer these questions, we have to start by defining what evidence-based medicine really is

Evidence based medicine (EBM), per the authors who first coined the term twenty years ago, is the process by which clinical expertise and best external evidence are integrated to make treatment decisions for a given patient. It is the integration of both quantitative data derived from research with qualitative data derived from experience, to achieve outcomes that are meaningful to the patient. Curing every illness is not the goal of EBM per se. Patient longevity, improvement of symptoms, improved quality of life, and minimization of inconvenience and cost are all outcomes emphasized in the most exemplary studies. Indeed, Evidence-Based Medicine does not really capture the stated intent of the individuals who pioneered the movement. “Evidence-informed healthcare” is more accurate, but for the purposes of this paper, we’ll continue to use the term EBM.

Regarding external evidence, there are different levels (see Figure 1, below), distinguished by the increasing cost, time and commitment needed to achieve their increasing freedom from bias. Randomized controlled trials (RCTs) provide the top level of external evidence, but even here, bias can be introduced. Studies may be double-blinded vs. triple-blinded, for example, and many treatments will never be tested at all, since they lack a sponsor with both adequate resources and in interest in the outcome.

The lowest level of external evidence is open to influence by personal bias and is represented by patient reports, case reports and expert opinion. Note that this is not to say that clinical expertise and case reports are irrelevant. They are simply weak as a form of external evidence but are entirely relevant to base a decision upon in evidence-based practice.³ Indeed, the very founders of the movement state that anecdotal clinical and patient experience is necessary in evidence-based practice, and that without it, “practice risks becoming tyrannised by evidence”.³ Furthermore, case reports are entirely necessary to stimulate the process of evidence accrual that will eventually result in the gold standard of evidence-based medicine – the clinical trial.

Level I	Experimental Studies	Randomized Controlled Trial	Participants randomly assigned to groups; gold standard of external evidence, but not always possible for ethical, practical and monetary reasons
Level II		Controlled Trial	No random assignment
Level III	Observational (Retrospective) Studies	Cohort	Outcomes from one group exposed to a particular variable are tallied over time and compared to outcomes from another unexposed group; correlation does not imply causation, however
Level IV		Case-Control	Outcomes of cases receiving a particular treatment are compared to those of similar cases not receiving the treatment
Level V		Case Series	Single or multiple cases published
Level VI	Experimental Studies	Laboratory Research	Before a treatment is tested in ‘real life’, some basis for expectation of results must be acquired through ‘bench studies’

 Above: Fig. 1 Levels of External Evidence to Inform Clinical Decisions

The levels of external evidence used to inform clinical decisions can be diagrammed as a pyramid.

Above: Fig. 2, External Evidence Pyramid

In general, they represent a progression or a set of phases of clinical research, although not all problems can or will be researched in clinical trials. Each tier is basically marked by:

• Higher cost
• Fewer studies
• Increasing time to achieve
• Increasing freedom from bias
• Increasing prospectivity

Development of clinical interventions begins at the bottom tier. Only years later, after a therapy has already been implemented on a wide scale, will sufficient evidence accrue to achieve the top two tiers. Critics, colleagues, regulators and professional associations thus display an ignorance of the scientific process when they propose to dismiss or ban holistic therapies from use for which there are no clinical trials showing efficacy. It’s an example of ‘putting the cart before the horse’, since individual case experience, accrued often one patient at a time, is an absolute requirement to eventually performing a clinical trial. Tolerance is thus required at the regulatory and professional association level so that this initial work can proceed, on the understanding that the intent is to eventually identify methodologies that assist all patients and clients. Supportive laboratory research alone (the bottom tier) should be considered sufficient to allow the work to proceed.

Randomized clinical trials provide the fodder for the two highest levels of evidence: the meta-analysis and the systematic review. Meta-analyses calculate overall risk of harm and benefit of an intervention, using several randomized controlled trials. A weighted average of clinical effect is calculated based on the sample size of each study to gain more reliable impression of a treatment’s efficacy.

Systematic reviews, in contrast, are commentaries that critically evaluate multiple studies focused on a single question. Where possible, they often include meta-analyses, assuming there was enough similarity in research methods and outcome assessment for data to be grouped together.

Perhaps the main generator of systematic reviews is the Cochrane Review, named in honor of Archie Cochrane, who championed in the 1970s the application of statistical methods to objectively determine treatment efficacy. The main objective of the Cochrane Review in performing its reviews is to make a determination as to whether a given treatment is beneficial or harmful based on analysis of all available clinical evidence. As of 2004 Cochrane had already reviewed about 150 different complementary therapies and overall concluded²:

• 38.4% had a positive or possibly positive effect
• 4.8% had no effect
• 0.69% had a harmful effect,
• 56.6% had insufficient evidence of any effect

Regarding conventional therapeutics¹

• 41.3% had positive or possibly positive effects
• 20% had no effect
• 8.1% had harmful effects
• 21.3% had insufficient evidence of an effect

Overall, then, the evidence base supporting complementary therapies is stronger than most would believe, disclosing a latent bias within our profession. In addition, holistic therapies were overall as safe and effective as conventional therapies.

Note, however, that holistic medicine is in no position to ‘rest on its laurels’. Almost all the systematic reviews published by Cochrane of holistic treatments do not include meta-analyses. Large sample sizes and standardization of research methodology do not yet exist for holistic therapies, so meta-analyses cannot be performed. Claims of efficacy thus lack mathematical rigor and clout. Few if any of these studies are specific to veterinary medicine, and most holistic therapies remain untested. Thus, many more trips up the pyramid steps are needed before we can say that holistic medicine has an adequate evidence base. Since most holistic therapies already have substantial basis in laboratory research, the bottom rung we should start from is the assembling of case series.

References Cited

1. Ezzo J, Bausell B, Moerman DE, Berman B, Hadhazy V (2001). “Reviewing the reviews. How strong is the evidence? How clear are the conclusions?”. Int J Technol Assess Health Care 17 (4): 457–466. PMID 11758290.
2. Committee on the Use of Complementary and Alternative Medicine by the American Public (2005). “5 State of Emerging Evidence on CAM: Cochrane Review Evidence for CAM”. Complementary and Alternative Medicine in the United States. Washington DC: The National Academies Press. pp. 135–6. ISBN 978-0-309-09270-8.
3. Sackett D, Rosenberg WCM, Gray, JAM, Haynes RB, and Richardson WS (1996). Evidence-based medicine: what it is and what it isn’t. BMJ vol. 312: 71-72