Richard David Feinman

The phrase “Evidence-based Medicine” (EBM) guarantees its proponents a certain degree of protection. After all, who would be against medicine that is based on the data, on hard facts rather than opinion. On the other hand, a study that needs to cloak itself in such a self-aggrandizing phrase must raise a few eyebrows; as usual, the Dietary Guidelines, moves to the top of the list in that category but there are many examples.  Martin Tobin, professor of Medicine at Loyola College provided an excellent deconstruction of evidence based medicine [1]. Some of his points were that the grading system has divorced itself from basic science.  For example, he points out that:

• “ homeopathy uses drugs in which less than one molecule of active agent is present. … A meta-analysis of 89 placebo-controlled trials revealed a combined odds of 2.45 in favor of homeopathy. EBM grades meta-analysis as level 1 evidence but completely ignores scientific theory. There is nothing necessarily wrong with this particular meta-analysis, but the example illustrates how a system that grades findings of all meta-analyses as level 1 evidence is inherently flawed.  A grading system that ranks homeopathy as sounder evidence than centuries of pharmacologic science commits the reductio ad absurdum fallacy in logic.” [1]

Among the things that we found in our critique of the USDA dietary guidelines Report [2] was that the cited evidence did not meet their own standards. They were critical of low-carbohydrate diets on the basis of studies that their own analysis gave a “neutral” quality rating, even those that took dietary assessment at baseline and then assessed  cardiovascular mortality up to 12 years later.

But it is really the idea that there is some set of systematic definitions of science that everybody agrees on. My last post mentioned, by analogy with courts of law, the Frye standard which accepts as evidence, opinions supported by  “general acceptance’ in the scientific community.  While still accepted in some state courts, the federal courts have tried to go beyond trust in such narrow descriptions of science. In 1975, Congress established Federal Rules of Evidence.  The rules are quite general and the major impact is to broaden the range of evidence that could be considered.  Rule 401, defined relevance as  “evidence having any tendency to make the existence of any fact that is of consequence to the determination of the action more probable or less probable than it would be without the evidence,”  in other words, whatever works.  In a future post, I will discuss Daubert v. Merrell Dow, Inc. (pr. Dow-burt as in English), an outgrowth of the Rules of Evidence and generally considered the key judgment in the modern interaction of science and the law.  In the real world of jurisprudence, ideas on what constitutes scientific evidence have become problematical and Daubert may have had the paradoxical effect of restricting admissible data but, in the analogy with evidence in medicine, the Federal Rules of Evidence and Daubert have better captured the real quality of science in recognizing the need for flexibility. The kinds of absolute criteria — association does not imply causality, random controlled trials are a “gold standard,” etc. are at least different from the spirit of Daubert.

More important, nobody in any physical science would recognize the tables of levels of evidence.  A random controlled trial may be good for one kind of experiment but not for another and EBM is critical of “observational studies” but all of astronomy is observational.  In the end, most scientists would agree with the physicist Steven Weinberg, echoing Judge Potter Stewart’s famous take on pornography:

•  “There is no logical formula that establishes a sharp dividing line between a beautiful explanatory theory and a mere list of data, but we know the difference when we see it — we demand a simplicity and rigidity in our principles before we are willing to take them seriously [3].”

So where do these arbitrary guidelines in EBM come from?  They were set up by the  medical community, a community that is stereotyped as being untrained in science. I hate stereotypes, especially medical stereotypes since I think of myself as coming from a medical family (my father and oldest daughter are physicians) but stereotypes come from someplace and, of course, it is well known that physicians never study nutrition.  In the end, it makes me think of the undoubtedly apocryphal story about Mozart.

• A man comes to Mozart and wants to become a composer.  Mozart says that they have to study theory for  a couple of years, they should study orchestration and become proficient  at the piano, and goes on like this.  Finally, the man says “but you wrote your first symphony when you were 8 years old.”  Mozart says “Yes, but I didn’t ask anybody.”

Bibliography 

1. Tobin MJ: Counterpoint: evidence-based medicine lacks a sound scientific base. Chest 2008, 133(5):1071-1074; discussion 1074-1077.

2. Hite AH, Feinman RD, Guzman GE, Satin M, Schoenfeld PA, Wood RJ: In the face of contradictory evidence: report of the Dietary Guidelines for Americans Committee. Nutrition 2010, 26(10):915-924.

3. Weinberg S: Dreams of a final theory, 1st edn. New York: Pantheon Books; 1992.