…the association has to be strong and the causality has to be plausible and consistent. And you have to have some reason to make the observation; you can’t look at everything. And experimentally, observation may be all that you have — almost all of astronomy is observational. Of course, the great deconstructions of crazy nutritional science — several from Mike Eades blog and Tom Naughton’s hysterically funny-but-true course in how to be a scientist — are still right on but, strictly speaking, it is the faulty logic of the studies and the whacko observations that is the problem, not simply that they are observational. It is the strength and reliability of the association that tells you whether causality is implied. Reducing carbohydrates lowers triglycerides. There is a causal link. You have to be capable of the state of mind of the low-fat politburo not to see this (for example, Circulation, May 24, 2011; 123(20): 2292 – 2333).
It is frequently said that observational studies are only good for generating hypotheses but it is really the other way around. All studies are generated by hypotheses. As Einstein put it: your theory determines what you measure. I ran my post on the red meat story passed April Smith and her reaction was “why red meat? Why not pancakes” which is exactly right. Any number of things can be observed. Once you pick, you have a hypothesis.
Where did the first law of thermodynamics come from?
Thermodynamics is an interesting case. The history of the second law involves a complicated interplay of observation and theory. The idea that there was an absolute limit to how efficient you could make a machine and by extension that all real processes were necessarily inefficient largely comes from the brain power of Carnot. He saw that you could not extract as work all of the heat you put into a machine. Clausius encapsulated it into the idea of the entropy as in my Youtube video.
©2004 Robin A. Feinman
The origins of the first law, the conservation of energy, are a little stranger. It turns out that it was described more than twenty years after the second law and it has been attributed to several people, for a while, to the German physicist von Helmholtz. These days, credit is given to a somewhat eccentric German physician named Robert Julius Mayer. Although trained as a doctor, Mayer did not like to deal with patients and was instead more interested in physics and religion which he seemed to think were the same thing. He took a job as a shipboard physician on an expedition to the South Seas since that would give him time to work on his main interests. It was in Jakarta where, while treating an epidemic with the practice then of blood letting, that he noticed that the venous blood of the sailors was much brighter than when they were in colder climates as if “I had struck an artery.” He attributed this to a reduced need for the sailors to use oxygen for heat and from this observation, he somehow leapt to the grand principle of conservation of energy, that the total amount of heat and work and any other forms of energy does not change but can only be interconverted. It is still unknown what kind of connections in his brain led him to this conclusion. The period (1848) corresponds to the point at which science separated from philosophy. Mayer seems to have had one foot in each world and described things in the following incomprehensible way:
- If two bodies find themselves in a given difference, then they could remain in a state of rest after the annihilation of [that] difference if the forces that were communicated to them as a result of the leveling of the difference could cease to exist; but if they are assumed to be indestructible, then the still persisting forces, as causes of changes in relationship, will again reestablish the original present difference.
(I have not looked for it but one can only imagine what the original German was like). Warmth Disperses and Time Passes. The History of Heat, Von Baeyer’s popular book on thermodynamics, describes the ups and downs of Mayer’s life, including the death of three of his children which, in combination with rejection of his ideas, led to hospitalization but ultimate recognition and knighthood. Surely this was a great observational study although, as von Baeyer put it, it did require “the jumbled flashes of insight in that sweltering ship’s cabin on the other side of the world.”
It is also true that association does imply causation but, again, the association has to have some impact and the proposed causality has to make sense. In some way, purely observational experiments are rare. As Pasteur pointed out, even serendipity is favored by preparation. Most observational experiments must be a reflection of some hypothesis. Otherwise you’re wasting tax-payer’s money; a kiss of death on a grant application is to imply that “it would be good to look at.…” You always have to have something in mind. The great observational studies like the Framingham Study are bad because they have no null hypothesis. When the Framingham study first showed that there was no association between dietary total and saturated fat or dietary cholesterol, the hypothesis was quickly defended. The investigators were so tied to a preconceived hypothesis, that there was hardly any point in making the observations.
In fact, a negative result is always stronger than one showing consistency; consistent sunrises will go by the wayside if the sun fails to come up once. It is the lack of an association between the decrease in fat consumption during the epidemic of obesity and diabetes that is so striking. The figure above shows that the increase in carbohydrate consumption is consistent with the causal role of dietary carbohydrate in creating anabolic hormonal effects and with the poor satiating effects of carbohydrates — almost all of the increase of calories during the epidemic of obesity and diabetes has been due to carbohydrates. However, this observation is not as strong as the lack of an identifiable association of obesity and diabetes with fat consumption. It is the 14 % decrease in the absolute amount of saturated fat for men that is the problem. If the decrease in fat were associated with decrease in obesity, diabetes and cardiovascular disease, there is little doubt that the USDA would be quick to identify causality. In fact, whereas you can find the occasional low-fat trial that succeeds, if the diet-heart hypothesis were as described, they should not fail. There should not be a single Women’s Health Initiative, there should not be a single Framingham study, not one.
Sometimes more association would be better. Take intention-to-treat. Please. In this strange statistical idea, if you assign a person to a particular intervention, diet or drug, then you must include the outcome data (weight loss, change in blood pressure) for that person even if the do not comply with the protocol (go off the diet, stop taking the pills). Why would anybody propose such a thing, never mind actually insist on it as some medical journals or granting agencies do? When you actually ask people who support ITT, you don’t get coherent answers. They say that if you just look at per protocol data (only from people who stayed in the experiment), then by excluding the drop-outs, you would introduce bias but when you ask them to explain that you get something along the lines of Darwin and the peas growing on the wrong side of the pod. The basic idea, if there is one, is that the reason that people gave up on their diet or stopped taking the pills was because of an inherent feature of the intervention: made them sick, drowsy or something like that. While this is one possible hypothesis and should be tested, there are millions of others — the doctor was subtly discouraging about the diet, or the participants were like some of my relatives who can’t remember where they put their pills, or the diet book was written in Russian, or the diet book was not written in Russian etc. I will discuss ITT in a future post but for the issue at hand: if you do a per-protocol you will observe what happens to people when stay on their diet and you will have an association between the content of the diet and performance. With an ITT analysis, you will be able to observe what happens when people are told to follow a diet and you will have an association between assignment to a diet and performance. Both are observational experiments with an association between variables but they have different likelihood of providing a sense of causality.
Richard David Feinman 
Hi Dr. Feinman. You might be interested in the book “The Theory That Would Not Die”, about the history of Bayes Theorem. Some of what you discuss above sounds ripe for Bayesian hypothesis testing. I strongly suspect that most observational studies would yield posterior odds ratios for competing hypotheses close to 1, indicating that they actually contain rather little information for distinguishing different hypotheses.
Interestingly, a pivotal development in the practical use of Bayes’ Theorem (not discussed in the book above) involved the generalization of the concept of entropy; or rather the recognition that statistical mechanics was an application. If you’re brave enough to face non-equilibrium thermodynamics, you might want check out “Probability Theory: The Logic of Science”. Amongst other things it discusses how entropy is really a measure of “symmetry of ignorance”.
I don’t know much about Bayesian theory but my understanding is that you can refine a hypothesis or sharpen its probabilty by an iterative process from successive trials but I assume that means the results from each iteration are meaningful and I am not sure that the kinds of studies I am critical of really give you things you can use but thanks for these leads. I am not sure what statistical mechanics as an application means but will look further.
Here’s a (not terribly good) description of the Maximum Entropy approach to statistical mechanics: http://en.wikipedia.org/wiki/Maximum_entropy_thermodynamics
The “iterations” you’re referring to are Bayesian updating, the mathematical recipe for updating beliefs with new information.
The major philosophical difference between the Bayesian and classical frequentist approach is that Bayesians treat probability as a “degree of belief”, while frequentists restrict it to mean relative frequency. You can derive the frequentist results from a Bayesian framework, but not vice versa. Now, the classical statistics we all learned in school often represent the easiest way to solve certain problems where we have lots of data and rather little uncertainty, say a large well-randomized and highly controlled clinical trial. Here the frequentist approach provides nice approximate recipes for doing calculations, and the answers you obtain are essentially the same as what you would have gotten doing a full-blown (and far more difficult) Bayesian analysis.
But for observational studies, while we may have a lot of data, we also have large uncertainties around the sorts of questions we wish to ask of that data, basically a lot of “know what you don’t know”. Observational studies provide lots of correlations, for instance, but the question we want to ask is something like “does saturated fat cause heart disease?”, or more precisely “which is more likely to cause saturated fat or shaving?” We throw out the shaving hypothesis because our prior knowledge of biochemistry tells us it is highly unlikely that there is a causal link between shaving and heart disease.
A Bayesian analysis puts this sort of judgment call on quantitative footing, because one can include the prior probabilities (independent of the data under examination) that various hypotheses are true. It doesn’t save you from bad inputs (e.g. the likely over-weighting of the saturated fat hypothesis due to social pressure). But it does at least force you to quantify and make transparent these prior beliefs, and reason consistently using those inputs. And this then opens the door to asking questions of those prior beliefs, i.e. “can I deconstruct my prior about the causal role of saturated fat in heart disease into other hypotheses based on my knowledge of biochemistry?”
Thanks for a great post, this was needed! It would be interesting to read your take on the latest RCTs and observationals studies on n-3 fatty acids. Outcomes have often been modest or controversial in terms of diabetes or CV health., espsicially as you bear in mind how much omega-3 fats are praised.
[…] And Association does Imply Causality but… August 29, 2011By: rdfeinman Read the Full Post at: Richard David Feinman …the association has to be strong and the causality has to be plausible and consistent. And […]
The conservation of energy comes from Emmy Noethet’s mathematical theorums and has to do with symmetries found in Nature. However, it is only conserved locally. We already live in a Univers where energy is NOT conserved cosmically, as the expanding universe totally “violates” it.
Our science principles are not rules or mandates imposed on Nature.
Physics is not done by fiat and our laws are always open to being wrong or perfected.
All true but probably off to the side of the problem at hand.