…the association has to be strong and the causality has to be plausible and consistent. And you have to have some reason to make the observation; you can’t look at everything. And experimentally, observation may be all that you have — almost all of astronomy is observational. Of course, the great deconstructions of crazy nutritional science — several from Mike Eades blog and Tom Naughton’s hysterically funny-but-true course in how to be a scientist — are still right on but, strictly speaking, it is the faulty logic of the studies and the whacko observations that is the problem, not simply that they are observational. It is the strength and reliability of the association that tells you whether causality is implied. Reducing carbohydrates lowers triglycerides. There is a causal link. You have to be capable of the state of mind of the low-fat politburo not to see this (for example, Circulation, May 24, 2011; 123(20): 2292 – 2333).
It is frequently said that observational studies are only good for generating hypotheses but it is really the other way around. All studies are generated by hypotheses. As Einstein put it: your theory determines what you measure. I ran my post on the red meat story passed April Smith and her reaction was “why red meat? Why not pancakes” which is exactly right. Any number of things can be observed. Once you pick, you have a hypothesis.
Where did the first law of thermodynamics come from?
Thermodynamics is an interesting case. The history of the second law involves a complicated interplay of observation and theory. The idea that there was an absolute limit to how efficient you could make a machine and by extension that all real processes were necessarily inefficient largely comes from the brain power of Carnot. He saw that you could not extract as work all of the heat you put into a machine. Clausius encapsulated it into the idea of the entropy as in my Youtube video.
The origins of the first law, the conservation of energy, are a little stranger. It turns out that it was described more than twenty years after the second law and it has been attributed to several people, for a while, to the German physicist von Helmholtz. These days, credit is given to a somewhat eccentric German physician named Robert Julius Mayer. Although trained as a doctor, Mayer did not like to deal with patients and was instead more interested in physics and religion which he seemed to think were the same thing. He took a job as a shipboard physician on an expedition to the South Seas since that would give him time to work on his main interests. It was in Jakarta where, while treating an epidemic with the practice then of blood letting, that he noticed that the venous blood of the sailors was much brighter than when they were in colder climates as if “I had struck an artery.” He attributed this to a reduced need for the sailors to use oxygen for heat and from this observation, he somehow leapt to the grand principle of conservation of energy, that the total amount of heat and work and any other forms of energy does not change but can only be interconverted. It is still unknown what kind of connections in his brain led him to this conclusion. The period (1848) corresponds to the point at which science separated from philosophy. Mayer seems to have had one foot in each world and described things in the following incomprehensible way:
- If two bodies find themselves in a given difference, then they could remain in a state of rest after the annihilation of [that] difference if the forces that were communicated to them as a result of the leveling of the difference could cease to exist; but if they are assumed to be indestructible, then the still persisting forces, as causes of changes in relationship, will again reestablish the original present difference.
(I have not looked for it but one can only imagine what the original German was like). Warmth Disperses and Time Passes. The History of Heat, Von Baeyer’s popular book on thermodynamics, describes the ups and downs of Mayer’s life, including the death of three of his children which, in combination with rejection of his ideas, led to hospitalization but ultimate recognition and knighthood. Surely this was a great observational study although, as von Baeyer put it, it did require “the jumbled flashes of insight in that sweltering ship’s cabin on the other side of the world.”
It is also true that association does imply causation but, again, the association has to have some impact and the proposed causality has to make sense. In some way, purely observational experiments are rare. As Pasteur pointed out, even serendipity is favored by preparation. Most observational experiments must be a reflection of some hypothesis. Otherwise you’re wasting tax-payer’s money; a kiss of death on a grant application is to imply that “it would be good to look at.…” You always have to have something in mind. The great observational studies like the Framingham Study are bad because they have no null hypothesis. When the Framingham study first showed that there was no association between dietary total and saturated fat or dietary cholesterol, the hypothesis was quickly defended. The investigators were so tied to a preconceived hypothesis, that there was hardly any point in making the observations.
In fact, a negative result is always stronger than one showing consistency; consistent sunrises will go by the wayside if the sun fails to come up once. It is the lack of an association between the decrease in fat consumption during the epidemic of obesity and diabetes that is so striking. The figure above shows that the increase in carbohydrate consumption is consistent with the causal role of dietary carbohydrate in creating anabolic hormonal effects and with the poor satiating effects of carbohydrates — almost all of the increase of calories during the epidemic of obesity and diabetes has been due to carbohydrates. However, this observation is not as strong as the lack of an identifiable association of obesity and diabetes with fat consumption. It is the 14 % decrease in the absolute amount of saturated fat for men that is the problem. If the decrease in fat were associated with decrease in obesity, diabetes and cardiovascular disease, there is little doubt that the USDA would be quick to identify causality. In fact, whereas you can find the occasional low-fat trial that succeeds, if the diet-heart hypothesis were as described, they should not fail. There should not be a single Women’s Health Initiative, there should not be a single Framingham study, not one.
Sometimes more association would be better. Take intention-to-treat. Please. In this strange statistical idea, if you assign a person to a particular intervention, diet or drug, then you must include the outcome data (weight loss, change in blood pressure) for that person even if the do not comply with the protocol (go off the diet, stop taking the pills). Why would anybody propose such a thing, never mind actually insist on it as some medical journals or granting agencies do? When you actually ask people who support ITT, you don’t get coherent answers. They say that if you just look at per protocol data (only from people who stayed in the experiment), then by excluding the drop-outs, you would introduce bias but when you ask them to explain that you get something along the lines of Darwin and the peas growing on the wrong side of the pod. The basic idea, if there is one, is that the reason that people gave up on their diet or stopped taking the pills was because of an inherent feature of the intervention: made them sick, drowsy or something like that. While this is one possible hypothesis and should be tested, there are millions of others — the doctor was subtly discouraging about the diet, or the participants were like some of my relatives who can’t remember where they put their pills, or the diet book was written in Russian, or the diet book was not written in Russian etc. I will discuss ITT in a future post but for the issue at hand: if you do a per-protocol you will observe what happens to people when stay on their diet and you will have an association between the content of the diet and performance. With an ITT analysis, you will be able to observe what happens when people are told to follow a diet and you will have an association between assignment to a diet and performance. Both are observational experiments with an association between variables but they have different likelihood of providing a sense of causality.