Archive for June, 2011

No Sunshine Down Under: Comments on Larsen, et al.

Posted: June 28, 2011 in Association and Causality, Evidence Based Medicine, Intention to Treat, Lipophobes, low-carbohydrate diet, Observational Studies, The Nutrition Story
10

“These results suggest that there is no superior long-term metabolic benefit of a high-protein diet over a high-carbohydrate in the management of type 2 diabetes.”  The conclusion is from a paper by Larsen, et al. [1] which, based on that statement in the Abstract, I would not normally bother to read; it is good that you have to register trials and report failures but from a broader perspective, finding nothing is not great news and just because Larsen couldn’t do it, doesn’t mean it can’t be done.  However, in this case, I received an email from International Diabetes published bilingually in Beijing: “Each month we run a monthly column where choose a hot-topic article… and invite expert commentary opinion about that article” so I agreed to write an opinion. The following is my commentary:

“…no superior long-term metabolic benefit of a high-protein diet over a high-carbohydrate ….” A slightly more positive conclusion might have been that “a high-protein diet is as good as a high carbohydrate diet.”  After all, equal is equal. The article is, according to the authors, about “high-protein, low-carbohydrate” so rather than describing a comparison of apples and pears, the conclusion should emphasize low carbohydrate vs high carbohydrate.   It is carbohydrate, not protein, that is the key question in diabetes but clarity was probably not the idea. The paper by Larsen, et al. [1] represents a kind of classic example of the numerous studies in the literature whose goal is to discourage people with diabetes from trying a diet based on carbohydrate restriction, despite its intuitive sense (diabetes is a disease of carbohydrate intolerance) and despite its established efficacy and foundations in basic biochemistry.  The paper is characterized by blatant bias, poor experimental design and mind-numbing statistics rather than clear graphic presentation of the data. I usually try to take a collegial approach in these things but this article does have a unique and surprising feature, a “smoking gun” that suggests that the authors were actually aware of the correct way to perform the experiment or at least to report the data.

Right off, the title tells you that we are in trouble. “The effect of high-protein, low-carbohydrate diets in the treatment…” implying that all such diets are the same even though  there are several different versions, some of which (by virtue of better design) will turn out to have had much better performance than the diet studied here and, almost all of which are not “high protein.” Protein is one of the more stable features of most diets — the controls in this experiment, for example, did not substantially lower their protein even though advised to do so –and most low-carbohydrate diets advise only carbohydrate restriction.  While low-carbohydrate diets do not counsel against increased protein, they do not necessarily recommend it.  In practice, most carbohydrate-restricted diets are hypocaloric and the actual behavior of dieters shows that they generally do not add back either protein or fat, an observation first made by LaRosa in 1980.

Atkins-bashing is not as easy as it used to be when there was less data and one could run on “concerns.” As low-fat diets continue to fail at both long-term and short-term trials — think Women’s Health Initiative [2] — and carbohydrate restriction continues to show success and continues to bear out the predictions from the basic biochemistry of the insulin-glucose axis  [3], it becomes harder to find fault.  One strategy is to take advantage of the lack of formal definitions of low-carbohydrate diets to set up a straw man.  The trick is to test a moderately high carbohydrate diet and show that, on average, as here, there is no difference in hemoglobin A1c, triglycerides and total cholesterol, etc. when compared to a higher carbohydrate diet as control —  the implication is that in a draw, the higher carbohydrate diet wins.  So, Larsen’s low carbohydrate diet contains 40 % of energy as carbohydrate.  Now, none of the researchers who have demonstrated the potential of carbohydrate restriction would consider 40 % carbohydrate, as used in this study, to be a low-carbohydrate diet. In fact, 40 % is close to what the American population consumed before the epidemic of obesity and diabetes. Were we all on a low carbohydrate diet before Ancel Keys?

What happened?  As you might guess, there weren’t notable differences on most outcomes but like other such studies in the literature, the authors report only group statistics so you don’t really know who ate what and they use an intention-to-treat (ITT) analysis. According to ITT, a research report should include data from those subjects that dropped out of the study (here, about 19 % of each group). You read that correctly.  The idea is based on the assumption (insofar as it has any justification at all) that compliance is an inherent feature of the diet (“without carbs, I get very dizzy”) rather than a consequence of bias transmitted from the experimenter, or distance from the hospital, or any of a thousand other things.  While ITT has been defended vehemently, the practice is totally counter-intuitive, and has been strongly attacked on any number of grounds, the most important of which is that, in diet experiments, it makes the better diet look worse.  Whatever the case that can be made, however, there is no justification for reporting only intention-to-treat data, especially since, in this paper, the authors consider as one of the “strengths of the study … the measurement of dietary compliance.”

The reason that this is all more than technical statistical detail, is that the actual reported data show great variability (technically, the 95 % confidence intervals are large).  To most people, a diet experiment is supposed to give a prospective dieter information about outcome.  Most patients would like to know: if I stay on this diet, how will I do.  It is not hard to understand that if you don’t stay on the diet, you can’t expect good results.  Nobody knows what 81 % staying on the diet could mean.  In the same way, nobody loses an average amount of weight. If you look at  the spread in performance and in what was consumed by individuals on this diet, you can see that there is big individual variation Also, being “on a diet”, or being “assigned to a diet” is very different than actually carrying out dieting behavior, that is, eating a particular collection of food.  When there is wide variation, a person in the low-carb group may be eating more carbs than some person in the high-carb group.  It may be worth testing the effect of having the doctor tell you to eat fewer carbs, but if you are trying to lose weight, you want them to test the effect of actually eating fewer carbs.

When I review papers like this for a journal I insist that the authors present individual data in graphic form.  The question in low-carbohydrate diets is the effect of amount of carbohydrate consumed on the outcomes.  Making a good case to the reader involves showing individual data.  As a reviewer, I would have had the authors plot each individual’s consumption of carbohydrate vs for example, individual changes in triglyceride and especially HbA1c.  Both of these are expected to be dependent on carbohydrate consumption.  In fact, this is the single most common criticism I make as reviewer or that I made when I was co-editor-in chief at Nutrition and Metabolism.

So what is the big deal?  This is not the best presentation of the data and it is really hard to tell what the real effect of carbohydrate restriction is. Everybody makes mistakes and few of my own papers are without some fault or other. But there’s something else here.  In reading a paper like this, unless you suspect that something wasn’t done correctly, you don’t tend to read the Statistical analysis section of the Methods very carefully (computers have usually done most of the work).  In this paper, however, the following remarkable paragraph jumps out at you.  A real smoking gun:

  • “As this study involved changes to a number of dietary variables (i.e. intakes of calories, protein and carbohydrate), subsidiary correlation analyses were performed to identify whether study endpoints were a function of the change in specific dietary variables. The regression analysis was performed for the per protocol population after pooling data from both groups. “

What?  This is exactly what I would have told them to do.  (I’m trying to think back. I don’t think I reviewed this paper).  The authors actually must have plotted the true independent variable, dietary intake — carbohydrate, calories, etc. — against the outcomes, leaving out the people who dropped out of the study.  So what’s the answer?

  • “These tests were interpreted marginally as there was no formal adjustment of the overall type 1 error rate and the p values serve principally to generate hypotheses for validation in future studies.”

Huh?  They’re not going to tell us?  “Interpreted marginally?”  What the hell does that mean?  A type 1 error refers to a false positive, that is, they must have found a correlation between diet and outcome in distinction to what the conclusion of the paper is.  They “did not formally adjust for” the main conclusion?  And “p values serve principally to generate hypotheses?”  This is the catch-phrase that physicians are taught to dismiss experimental results that they don’t like.  Whether it means anything or not, in this case there was a hypothesis, stated right at the beginning of the paper in the Abstract: “…to determine whether high-protein diets are superior to high-carbohydrate diets for improving glycaemic control in individuals with type 2 diabetes.”

So somebody — presumably a reviewer — told them what to do but they buried the results.  My experience as an editor was, in fact, that there are people in nutrition who think that they are beyond peer review and I had had many fights with authors.  In this case, it looks like the actual outcome of the experiment may have actually been the opposite of what they say in the paper.  How can we find out?  Like most countries, Australia has what are called “sunshine laws,” that require government agencies to explain their actions.  There is a Australian Federal Freedom of Information Act (1992) and one for the the state of Victoria (1982). One of the authors is supported by NHMRC (National Health and Medical Research Council)  Fellowship so it may be they are obligated to share this marginal information with us.  Somebody should drop the government a line.

Bibliography

1. Larsen RN, Mann NJ, Maclean E, Shaw JE: The effect of high-protein, low-carbohydrate diets in the treatment of type 2 diabetes: a 12 month randomised controlled trial. Diabetologia 2011, 54(4):731-740.

2. Tinker LF, Bonds DE, Margolis KL, Manson JE, Howard BV, Larson J, Perri MG, Beresford SA, Robinson JG, Rodriguez B et al: Low-fat dietary pattern and risk of treated diabetes mellitus in postmenopausal women: the Women’s Health Initiative randomized controlled dietary modification trial. Arch Intern Med 2008, 168(14):1500-1511.

3. Volek JS, Phinney SD, Forsythe CE, Quann EE, Wood RJ, Puglisi MJ, Kraemer WJ, Bibus DM, Fernandez ML, Feinman RD: Carbohydrate Restriction has a More Favorable Impact on the Metabolic Syndrome than a Low Fat Diet. Lipids 2009, 44(4):297-309.

Nothing wrong with Observational Studies. And Association does Imply Causality but…

Posted: June 19, 2011 in Association and Causality, Evidence Based Medicine, Intention to Treat, low-carbohydrate diet, Observational Studies, thermodynamics
Tags: , ,
7

…the association has to be strong and the causality has to be plausible and consistent. And you have to have some reason to make the observation; you can’t look at everything.  And experimentally, observation may be all that you have — almost all of astronomy is observational.  Of course, the great deconstructions of crazy nutritional science — several from Mike Eades blog and Tom Naughton’s hysterically funny-but-true course in how to be a scientist —  are still right on but, strictly speaking, it is the faulty logic of the studies and the whacko observations that is the problem, not simply that they are observational.  It is the strength and reliability of the association that tells you whether causality is implied.  Reducing carbohydrates lowers triglycerides.  There is a causal link.  You have to be capable of the state of mind of the low-fat politburo not to see this (for example, Circulation, May 24, 2011; 123(20): 2292 – 2333).

It is frequently said that observational studies are only good for generating hypotheses but it is really the other way around.  All studies are generated by hypotheses.  As Einstein put it: your theory determines what you measure.  I ran my post on the red meat story passed April Smith  and her reaction was “why red meat? Why not pancakes” which is exactly right.  Any number of things can be observed. Once you pick, you have a hypothesis.

Where did the first law of thermodynamics come from?

Thermodynamics is an interesting case.  The history of the second law involves a complicated interplay of observation and theory.  The idea that there was an absolute limit to how efficient you could make a machine and by extension that all real processes were necessarily inefficient largely comes from the brain power of Carnot. He saw that you could not extract as work all of the heat you put into a machine. Clausius encapsulated it into the idea of the entropy as in my Youtube video.

©2004 Robin A. Feinman

 The origins of the first law, the conservation of energy, are a little stranger.  It turns out that it was described more than twenty years after the second law and it has been attributed to several people, for a while, to the German physicist von Helmholtz.  These days, credit is given to a somewhat eccentric German physician named Robert Julius Mayer. Although trained as a doctor, Mayer did not like to deal with patients and was instead more interested in physics and religion which he seemed to think were the same thing.  He took a job as a shipboard physician on an expedition to the South Seas since that would give him time to work on his main interests.  It was in Jakarta where, while treating an epidemic with the practice then of blood letting, that he noticed that the venous blood of the sailors was much brighter than when they were in colder climates as if “I had struck an artery.” He attributed this to a reduced need for the sailors to use oxygen for heat and from this observation, he somehow leapt to the grand principle of conservation of energy, that the total amount of heat and work and any other forms of energy does not change but can only be interconverted. It is still unknown what kind of connections in his brain led him to this conclusion.  The period (1848) corresponds to the point at which science separated from philosophy. Mayer seems to have had one foot in each world and described things in the following incomprehensible way:

  • If two bodies find themselves in a given difference, then they could remain  in a state of rest after the annihilation of [that] difference if the  forces that were communicated to them as a result of the leveling of  the difference could cease to exist; but if they are assumed to be indestructible,  then the still persisting forces, as causes of changes in relationship,  will again reestablish the original present difference.

(I have not looked for it but one can only imagine what the original German was like). Warmth Disperses and Time Passes. The History of Heat, Von Baeyer’s popular book on thermodynamics, describes the ups and downs of Mayer’s life, including the death of three of his children which, in combination with rejection of his ideas, led to hospitalization but ultimate recognition and knighthood.  Surely this was a great observational study although, as von Baeyer put it, it did require “the jumbled flashes of insight in that sweltering ship’s cabin on the other side of the world.”

It is also true that association does imply causation but, again, the association has to have some impact and the proposed causality has to make sense.  In some way, purely observational experiments are rare.  As Pasteur pointed out, even serendipity is favored by preparation.  Most observational experiments must be a reflection of some hypothesis. Otherwise you’re wasting tax-payer’s money; a kiss of death on a grant application is to imply that “it would be good to look at.…”  You always have to have something in mind.  The great observational studies like the Framingham Study are bad because they have no null hypothesis. When the Framingham study first showed that there was no association between dietary total and saturated fat or dietary cholesterol, the hypothesis was quickly defended. The investigators were so tied to a preconceived hypothesis, that there was hardly any point in making the observations.

In fact, a negative result is always stronger than one showing consistency; consistent sunrises will go by the wayside if the sun fails to come up once.  It is the lack of an association between the decrease in fat consumption during the epidemic of obesity and diabetes that is so striking.  The figure above shows that the  increase in carbohydrate consumption is consistent with the causal role of dietary carbohydrate in creating anabolic hormonal effects and with the poor satiating effects of carbohydrates — almost all of the increase of calories during the epidemic of obesity and diabetes has been due to carbohydrates.  However, this observation is not as strong as the lack of an identifiable association of obesity and diabetes with fat consumption.  It is the 14 % decrease in the absolute amount of saturated fat for men that is the problem.  If the decrease in fat were associated with decrease in obesity, diabetes and cardiovascular disease, there is little doubt that the USDA would be quick to identify causality.  In fact, whereas you can find the occasional low-fat trial that succeeds, if the diet-heart hypothesis were as described, they should not fail. There should not be a single Women’s Health Initiative, there should not be a single Framingham study, not one.

Sometimes more association would be better.  Take intention-to-treat. Please. In this strange statistical idea, if you assign a person to a particular intervention, diet or drug, then you must include the outcome data (weight loss, change in blood pressure) for that person even if the do not comply with the protocol (go off the diet, stop taking the pills).  Why would anybody propose such a thing, never mind actually insist on it as some medical journals or granting agencies do?  When you actually ask people who support ITT, you don’t get coherent answers.  They say that if you just look at per protocol data (only from people who stayed in the experiment), then by excluding the drop-outs, you would introduce bias but when you ask them to explain that you get something along the lines of Darwin and the peas growing on the wrong side of the pod. The basic idea, if there is one, is that the reason that people gave up on their diet or stopped taking the pills was because of an inherent feature of the intervention: made them sick, drowsy or something like that.  While this is one possible hypothesis and should be tested, there are millions of others — the doctor was subtly discouraging about the diet, or the participants were like some of my relatives who can’t remember where they put their pills, or the diet book was written in Russian, or the diet book was not written in Russian etc. I will discuss ITT in a future post but for the issue at hand:  if you do a per-protocol you will observe what happens to people when stay on their diet and you will have an association between the content of the diet and performance.  With an ITT analysis, you will be able to observe what happens when people are told to follow a diet and you will have an association between assignment to a diet and performance.  Both are observational experiments with an association between variables but they have different likelihood of providing a sense of causality.

Metabolic advantage, “a calorie is a calorie,” and why the First Law of Thermodynamics does not apply.

Posted: June 6, 2011 in low-carbohydrate diet, The Nutrition Story, thermodynamics
Tags: , , , ,
32


The big news in the low carb world is that Consumer Reports has published, for the first time, faint praise for the Atkins diet. However, the vision one might have of CR employees testing running shoes on treadmills doesn’t really apply here. They did not put anybody on a diet, even for a day. They didn’t have to. They have the standards from the government. Conform to the USDA Guidelines and CR will give you thumbs up. It probably doesn’t matter since, these days, most people buy a food processor by checking out the reviews on the internet — there are now many reviews online of what it’s like to actually be on a low-carbohydrate diet, so rather than follow CR’s imaginings of what it’s like, you can check out what users say — Jimmy Moore, Tom Naughton and Laura Dolson together get about 1.5 million posts per month with many tests and best buy recommendations. What caught my eye, though, is the ubiquitous Dean Ornish; the ratio of words written about the Ornish diet to the number of people who actually use it is probably closing in on a googol (as it was originally spelled). The article says: “to lose weight, you have to burn up more calories than you take in, no matter what kind of diet you’re on. ‘The first law of thermodynamics still applies,’ says Dean Ornish, M.D.

That’s how I got into this field. My colleague Gene Fine, and I published our first papers in nutrition on the subject of metabolic advantage and thermodynamics and we gave ourselves credit for reducing the number of people invoking laws of thermodynamics. “Metabolic advantage” refers to the idea that you can lose more weight, calorie-for-calorie on a particular diet, usually a low-carbohydrate diet. (The term was used in a paper by Browning to mean the benefits in lipid metabolism of a low-carbohydrate diet, but in nutrition you can re-define anything you want and you don’t have to cite anybody else’s work if you don’t want to). The idea of metabolic advantage stands in opposition to the idea that “a calorie is a calorie” which is, of course, the backbone of establishment nutrition and all our woe. As in the CR article, whenever the data show that a low-carbohydrate diet is more effective for weight loss, somebody always jumps in to say that it would violate the laws of thermodynamics. Those of us who have studied or use thermodynamics recognize that it is a rather difficult subject — somebody called it the physics of partial differential equations — and we’re amazed at how many experts have popped up in the nutrition field.

Finding the right diet doesn’t require knowing much thermodynamics but it is an interesting subject and so I’ll try to explain what it is about and how it’s used in biochemistry. The physics of heat, work and energy, thermodynamics was developed in the nineteenth century in the context of the industrial revolution — how efficiently you could make a steam engine operate was a big deal.  Described by Prigogine as the first revolutionary science, it has some interesting twists and intellectual connections. The key revolutionary concept is embodied tin the second law which describes the efficiency of physical processes.  It has broad philosophical meaning.  The primary concept, the entropy, is also used in communication and  the content of messages in information theory.  The entropy of a message is, in one context, how much a message has been garbled in transmission.  The history of thermodynamics also has some very strange characters, besides me and Gene, so I will try to describe them too.

First, we can settle the question of metabolic advantage, or more precisely, energy inefficiency. The question is whether all of the calories in food are available for weight gain or loss (or exercise) regardless of the composition of the diet. Right off, metabolic advantage is an inherent property of higher protein diets and low carbohydrate diets. In the first case, the thermic effect of feeding (TEF) is a measure of how many of the calories in food are wasted in the process of digestion, absorption, low-level chemical transformation, etc. TEF (old name: specific dynamic action) is well known and well studied. Nobody disputes that the TEF can be substantial for protein, typically 20 % of calories. It is much less for carbohydrate and still less for fat. So, substituting any protein for either of the other macronutrients will lead to energy inefficiency (the calories will be wasted as heat). A second unambiguous point is that in the case of low-carbohydrate diets, in order to maintain blood glucose, the process of gluconeogenesis is required. You learn in biochemistry courses that it requires a good deal of energy to convert protein (the major source for gluconeogenesis) into glucose.

So, right off, metabolic advantage or energy inefficiency is known and measurable. Critics of carb restriction as a strategy admit that it occurs but say that it is too small in a practical sense to be worth considering when you are trying to lose weight. These are usually the same people who tell you that the best way to lose weight is through accumulation of small changes in daily weight loss by reducing 100 kcal a day or something like that. In any case, there is a big difference between things that are not practical or have only small effects and things that are theoretically impossible. If metabolic advantage were really impossible theoretically, that would be it. We could stop looking for the best diet and only calories would count. Since we know energy inefficiency is possible and measurable, shouldn’t we be trying to maximize it.  But what is the story on thermodynamics? What is it? Why do people think that metabolic advantage violates thermodynamics? What is their mistake? More specifically, doesn’t the first law of thermodynamics say that calories are conserved? Well, there is more than one law of thermodynamics and even the first law has to be applied correctly. Let me explain. (Note in passing that the dietary calorie is a physical kilocalorie (kcal; 1000 calories).

There are four laws of thermodynamics. Two are technical. The zeroth law says, in essence, that if two bodies have the same temperature as a third, they have the same temperature as each other. This sounds obvious but, in fact, it is an observational law — it always turns out that way. The law is necessary to make sure everything else is for real. If anybody ever finds an experimental case where it is not true, the whole business will come crashing down. The third law describes what happens at the special condition known as the absolute zero of temperature. In essence, the zeroth and third laws, allow everything else to be calculated and practical thermodynamics like bioenergetics pretty much assumes it in the background.

The second law is what thermodynamics is really about — it was actually formulated before the first law — but since the first law is usually invoked in nutrition, let’s consider this first. The first law is the conservation of energy law. Here’s how it works: thermodynamics considers systems and surroundings. The thing that you are interested in — living system, a single cell, a machine, whatever, is called the system — everything outside is the surroundings or environment. The first law says that any energy lost by the system must be gained by the environment and any energy taken up by the system must have come from the environment. Its application to chemical systems, which is what applies to nutrition, is that we can attribute to chemical systems, a so-called internal energy, usually written with symbol U (so as not to confuse it with the electrical potential, E). In thermodynamics, you usually look at changes, and the first law says that you can calculate ΔU, the change in U of a system, by adding up the changes in heat added to the system and work done by the system (you can see the roots of thermo in heat machines: we add heat and get work). In chemical systems, the energy can also change due to chemical reactions. Still, if you add up all the changes in the system plus the environment, all the heat, work and chemical changes, the energy is neither created nor destroyed. It is conserved.

Now, why doesn’t the first law apply to nutrition the way Ornish thinks it does? To understand this, you have to know what is done in chemical thermodynamics and bioenergetics, (thermo applied to living systems). If you want to. In nutrition, you can make up your own stuff. But, if you want to do what is done in chemical thermodynamics, you focus on the system itself, not the system plus the environment. So, from the standpoint of chemical thermodynamics, the calories in food represent the heat generated by complete oxidation of food in a calorimeter.



In a calorimeter, the food is placed in a small container with oxygen under pressure and ignited. The heat generated is determined from the increase in temperature of the water bath. (Before the food measurement, we determine the heat capacity of the water bath, that is, how much heat it takes to raise the temperature). The heat is how we define the calories in the food. The box around the sample in the figure shows that we are measuring the heat produced by the system, not the system plus the environment, that is, not applying the first law. If you applied the first law, the calories associated with the food would be zero, because any heat lost in combustion of the food would show up in the water bath of the calorimeter. The calories per gram of carbohydrate would be 0 instead of 4, the calories per gram of fat would be 0 not 9, etc. So, in studying reactions in chemical thermodynamics, energy is not conserved, it is dissipated. When systems dissipate energy, the change is indicated with a minus sign, so for oxidation of food, generally: ΔU < 0. So, no, the first law does not apply. That’s one of the reasons that “a calorie is not a calorie.”
There is an additional point that we assumed in passing. In chemical thermodynamics, the energy goes with the reaction, not with the food. It is not like particle physics where we give the mass of a particle in electron-volts, a measure of energy, because of E=mc2. What this means, practically, is that the 4 kcal per gram of carbohydrate is for the reaction of complete oxidation. Do anything else, make DNA, make protein and all bets are off.
The bottom line is that, contrary to what is usually said, thermodynamics does not predict energy balance and we should not be surprised when one diet is more or less efficient than another. In fact, the question to be answered is why energy balance is ever found. “A calories is a calorie” is frequently what is observed (although there is always a question as to how we make the measurement). The answer is that insofar as there is energy balance, it is a question of the unique behavior of living systems, not physical laws. Two similar subjects of similar age and genetic make-up may, under the right conditions, respond to different diets so that most of what they do is oxidize food and the contributions of DNA or protein synthesis, growth, etc. may be similar and may cancel out so that the major contribution to energy exchange is the heat of combustion.
But thermodynamics is really not about the first law which, while its history is a little odd, it is not revolutionary. Intellectually, the first law is related to conservation of matter. Thermodynamics is about the second law. The second law says that there is a physical parameter, called the entropy, almost always written S, and the change in entropy, ΔS, in any real process, always increases. In ideal, theoretical processes, ΔS may be zero, but it never goes down. In other words, looking at the universe, (any system and its surroundings), energy is conserved but entropy increases. The first law is a conservation law but the second law is a dissipation law. We identify the entropy with the organization, order or information in a system. Systems proceed naturally to the most probable state. In one of the best popular introductions to the subjects, von Baeyer’s Warmth Disperses and Time Passes, entropy is described in terms of the evolution of the organization of his teenage daughter’s room.  To finish up on calorimeters, though, there is Lavoisier’s whole animal calorimeter.



One of Lavoisier’s great contributions was to show that combustion was due to a combination with oxygen rather than the release of a substance, then known as the phlogiston. Lavoisier had the insight that in an animal, the combination of oxygen with food to produce carbon dioxide was the same kind of process. The whole animal calorimeter was a clever way to show this. The animal is placed in the basket compartment f. The inner jacket, b, is packed with ice. The outer jacket, a, is also packed with ice to keep the inner jacket, cold. The heat generated by the animal melts the ice in the inner jacket which is collected in container, Fig 8. Lavoisier showed that the amount of carbon dioxide formed was proportional to the heat generated as it would be if an animal were carrying out the same chemical reactions that occur, for example, in burning of charcoal. “La vie est donc une combustion.” His collaborator in this experiment was the famous mathematician Laplace and people sometimes wonder how he got a serious mathematician like Laplace to work on what is, well, nutrition. It seems likely that it was because Laplace owed him a lot of money.

Evidence Based Medicine and Admissibility. IV. The Oslo Diet-Heart Study.

Posted: June 1, 2011 in Evidence Based Medicine, Lipophobes, The Nutrition Story
Tags: , , , , ,
15

“In the Viking era, they were already using skis…and over the centuries, the Norwegians have proved themselves good at little else.”

–John Cleese, Norway, Home of Giants.

With the 3-foot bookshelf of popular attacks on the low-fat-diet-heart idea it is pretty remarkable that there is only one defense.  Daniel Steinberg’s Cholesterol Wars. The Skeptics vs. The Preponderance of Evidence is probably more accurately called a witness for the prosecution since low-fat, in some way or other is still the law of the land.

The Skeptics vs. the Preponderance of Evidence

The Skeptics vs. the Preponderance of Evidence

The book is very informative, if biased, and it provides an historical perspective describing the difficulty of establishing the cholesterol hypothesis. Oddly, though,  it still appears to be very defensive for a witness for the prosecution.  In any case, Steinberg introduces into evidence the Oslo Diet-Heart Study [2] with a serious complaint:

“Here was a carefully conducted study reported in 1966 with a statistically significant reduction in reinfarction [recurrence of heart attack] rate.  Why did it not receive the attention it deserved?”

“The key element,” he says, “was a sharp reduction in saturated fat and cholesterol intake and an increase in polyunsaturated fat intake. In fact. each experimental subject had to consume a pint of soybean oil every week, adding it to salad dressing or using it in cooking or, if necessary, just gulping it down!”

Whatever it deserved, the Oslo Diet-Heart Study did receive a good deal of attention.  The Women’s Health Initiative (WHI), liked it.  The WHI was the most expensive failure to date. It found that “over a mean of 8.1 years, a dietary intervention that reduced total fat intake and increased intakes of vegetables, fruits, and grains did not significantly reduce the risk of CHD, stroke, or CVD in postmenopausal women.” [3]

The WHI, adopted a “win a few, lose a few” attitude, comparing its results to the literature, where some studies showed an effect of reducing dietary fat and some did not — this made me wonder: if the case is so clear, whey are there any failures.  Anyway, it cited the Oslo Diet-Heart Study as one of the winners and attributed the outcome to the substantial lowering of plasma cholesterol.

So, “cross-examination” would tell us why, if  “a statistically significant reduction in reinfarction  rate”  it did “not receive the attention it deserved?”

First, the effect of diet on cholesterol over five years:

The results look good although, since all the numbers are considered fairly high, and since the range of values is not shown, it is hard to tell just how impressive the results really are. But we will stipulate that you can lower cholesterol on a low-fat diet. But what about the payoff? What about the outcomes?

The results are shown in Table 5 of the original paper:   Steinberg described how in the first 5 years: “58 patients of the 206 in the control group (28%) had a second heart attack” (first 3 lines under first line of blue-highlighting) but only

“…  32 of the 206 in the diet (16%)…”  which does sound pretty good.

In the end, though, it’s really the total deaths from cardiac disease.  The second blue-highlighted line in Table 5 shows the two final outcome.  How should we compare these.

1. The odds ratio or relative risk is just the ratio of the two outcomes (since there are the same number of subjects) = CHD mortality (diet)/ CHD mortality control) = 94/79 =  1.19.  This seems strikingly close to 1.0, that is, flip of a coin.  These days the media, or the report itself, would report that there was a 19 % reduction in total CHD mortality.

2, If you look at the absolute values, however, the  mortality in the controls is 94/206 = 45.6 % but the diet group had reduced this  to 79/206 = 38.3 % so the change in absolute risk is  45.6 % – 38.3 % or only 7.3 % which is less impressive but still not too bad.

3. So for every 206 people, we save 94-79 = 15 lives, or dividing 206/15 = 14 people needed to treat to save one life. (Usually abbreviated NNT). That doesn’t sound too bad.  Not penicillin but could be beneficial. I think…

Smoke and mirrors.

It’s what comes next that is so distressing.  Table 10 pools the two groups, the diet and the control group and now compares  the effect of smoking: on the whole population,  the ratio of CHD deaths in smokers vs non-smokers is 119/54 = 2.2 (magenta highlight) which is somewhat more impressive than the 1.19 effect we just saw.  Now,

1. The absolute difference in risk is (119-54)/206 = 31.6 % which sounds like a meaningful number.

2. The number needed to treat is 206/64 = 3.17  or only about 3 people need to quit smoking to see one less death

In fact, in some sense, the Oslo Diet-Heart Study provides smoking-CHD risk as an example of a meaningful association that one can take seriously. If only such a significant change had actually been found for the diet effect.

So what do the authors make of this? Their conclusion is that “When combining data from both groups, a three-fold greater CHD mortality rate is demonstrable among the hypercholesterolemic, hypertensive smokers than among those in whom these factors were low or absent.”  Clever but sneaky. The “hypercholesterolemic, hypertensive” part is irrelevant since you combined the groups. In other words, what started out as a diet study has become a “lifestyle study.”  They might has well have said “When combining data from fish and birds a significant number of wings were evident.” Members of the jury are shaking their heads.

Logistic regression. What is it? Can it help?

So they have mixed up smoking and diet. Isn’t there a way to tell which was more important?  Well, of course, there are several ways.  By coincidence, while I was writing this post, April Smith posted on facebook, the following challenge “The first person to explain logistic regression to me wins admission to SUNY Downstate Medical School!” I won although I am already at Downstate.  Logistic regression is, in fact, a statistical method that asks what the relative contribution of different inputs would have to be to fit the outcome and this could have been done but in this case, I would use my favorite statistical method, the Eyeball Test.  Looking at the data in Tables 5 and 10 for CHD deaths, you can see immediately what’s going on. Smoking is a bigger risk than diet.

If you really want a number, we calculated relative risk above. Again, we found for mortality, CHD (diet)/ CHD (control) = 94/79 =  1.19. But what happens if you took up smoking: Figure 10 shows that your chance of dying of heart disease would be increased by 119/54 = 2.2  or more than twice the risk.  Bottom line: you decided to add saturated fat to your diet, your risk would be 1.19 what it was before which might be a chance you could take faced with authentic Foie Gras.

Daniel Steinberg’s question:

“Here was a carefully conducted study reported in 1966 with a statistically significant reduction in reinfarction  rate.  Why did it not receive the attention it deserved?”

Well, it did. This is not the first critique.  Uffe Ravnskov described how the confusion of smoking and diet led to a new Oslo Trial which reductions in both were specifically recommended and, again, outcomes made diet look bad [4].  Ravnskov gave it the attention it deserved. But what about researchers writing in the scientific literature. Why do they not give the study the attention it deserves. Why do they not point out its status as a classic case of a saturated fat risk study with no null hypothesis.  It certainly deserves attention for its devious style. Of course, putting that in print would guarantee that your grant is never funded and your papers will be hard to publish.  So, why do researchers not give the Oslo-Diet-Heart study the attention it deserves?  Good question, Dan.

Bibliography

1. Steinberg D: The cholesterol wars : the skeptics vs. the preponderance of evidence, 1st edn. San Diego, Calif.: Academic Press; 2007.

2. Leren P: The Oslo diet-heart study. Eleven-year report. Circulation 1970, 42(5):935-942.

3. Howard BV, Van Horn L, Hsia J, Manson JE, Stefanick ML, Wassertheil-Smoller S, Kuller LH, LaCroix AZ, Langer RD, Lasser NL et al: Low-fat dietary pattern and risk of cardiovascular disease: the Women’s Health Initiative Randomized Controlled Dietary Modification Trial. JAMA 2006, 295(6):655-666.

4. Ravnskov U: The Cholesterol Myths: Exposing the Fallacy that Cholesterol and Saturated Fat Cause Heart Disease. Washington, DC: NewTrends Publishing, Inc.; 2000.