“These results suggest that there is no superior long-term metabolic benefit of a high-protein diet over a high-carbohydrate in the management of type 2 diabetes.”  The conclusion is from a paper by Larsen, et al. [1] which, based on that statement in the Abstract, I would not normally bother to read; it is good that you have to register trials and report failures but from a broader perspective, finding nothing is not great news and just because Larsen couldn’t do it, doesn’t mean it can’t be done.  However, in this case, I received an email from International Diabetes published bilingually in Beijing: “Each month we run a monthly column where choose a hot-topic article… and invite expert commentary opinion about that article” so I agreed to write an opinion. The following is my commentary:

“…no superior long-term metabolic benefit of a high-protein diet over a high-carbohydrate ….” A slightly more positive conclusion might have been that “a high-protein diet is as good as a high carbohydrate diet.”  After all, equal is equal. The article is, according to the authors, about “high-protein, low-carbohydrate” so rather than describing a comparison of apples and pears, the conclusion should emphasize low carbohydrate vs high carbohydrate.   It is carbohydrate, not protein, that is the key question in diabetes but clarity was probably not the idea. The paper by Larsen, et al. [1] represents a kind of classic example of the numerous studies in the literature whose goal is to discourage people with diabetes from trying a diet based on carbohydrate restriction, despite its intuitive sense (diabetes is a disease of carbohydrate intolerance) and despite its established efficacy and foundations in basic biochemistry.  The paper is characterized by blatant bias, poor experimental design and mind-numbing statistics rather than clear graphic presentation of the data. I usually try to take a collegial approach in these things but this article does have a unique and surprising feature, a “smoking gun” that suggests that the authors were actually aware of the correct way to perform the experiment or at least to report the data.

Right off, the title tells you that we are in trouble. “The effect of high-protein, low-carbohydrate diets in the treatment…” implying that all such diets are the same even though  there are several different versions, some of which (by virtue of better design) will turn out to have had much better performance than the diet studied here and, almost all of which are not “high protein.” Protein is one of the more stable features of most diets — the controls in this experiment, for example, did not substantially lower their protein even though advised to do so –and most low-carbohydrate diets advise only carbohydrate restriction.  While low-carbohydrate diets do not counsel against increased protein, they do not necessarily recommend it.  In practice, most carbohydrate-restricted diets are hypocaloric and the actual behavior of dieters shows that they generally do not add back either protein or fat, an observation first made by LaRosa in 1980.

Atkins-bashing is not as easy as it used to be when there was less data and one could run on “concerns.” As low-fat diets continue to fail at both long-term and short-term trials — think Women’s Health Initiative [2] — and carbohydrate restriction continues to show success and continues to bear out the predictions from the basic biochemistry of the insulin-glucose axis  [3], it becomes harder to find fault.  One strategy is to take advantage of the lack of formal definitions of low-carbohydrate diets to set up a straw man.  The trick is to test a moderately high carbohydrate diet and show that, on average, as here, there is no difference in hemoglobin A1c, triglycerides and total cholesterol, etc. when compared to a higher carbohydrate diet as control —  the implication is that in a draw, the higher carbohydrate diet wins.  So, Larsen’s low carbohydrate diet contains 40 % of energy as carbohydrate.  Now, none of the researchers who have demonstrated the potential of carbohydrate restriction would consider 40 % carbohydrate, as used in this study, to be a low-carbohydrate diet. In fact, 40 % is close to what the American population consumed before the epidemic of obesity and diabetes. Were we all on a low carbohydrate diet before Ancel Keys?

What happened?  As you might guess, there weren’t notable differences on most outcomes but like other such studies in the literature, the authors report only group statistics so you don’t really know who ate what and they use an intention-to-treat (ITT) analysis. According to ITT, a research report should include data from those subjects that dropped out of the study (here, about 19 % of each group). You read that correctly.  The idea is based on the assumption (insofar as it has any justification at all) that compliance is an inherent feature of the diet (“without carbs, I get very dizzy”) rather than a consequence of bias transmitted from the experimenter, or distance from the hospital, or any of a thousand other things.  While ITT has been defended vehemently, the practice is totally counter-intuitive, and has been strongly attacked on any number of grounds, the most important of which is that, in diet experiments, it makes the better diet look worse.  Whatever the case that can be made, however, there is no justification for reporting only intention-to-treat data, especially since, in this paper, the authors consider as one of the “strengths of the study … the measurement of dietary compliance.”

The reason that this is all more than technical statistical detail, is that the actual reported data show great variability (technically, the 95 % confidence intervals are large).  To most people, a diet experiment is supposed to give a prospective dieter information about outcome.  Most patients would like to know: if I stay on this diet, how will I do.  It is not hard to understand that if you don’t stay on the diet, you can’t expect good results.  Nobody knows what 81 % staying on the diet could mean.  In the same way, nobody loses an average amount of weight. If you look at  the spread in performance and in what was consumed by individuals on this diet, you can see that there is big individual variation Also, being “on a diet”, or being “assigned to a diet” is very different than actually carrying out dieting behavior, that is, eating a particular collection of food.  When there is wide variation, a person in the low-carb group may be eating more carbs than some person in the high-carb group.  It may be worth testing the effect of having the doctor tell you to eat fewer carbs, but if you are trying to lose weight, you want them to test the effect of actually eating fewer carbs.

When I review papers like this for a journal I insist that the authors present individual data in graphic form.  The question in low-carbohydrate diets is the effect of amount of carbohydrate consumed on the outcomes.  Making a good case to the reader involves showing individual data.  As a reviewer, I would have had the authors plot each individual’s consumption of carbohydrate vs for example, individual changes in triglyceride and especially HbA1c.  Both of these are expected to be dependent on carbohydrate consumption.  In fact, this is the single most common criticism I make as reviewer or that I made when I was co-editor-in chief at Nutrition and Metabolism.

So what is the big deal?  This is not the best presentation of the data and it is really hard to tell what the real effect of carbohydrate restriction is. Everybody makes mistakes and few of my own papers are without some fault or other. But there’s something else here.  In reading a paper like this, unless you suspect that something wasn’t done correctly, you don’t tend to read the Statistical analysis section of the Methods very carefully (computers have usually done most of the work).  In this paper, however, the following remarkable paragraph jumps out at you.  A real smoking gun:

  • “As this study involved changes to a number of dietary variables (i.e. intakes of calories, protein and carbohydrate), subsidiary correlation analyses were performed to identify whether study endpoints were a function of the change in specific dietary variables. The regression analysis was performed for the per protocol population after pooling data from both groups. “

What?  This is exactly what I would have told them to do.  (I’m trying to think back. I don’t think I reviewed this paper).  The authors actually must have plotted the true independent variable, dietary intake — carbohydrate, calories, etc. — against the outcomes, leaving out the people who dropped out of the study.  So what’s the answer?

  • “These tests were interpreted marginally as there was no formal adjustment of the overall type 1 error rate and the p values serve principally to generate hypotheses for validation in future studies.”

Huh?  They’re not going to tell us?  “Interpreted marginally?”  What the hell does that mean?  A type 1 error refers to a false positive, that is, they must have found a correlation between diet and outcome in distinction to what the conclusion of the paper is.  They “did not formally adjust for” the main conclusion?  And “p values serve principally to generate hypotheses?”  This is the catch-phrase that physicians are taught to dismiss experimental results that they don’t like.  Whether it means anything or not, in this case there was a hypothesis, stated right at the beginning of the paper in the Abstract: “…to determine whether high-protein diets are superior to high-carbohydrate diets for improving glycaemic control in individuals with type 2 diabetes.”

So somebody — presumably a reviewer — told them what to do but they buried the results.  My experience as an editor was, in fact, that there are people in nutrition who think that they are beyond peer review and I had had many fights with authors.  In this case, it looks like the actual outcome of the experiment may have actually been the opposite of what they say in the paper.  How can we find out?  Like most countries, Australia has what are called “sunshine laws,” that require government agencies to explain their actions.  There is a Australian Federal Freedom of Information Act (1992) and one for the the state of Victoria (1982). One of the authors is supported by NHMRC (National Health and Medical Research Council)  Fellowship so it may be they are obligated to share this marginal information with us.  Somebody should drop the government a line.

Bibliography

1. Larsen RN, Mann NJ, Maclean E, Shaw JE: The effect of high-protein, low-carbohydrate diets in the treatment of type 2 diabetes: a 12 month randomised controlled trial. Diabetologia 2011, 54(4):731-740.

2. Tinker LF, Bonds DE, Margolis KL, Manson JE, Howard BV, Larson J, Perri MG, Beresford SA, Robinson JG, Rodriguez B et al: Low-fat dietary pattern and risk of treated diabetes mellitus in postmenopausal women: the Women’s Health Initiative randomized controlled dietary modification trial. Arch Intern Med 2008, 168(14):1500-1511.

3. Volek JS, Phinney SD, Forsythe CE, Quann EE, Wood RJ, Puglisi MJ, Kraemer WJ, Bibus DM, Fernandez ML, Feinman RD: Carbohydrate Restriction has a More Favorable Impact on the Metabolic Syndrome than a Low Fat Diet. Lipids 2009, 44(4):297-309.

…the association has to be strong and the causality has to be plausible and consistent. And you have to have some reason to make the observation; you can’t look at everything.  And experimentally, observation may be all that you have — almost all of astronomy is observational.  Of course, the great deconstructions of crazy nutritional science — several from Mike Eades blog and Tom Naughton’s hysterically funny-but-true course in how to be a scientist —  are still right on but, strictly speaking, it is the faulty logic of the studies and the whacko observations that is the problem, not simply that they are observational.  It is the strength and reliability of the association that tells you whether causality is implied.  Reducing carbohydrates lowers triglycerides.  There is a causal link.  You have to be capable of the state of mind of the low-fat politburo not to see this (for example, Circulation, May 24, 2011; 123(20): 2292 – 2333).

It is frequently said that observational studies are only good for generating hypotheses but it is really the other way around.  All studies are generated by hypotheses.  As Einstein put it: your theory determines what you measure.  I ran my post on the red meat story passed April Smith  and her reaction was “why red meat? Why not pancakes” which is exactly right.  Any number of things can be observed. Once you pick, you have a hypothesis.

Where did the first law of thermodynamics come from?

Thermodynamics is an interesting case.  The history of the second law involves a complicated interplay of observation and theory.  The idea that there was an absolute limit to how efficient you could make a machine and by extension that all real processes were necessarily inefficient largely comes from the brain power of Carnot. He saw that you could not extract as work all of the heat you put into a machine. Clausius encapsulated it into the idea of the entropy as in my Youtube video.

©2004 Robin A. Feinman

 The origins of the first law, the conservation of energy, are a little stranger.  It turns out that it was described more than twenty years after the second law and it has been attributed to several people, for a while, to the German physicist von Helmholtz.  These days, credit is given to a somewhat eccentric German physician named Robert Julius Mayer. Although trained as a doctor, Mayer did not like to deal with patients and was instead more interested in physics and religion which he seemed to think were the same thing.  He took a job as a shipboard physician on an expedition to the South Seas since that would give him time to work on his main interests.  It was in Jakarta where, while treating an epidemic with the practice then of blood letting, that he noticed that the venous blood of the sailors was much brighter than when they were in colder climates as if “I had struck an artery.” He attributed this to a reduced need for the sailors to use oxygen for heat and from this observation, he somehow leapt to the grand principle of conservation of energy, that the total amount of heat and work and any other forms of energy does not change but can only be interconverted. It is still unknown what kind of connections in his brain led him to this conclusion.  The period (1848) corresponds to the point at which science separated from philosophy. Mayer seems to have had one foot in each world and described things in the following incomprehensible way:

  • If two bodies find themselves in a given difference, then they could remain  in a state of rest after the annihilation of [that] difference if the  forces that were communicated to them as a result of the leveling of  the difference could cease to exist; but if they are assumed to be indestructible,  then the still persisting forces, as causes of changes in relationship,  will again reestablish the original present difference.

(I have not looked for it but one can only imagine what the original German was like). Warmth Disperses and Time Passes. The History of Heat, Von Baeyer’s popular book on thermodynamics, describes the ups and downs of Mayer’s life, including the death of three of his children which, in combination with rejection of his ideas, led to hospitalization but ultimate recognition and knighthood.  Surely this was a great observational study although, as von Baeyer put it, it did require “the jumbled flashes of insight in that sweltering ship’s cabin on the other side of the world.”

It is also true that association does imply causation but, again, the association has to have some impact and the proposed causality has to make sense.  In some way, purely observational experiments are rare.  As Pasteur pointed out, even serendipity is favored by preparation.  Most observational experiments must be a reflection of some hypothesis. Otherwise you’re wasting tax-payer’s money; a kiss of death on a grant application is to imply that “it would be good to look at.…”  You always have to have something in mind.  The great observational studies like the Framingham Study are bad because they have no null hypothesis. When the Framingham study first showed that there was no association between dietary total and saturated fat or dietary cholesterol, the hypothesis was quickly defended. The investigators were so tied to a preconceived hypothesis, that there was hardly any point in making the observations.

In fact, a negative result is always stronger than one showing consistency; consistent sunrises will go by the wayside if the sun fails to come up once.  It is the lack of an association between the decrease in fat consumption during the epidemic of obesity and diabetes that is so striking.  The figure above shows that the  increase in carbohydrate consumption is consistent with the causal role of dietary carbohydrate in creating anabolic hormonal effects and with the poor satiating effects of carbohydrates — almost all of the increase of calories during the epidemic of obesity and diabetes has been due to carbohydrates.  However, this observation is not as strong as the lack of an identifiable association of obesity and diabetes with fat consumption.  It is the 14 % decrease in the absolute amount of saturated fat for men that is the problem.  If the decrease in fat were associated with decrease in obesity, diabetes and cardiovascular disease, there is little doubt that the USDA would be quick to identify causality.  In fact, whereas you can find the occasional low-fat trial that succeeds, if the diet-heart hypothesis were as described, they should not fail. There should not be a single Women’s Health Initiative, there should not be a single Framingham study, not one.

Sometimes more association would be better.  Take intention-to-treat. Please. In this strange statistical idea, if you assign a person to a particular intervention, diet or drug, then you must include the outcome data (weight loss, change in blood pressure) for that person even if the do not comply with the protocol (go off the diet, stop taking the pills).  Why would anybody propose such a thing, never mind actually insist on it as some medical journals or granting agencies do?  When you actually ask people who support ITT, you don’t get coherent answers.  They say that if you just look at per protocol data (only from people who stayed in the experiment), then by excluding the drop-outs, you would introduce bias but when you ask them to explain that you get something along the lines of Darwin and the peas growing on the wrong side of the pod. The basic idea, if there is one, is that the reason that people gave up on their diet or stopped taking the pills was because of an inherent feature of the intervention: made them sick, drowsy or something like that.  While this is one possible hypothesis and should be tested, there are millions of others — the doctor was subtly discouraging about the diet, or the participants were like some of my relatives who can’t remember where they put their pills, or the diet book was written in Russian, or the diet book was not written in Russian etc. I will discuss ITT in a future post but for the issue at hand:  if you do a per-protocol you will observe what happens to people when stay on their diet and you will have an association between the content of the diet and performance.  With an ITT analysis, you will be able to observe what happens when people are told to follow a diet and you will have an association between assignment to a diet and performance.  Both are observational experiments with an association between variables but they have different likelihood of providing a sense of causality.


The big news in the low carb world is that Consumer Reports has published, for the first time, faint praise for the Atkins diet. However, the vision one might have of CR employees testing running shoes on treadmills doesn’t really apply here. They did not put anybody on a diet, even for a day. They didn’t have to. They have the standards from the government. Conform to the USDA Guidelines and CR will give you thumbs up. It probably doesn’t matter since, these days, most people buy a food processor by checking out the reviews on the internet — there are now many reviews online of what it’s like to actually be on a low-carbohydrate diet, so rather than follow CR’s imaginings of what it’s like, you can check out what users say — Jimmy Moore, Tom Naughton and Laura Dolson together get about 1.5 million posts per month with many tests and best buy recommendations. What caught my eye, though, is the ubiquitous Dean Ornish; the ratio of words written about the Ornish diet to the number of people who actually use it is probably closing in on a googol (as it was originally spelled). The article says: “to lose weight, you have to burn up more calories than you take in, no matter what kind of diet you’re on. ‘The first law of thermodynamics still applies,’ says Dean Ornish, M.D.

That’s how I got into this field. My colleague Gene Fine, and I published our first papers in nutrition on the subject of metabolic advantage and thermodynamics and we gave ourselves credit for reducing the number of people invoking laws of thermodynamics. “Metabolic advantage” refers to the idea that you can lose more weight, calorie-for-calorie on a particular diet, usually a low-carbohydrate diet. (The term was used in a paper by Browning to mean the benefits in lipid metabolism of a low-carbohydrate diet, but in nutrition you can re-define anything you want and you don’t have to cite anybody else’s work if you don’t want to). The idea of metabolic advantage stands in opposition to the idea that “a calorie is a calorie” which is, of course, the backbone of establishment nutrition and all our woe. As in the CR article, whenever the data show that a low-carbohydrate diet is more effective for weight loss, somebody always jumps in to say that it would violate the laws of thermodynamics. Those of us who have studied or use thermodynamics recognize that it is a rather difficult subject — somebody called it the physics of partial differential equations — and we’re amazed at how many experts have popped up in the nutrition field.

Finding the right diet doesn’t require knowing much thermodynamics but it is an interesting subject and so I’ll try to explain what it is about and how it’s used in biochemistry. The physics of heat, work and energy, thermodynamics was developed in the nineteenth century in the context of the industrial revolution — how efficiently you could make a steam engine operate was a big deal.  Described by Prigogine as the first revolutionary science, it has some interesting twists and intellectual connections. The key revolutionary concept is embodied tin the second law which describes the efficiency of physical processes.  It has broad philosophical meaning.  The primary concept, the entropy, is also used in communication and  the content of messages in information theory.  The entropy of a message is, in one context, how much a message has been garbled in transmission.  The history of thermodynamics also has some very strange characters, besides me and Gene, so I will try to describe them too.

First, we can settle the question of metabolic advantage, or more precisely, energy inefficiency. The question is whether all of the calories in food are available for weight gain or loss (or exercise) regardless of the composition of the diet. Right off, metabolic advantage is an inherent property of higher protein diets and low carbohydrate diets. In the first case, the thermic effect of feeding (TEF) is a measure of how many of the calories in food are wasted in the process of digestion, absorption, low-level chemical transformation, etc. TEF (old name: specific dynamic action) is well known and well studied. Nobody disputes that the TEF can be substantial for protein, typically 20 % of calories. It is much less for carbohydrate and still less for fat. So, substituting any protein for either of the other macronutrients will lead to energy inefficiency (the calories will be wasted as heat). A second unambiguous point is that in the case of low-carbohydrate diets, in order to maintain blood glucose, the process of gluconeogenesis is required. You learn in biochemistry courses that it requires a good deal of energy to convert protein (the major source for gluconeogenesis) into glucose.

So, right off, metabolic advantage or energy inefficiency is known and measurable. Critics of carb restriction as a strategy admit that it occurs but say that it is too small in a practical sense to be worth considering when you are trying to lose weight. These are usually the same people who tell you that the best way to lose weight is through accumulation of small changes in daily weight loss by reducing 100 kcal a day or something like that. In any case, there is a big difference between things that are not practical or have only small effects and things that are theoretically impossible. If metabolic advantage were really impossible theoretically, that would be it. We could stop looking for the best diet and only calories would count. Since we know energy inefficiency is possible and measurable, shouldn’t we be trying to maximize it.  But what is the story on thermodynamics? What is it? Why do people think that metabolic advantage violates thermodynamics? What is their mistake? More specifically, doesn’t the first law of thermodynamics say that calories are conserved? Well, there is more than one law of thermodynamics and even the first law has to be applied correctly. Let me explain. (Note in passing that the dietary calorie is a physical kilocalorie (kcal; 1000 calories).

There are four laws of thermodynamics. Two are technical. The zeroth law says, in essence, that if two bodies have the same temperature as a third, they have the same temperature as each other. This sounds obvious but, in fact, it is an observational law — it always turns out that way. The law is necessary to make sure everything else is for real. If anybody ever finds an experimental case where it is not true, the whole business will come crashing down. The third law describes what happens at the special condition known as the absolute zero of temperature. In essence, the zeroth and third laws, allow everything else to be calculated and practical thermodynamics like bioenergetics pretty much assumes it in the background.

The second law is what thermodynamics is really about — it was actually formulated before the first law — but since the first law is usually invoked in nutrition, let’s consider this first. The first law is the conservation of energy law. Here’s how it works: thermodynamics considers systems and surroundings. The thing that you are interested in — living system, a single cell, a machine, whatever, is called the system — everything outside is the surroundings or environment. The first law says that any energy lost by the system must be gained by the environment and any energy taken up by the system must have come from the environment. Its application to chemical systems, which is what applies to nutrition, is that we can attribute to chemical systems, a so-called internal energy, usually written with symbol U (so as not to confuse it with the electrical potential, E). In thermodynamics, you usually look at changes, and the first law says that you can calculate ΔU, the change in U of a system, by adding up the changes in heat added to the system and work done by the system (you can see the roots of thermo in heat machines: we add heat and get work). In chemical systems, the energy can also change due to chemical reactions. Still, if you add up all the changes in the system plus the environment, all the heat, work and chemical changes, the energy is neither created nor destroyed. It is conserved.

Now, why doesn’t the first law apply to nutrition the way Ornish thinks it does? To understand this, you have to know what is done in chemical thermodynamics and bioenergetics, (thermo applied to living systems). If you want to. In nutrition, you can make up your own stuff. But, if you want to do what is done in chemical thermodynamics, you focus on the system itself, not the system plus the environment. So, from the standpoint of chemical thermodynamics, the calories in food represent the heat generated by complete oxidation of food in a calorimeter.

In a calorimeter, the food is placed in a small container with oxygen under pressure and ignited. The heat generated is determined from the increase in temperature of the water bath. (Before the food measurement, we determine the heat capacity of the water bath, that is, how much heat it takes to raise the temperature). The heat is how we define the calories in the food. The box around the sample in the figure shows that we are measuring the heat produced by the system, not the system plus the environment, that is, not applying the first law. If you applied the first law, the calories associated with the food would be zero, because any heat lost in combustion of the food would show up in the water bath of the calorimeter. The calories per gram of carbohydrate would be 0 instead of 4, the calories per gram of fat would be 0 not 9, etc. So, in studying reactions in chemical thermodynamics, energy is not conserved, it is dissipated. When systems dissipate energy, the change is indicated with a minus sign, so for oxidation of food, generally: ΔU < 0. So, no, the first law does not apply. That’s one of the reasons that “a calorie is not a calorie.”
There is an additional point that we assumed in passing. In chemical thermodynamics, the energy goes with the reaction, not with the food. It is not like particle physics where we give the mass of a particle in electron-volts, a measure of energy, because of E=mc2. What this means, practically, is that the 4 kcal per gram of carbohydrate is for the reaction of complete oxidation. Do anything else, make DNA, make protein and all bets are off.
The bottom line is that, contrary to what is usually said, thermodynamics does not predict energy balance and we should not be surprised when one diet is more or less efficient than another. In fact, the question to be answered is why energy balance is ever found. “A calories is a calorie” is frequently what is observed (although there is always a question as to how we make the measurement). The answer is that insofar as there is energy balance, it is a question of the unique behavior of living systems, not physical laws. Two similar subjects of similar age and genetic make-up may, under the right conditions, respond to different diets so that most of what they do is oxidize food and the contributions of DNA or protein synthesis, growth, etc. may be similar and may cancel out so that the major contribution to energy exchange is the heat of combustion.
But thermodynamics is really not about the first law which, while its history is a little odd, it is not revolutionary. Intellectually, the first law is related to conservation of matter. Thermodynamics is about the second law. The second law says that there is a physical parameter, called the entropy, almost always written S, and the change in entropy, ΔS, in any real process, always increases. In ideal, theoretical processes, ΔS may be zero, but it never goes down. In other words, looking at the universe, (any system and its surroundings), energy is conserved but entropy increases. The first law is a conservation law but the second law is a dissipation law. We identify the entropy with the organization, order or information in a system. Systems proceed naturally to the most probable state. In one of the best popular introductions to the subjects, von Baeyer’s Warmth Disperses and Time Passes, entropy is described in terms of the evolution of the organization of his teenage daughter’s room.  To finish up on calorimeters, though, there is Lavoisier’s whole animal calorimeter.

One of Lavoisier’s great contributions was to show that combustion was due to a combination with oxygen rather than the release of a substance, then known as the phlogiston. Lavoisier had the insight that in an animal, the combination of oxygen with food to produce carbon dioxide was the same kind of process. The whole animal calorimeter was a clever way to show this. The animal is placed in the basket compartment f. The inner jacket, b, is packed with ice. The outer jacket, a, is also packed with ice to keep the inner jacket, cold. The heat generated by the animal melts the ice in the inner jacket which is collected in container, Fig 8. Lavoisier showed that the amount of carbon dioxide formed was proportional to the heat generated as it would be if an animal were carrying out the same chemical reactions that occur, for example, in burning of charcoal. “La vie est donc une combustion.” His collaborator in this experiment was the famous mathematician Laplace and people sometimes wonder how he got a serious mathematician like Laplace to work on what is, well, nutrition. It seems likely that it was because Laplace owed him a lot of money.

“In the Viking era, they were already using skis…and over the centuries, the Norwegians have proved themselves good at little else.”

–John Cleese, Norway, Home of Giants.

With the 3-foot bookshelf of popular attacks on the low-fat-diet-heart idea it is pretty remarkable that there is only one defense.  Daniel Steinberg’s Cholesterol Wars. The Skeptics vs. The Preponderance of Evidence is probably more accurately called a witness for the prosecution since low-fat, in some way or other is still the law of the land.

The Skeptics vs. the Preponderance of Evidence

The Skeptics vs. the Preponderance of Evidence

The book is very informative, if biased, and it provides an historical perspective describing the difficulty of establishing the cholesterol hypothesis. Oddly, though,  it still appears to be very defensive for a witness for the prosecution.  In any case, Steinberg introduces into evidence the Oslo Diet-Heart Study [2] with a serious complaint:

“Here was a carefully conducted study reported in 1966 with a statistically significant reduction in reinfarction [recurrence of heart attack] rate.  Why did it not receive the attention it deserved?”

“The key element,” he says, “was a sharp reduction in saturated fat and cholesterol intake and an increase in polyunsaturated fat intake. In fact. each experimental subject had to consume a pint of soybean oil every week, adding it to salad dressing or using it in cooking or, if necessary, just gulping it down!”

Whatever it deserved, the Oslo Diet-Heart Study did receive a good deal of attention.  The Women’s Health Initiative (WHI), liked it.  The WHI was the most expensive failure to date. It found that “over a mean of 8.1 years, a dietary intervention that reduced total fat intake and increased intakes of vegetables, fruits, and grains did not significantly reduce the risk of CHD, stroke, or CVD in postmenopausal women.” [3]

The WHI, adopted a “win a few, lose a few” attitude, comparing its results to the literature, where some studies showed an effect of reducing dietary fat and some did not — this made me wonder: if the case is so clear, whey are there any failures.  Anyway, it cited the Oslo Diet-Heart Study as one of the winners and attributed the outcome to the substantial lowering of plasma cholesterol.

So, “cross-examination” would tell us why, if  “a statistically significant reduction in reinfarction  rate”  it did “not receive the attention it deserved?”

First, the effect of diet on cholesterol over five years:

The results look good although, since all the numbers are considered fairly high, and since the range of values is not shown, it is hard to tell just how impressive the results really are. But we will stipulate that you can lower cholesterol on a low-fat diet. But what about the payoff? What about the outcomes?

The results are shown in Table 5 of the original paper:   Steinberg described how in the first 5 years: “58 patients of the 206 in the control group (28%) had a second heart attack” (first 3 lines under first line of blue-highlighting) but only

“…  32 of the 206 in the diet (16%)…”  which does sound pretty good.

In the end, though, it’s really the total deaths from cardiac disease.  The second blue-highlighted line in Table 5 shows the two final outcome.  How should we compare these.

1. The odds ratio or relative risk is just the ratio of the two outcomes (since there are the same number of subjects) = CHD mortality (diet)/ CHD mortality control) = 94/79 =  1.19.  This seems strikingly close to 1.0, that is, flip of a coin.  These days the media, or the report itself, would report that there was a 19 % reduction in total CHD mortality.

2, If you look at the absolute values, however, the  mortality in the controls is 94/206 = 45.6 % but the diet group had reduced this  to 79/206 = 38.3 % so the change in absolute risk is  45.6 % – 38.3 % or only 7.3 % which is less impressive but still not too bad.

3. So for every 206 people, we save 94-79 = 15 lives, or dividing 206/15 = 14 people needed to treat to save one life. (Usually abbreviated NNT). That doesn’t sound too bad.  Not penicillin but could be beneficial. I think…

Smoke and mirrors.

It’s what comes next that is so distressing.  Table 10 pools the two groups, the diet and the control group and now compares  the effect of smoking: on the whole population,  the ratio of CHD deaths in smokers vs non-smokers is 119/54 = 2.2 (magenta highlight) which is somewhat more impressive than the 1.19 effect we just saw.  Now,

1. The absolute difference in risk is (119-54)/206 = 31.6 % which sounds like a meaningful number.

2. The number needed to treat is 206/64 = 3.17  or only about 3 people need to quit smoking to see one less death

In fact, in some sense, the Oslo Diet-Heart Study provides smoking-CHD risk as an example of a meaningful association that one can take seriously. If only such a significant change had actually been found for the diet effect.

So what do the authors make of this? Their conclusion is that “When combining data from both groups, a three-fold greater CHD mortality rate is demonstrable among the hypercholesterolemic, hypertensive smokers than among those in whom these factors were low or absent.”  Clever but sneaky. The “hypercholesterolemic, hypertensive” part is irrelevant since you combined the groups. In other words, what started out as a diet study has become a “lifestyle study.”  They might has well have said “When combining data from fish and birds a significant number of wings were evident.” Members of the jury are shaking their heads.

Logistic regression. What is it? Can it help?

So they have mixed up smoking and diet. Isn’t there a way to tell which was more important?  Well, of course, there are several ways.  By coincidence, while I was writing this post, April Smith posted on facebook, the following challenge “The first person to explain logistic regression to me wins admission to SUNY Downstate Medical School!” I won although I am already at Downstate.  Logistic regression is, in fact, a statistical method that asks what the relative contribution of different inputs would have to be to fit the outcome and this could have been done but in this case, I would use my favorite statistical method, the Eyeball Test.  Looking at the data in Tables 5 and 10 for CHD deaths, you can see immediately what’s going on. Smoking is a bigger risk than diet.

If you really want a number, we calculated relative risk above. Again, we found for mortality, CHD (diet)/ CHD (control) = 94/79 =  1.19. But what happens if you took up smoking: Figure 10 shows that your chance of dying of heart disease would be increased by 119/54 = 2.2  or more than twice the risk.  Bottom line: you decided to add saturated fat to your diet, your risk would be 1.19 what it was before which might be a chance you could take faced with authentic Foie Gras.

Daniel Steinberg’s question:

“Here was a carefully conducted study reported in 1966 with a statistically significant reduction in reinfarction  rate.  Why did it not receive the attention it deserved?”

Well, it did. This is not the first critique.  Uffe Ravnskov described how the confusion of smoking and diet led to a new Oslo Trial which reductions in both were specifically recommended and, again, outcomes made diet look bad [4].  Ravnskov gave it the attention it deserved. But what about researchers writing in the scientific literature. Why do they not give the study the attention it deserves. Why do they not point out its status as a classic case of a saturated fat risk study with no null hypothesis.  It certainly deserves attention for its devious style. Of course, putting that in print would guarantee that your grant is never funded and your papers will be hard to publish.  So, why do researchers not give the Oslo-Diet-Heart study the attention it deserves?  Good question, Dan.

Bibliography

1. Steinberg D: The cholesterol wars : the skeptics vs. the preponderance of evidence, 1st edn. San Diego, Calif.: Academic Press; 2007.

2. Leren P: The Oslo diet-heart study. Eleven-year report. Circulation 1970, 42(5):935-942.

3. Howard BV, Van Horn L, Hsia J, Manson JE, Stefanick ML, Wassertheil-Smoller S, Kuller LH, LaCroix AZ, Langer RD, Lasser NL et al: Low-fat dietary pattern and risk of cardiovascular disease: the Women’s Health Initiative Randomized Controlled Dietary Modification Trial. JAMA 2006, 295(6):655-666.

4. Ravnskov U: The Cholesterol Myths: Exposing the Fallacy that Cholesterol and Saturated Fat Cause Heart Disease. Washington, DC: NewTrends Publishing, Inc.; 2000.

In 1985 an NIH Consensus Conference was able to “establish beyond any reasonable doubt the close relationship between elevated blood cholesterol levels (as measured in serum or plasma) and coronary heart disease” (JAMA 1985, 253:2080-2086).

I have been making an analogy between scientific behavior and the activities of the legal system and following that idea, the wording of the conference conclusion suggests a criminal indictment. Since the time of the NIH conference, however, data on the role of cholesterol fractions, the so-called “good (HDL)” and “bad (LDL)” cholesterols and, most recently, the apparent differences in the atherogenicity of different LDL sub-fractions would seem to have provided some reasonable doubt. What has actually happened is that the nutrition establishment, the lipophobes as Michael Pollan calls them, has extended the indictment to include dietary fat, especially saturated fat at least as accessories on the grounds that, as the Illinois Criminal Code put it “before or during the commission of an offense, and with the intent to promote or facilitate such commission, … solicits, aids, abets, agrees or attempts to aid… in the planning or commission of the offense. . . ..”

A major strategy in the indictment of saturated fat has been guilt by association.  The American Heart Association (AHA), which had long recommended margarine (the major source of trans-fats), has gone all out in condemning saturated fatty acids by linking them with trans-fats.  The AHA website has a truly deranged cartoon film of the evil brothers: “They’re a charming pair, Sat and Trans.  But that doesn’t mean they make good friends.  Read on to learn how they clog arteries and break hearts — and how to limit your time with them by avoiding the foods they’re in.”. While the risk of trans-fats is probably exaggerated — they are a small part of the diet — they have no benefit and nobody wants to defend them; dietary saturated fat, however, is a normal part of the diet, is made in your body and is less important in providing saturated fatty acids in the blood, than dietary carbohydrate.  Guilt by association is a tricky business in courts of law — just having a roommate who sells marijuana can get you into a good deal of trouble — but it takes more than somebody saying that you and the perpetrator make a charming pair.

The failure of the diet-cholesterol-heart hypothesis in clinical trials as been documented by numerous scientific articles and especially in popular books that document the original scientific sources. It is unknown what the reaction of the public is to these books.  However, amazingly, there is only one book I know of that takes the side of the lipophobes and that is Daniel Steinberg’s Cholesterol Wars. The Skeptics vs. the Preponderance of Evidence. A serious book with careful if slightly biased documentation and an uncommon willingness to answer the critics,  it is worth reading.  I will try to discuss it in detail in this and future posts.  First, the title indicates a step down from criminal prosecution.  “Preponderance of the evidence” is the standard for conviction in a civil court and is obviously a far weaker criterion.  One has to wonder why it is that the skeptics have the preponderance of the popular publications — if the scientific evidence is there and health agencies are so determined that the public know about this, why are there so few —  maybe only this one — rebutting the critics.

The Skeptics vs. the Preponderance of Evidence

In any case, what is Steinberg’s case?  The indictment on page 1 is somewhat different than one would have thought.

“….the [lipid] hypothesis relates to blood lipids not dietary lipids as the putative directly causative factor. Although diet, especially dietary lipid is an important determinant of blood lipid levels, many other factors play important roles. Moreover, there is a great deal of variability in response of individuals to dietary manipulations. Thus, it is essential to distinguish between the indirect “diet-heart” connection and the direct “blood lipid — hard” connection failure to make this distinction has been a frequent source of confusion. (his italics)”

What?  Are we really supposed to believe that diet is an incidental part of the lipid hypothesis?  Are we supposed to believe that our cholesterol is just a question of the variability of our response to diet.  Has the message really been that diet is not critical and that heart-disease is just the luck of the draw (until we start taking statins)?  This is certainly the source of confusion in my mind.  Of course by page 5, we are confronted with this:

“In 1966, Paul Leren published his classic five-year study of 412 patients who had had a prior myocardial infarction. He showed that substitution of polyunsaturated fat and saturated fat-rich butter-cream-venison diet favored by the Norwegians reduced their blood cholesterol by about 17 per cent and kept it down.  The number of secondary current events in the treated group was reduced by about one-third and the result was significant at the p < 0.03 level.”

In a future post, I will describe Paul Leren’s classic five-year study which, by 1970, had a follow-up to eleven years and the results will turn out not to be as compelling as described by Steinberg.  For the moment, it is worth considering that, given the strong message, from the AHA, from the American Diabetes Association, from the NIH Guidelines for Americans, the criterion really should be beyond a reasonable doubt. There shouldn’t be even a single failure like the Framingham Study or the Women’s Health Initiative. In fact, the preponderance of the evidence when you add them all up, isn’t there.

Following the government’s nutritional advice can make you fat and sick.

by STEVEN MALANGA
City Journal, May 10, 2011

Last October, embarrassing e-mails leaked from New York City’s Department of Health and Mental Hygiene disclosed that officials had stretched the limits of credible science in approving a 2009 antiobesity ad, which depicted a stream of soda pop transforming into human fat as it left the bottle. “The idea of a sugary drink becoming fat is absurd,” a scientific advisor warned the department in one of the e-mails, a view echoed by other experts whom the city consulted. Nevertheless, Gotham’s health commissioner, Thomas Farley, saw the ad as an effective way to scare people into losing weight, whatever its scientific inaccuracies, and overruled the experts. The dustup, observed the New York Times, “underlined complaints that Dr. Farley’s more lifestyle-oriented crusades are based on common-sense bromides that may not withstand strict scientific scrutiny.”

Under Farley and Mayor Michael Bloomberg, New York’s health department has been notoriously aggressive in pursuing such “lifestyle-oriented” campaigns (see the sidebar below). But America’s public-health officials have long been eager to issue nutrition advice ungrounded in science, and nowhere has this practice been more troubling than in the federal government’s dietary guidelines, first issued by a congressional committee in 1977 and updated every five years since 1980 by the United States Department of Agriculture. Controversial from the outset for sweeping aside conflicting research, the guidelines have come under increasing attack for being ineffective or even harmful, possibly contributing to a national obesity problem. Unabashed, public-health advocates have pushed ahead with contested new recommendations, leading some of our foremost medical experts to ask whether government should get out of the business of telling Americans what to eat—or, at the very least, adhere to higher standards of evidence.

Until the second half of the twentieth century, public medicine, which concerns itself with community-wide health prescriptions, largely focused on the germs that cause infectious diseases. Advances in microbiology led to the development of vaccines and antibiotics that controlled—and, in some cases, eliminated—a host of killers, including smallpox, diphtheria, and polio. These advances dramatically increased life expectancy in industrialized countries. In the United States, average life expectancy improved from 49 years at the beginning of the twentieth century to nearly 77 by the century’s end.

As the threat of communicable diseases receded, public medicine began to turn its attention to treating and preventing health problems that weren’t germ-caused, such as chronic heart disease and strokes, the death rates for which seemed to be soaring after World War II. Some observers cautioned that the apparent increase might be the result of diagnostic advances, which had improved doctors’ ability to detect heart ailments. This possibility, however, failed to deter the press and advocacy groups like the American Heart Association from declaring the arrival of a frightening epidemic.

One theory blamed the problem on the American diet, and in particular on cholesterol—both the kind that you ingest when you eat animal products and the kind that your body produces when you eat saturated fats. It wasn’t an unreasonable idea; cholesterol is, after all, one component of the plaque that clogs arteries and causes heart attacks and strokes. But isolating the true causes of coronary disease proved elusive. Multiple factors—not just diet but other personal habits, such as smoking, and genetics as well—were potential contributors. And measuring the influence of diet was especially difficult because of big variations among individuals in everything from blood composition to their response to different foods. Numerous studies on diet proved so inconclusive that in 1969, the National Institutes of Health found no hard evidence that what people ate had a significant impact on heart disease.

Nevertheless, in the 1970s, Democratic senator George McGovern’s Select Committee on Nutrition and Human Needs decided to fight the apparent epidemic by making recommendations on nutrition. “Our diets have changed radically within the past 50 years,” McGovern declared, “with great and very often harmful effects on our health.” As science writer Gary Taubes notes in Good Calories, Bad Calories, the McGovern committee, in coming up with its diet plan, had to choose among very different nutritional regimes that scientists and doctors were studying as potentially beneficial to those at risk for heart disease. Settling on the unproven theory that cholesterol was behind heart disease, the committee issued its guidelines in 1977, urging Americans to reduce the fat that they consumed from 40 percent to 30 percent of their daily calories, principally by eating less meat and fewer dairy products. The committee also advised raising carbohydrate intake to 60 percent of one’s calories and slashing one’s intake of cholesterol by a quarter.

Some of the country’s leading researchers spoke out against the guidelines and against population-wide dietary recommendations in general. Edward Ahrens, an expert in the chemistry of fatty substances at Rockefeller University, characterized the guidelines as “simplistic and a promoter of false hopes” and complained that they treated the population as “a homogenous group of [laboratory] rats while ignoring the wide variation” in individual diet and blood chemistry. The Food and Nutrition Board of the National Academy of Sciences released its own dietary suggestions, which saw “no reason for the average healthy American to restrict consumption of cholesterol, or reduce fat intake,” and just encouraged people to keep their weight within a normal range.

Even members of McGovern’s committee demurred. In a supplemental foreword to the second edition of the guidelines, ranking Republican senator Charles Percy acknowledged that the scientific record included “extreme diversity of opinion.” Canada’s Department of National Health and Welfare, Percy noted, had recently declared that “evidence is mounting that dietary cholesterol may not be important to the great majority of people”; Great Britain’s Department of Health and Social Security had reached a similar conclusion in 1974. Percy concluded that it was important to inform the public “not only about what is known, but what is controversial.”

Still, the low-fat guidelines gained traction in an era when food advocacy and vegetarianism were rising, as Taubes relates. In 1968, Paul Ehrlich had published his apocalyptic bestseller, The Population Bomb, prophesying mass starvation because the earth could no longer provide enough food for humanity. Ehrlich’s book was out of date as soon as it appeared, thanks to scientific advances that made agriculture more productive worldwide. But it nevertheless gave ammunition to advocates who urged people in developed countries to eat fewer animal products so that the world’s poor, supposedly hungrier and hungrier, could consume more of the grain that wealthy nations turned into feed for domestic animals. In 1971, Frances Moore Lappé’s vegetarian manifesto Diet for a Small Planet hit the bestseller list.

A new kind of health-care advocate, evincing a passion far removed from disinterested scientific inquiry, also took up the campaign for a vegetable-based, low-fat diet. A good example was the Center for Science in the Public Interest, which in 1975 organized a National Food Day that included, the New York Times reported, an “all-out attack” on foods that it considered harmful. On the hit list: prime beef, high in fat and cholesterol.

When the McGovern committee issued its guidelines, these advocacy groups attacked opponents as shills for the food industry—dismissing the National Research Council’s more restrained dietary recommendations, for instance, because some of the scientists who worked on them also served as consultants to industry groups like the Egg Council. By contrast, the advocates noted, the McGovern guidelines were largely the work of a committee staffer, a former newspaper reporter whose very lack of scientific expertise meant that he had no such conflicts.

But the line between advocate and policymaker was blurring on both sides of the debate. One of the important figures promoting the dietary guidelines was Assistant Secretary of Agriculture Carol Foreman, who had formerly been director of the Consumer Federation of America, a cosponsoring organization of National Food Day. “People were getting sick and dying because we ate too much,” she told Taubes. She urged government scientists to tell Americans what to eat, even if “it’s not the final answer.”

The McGovern dietary recommendations weren’t just ahead of the science, though; they were racing ahead of it. Two of the most important U.S. government–sponsored studies on the role of fat and cholesterol in heart disease didn’t appear until the early 1980s, long after the committee had promulgated its advice. The results hardly cleared things up. The first study, known as the Multiple Risk Factor Intervention Trial, followed 12,866 people between the ages of 35 and 57 at risk for heart disease. Some of these subjects were placed on a low-fat, low-cholesterol diet; others were merely told to keep seeing the family doctor. The study found no statistically significant difference in mortality rates between the two groups.

The results of the second study, the Lipid Research Clinics Coronary Primary Prevention Trial, appeared in 1984 and continue to spark debate. Using the drug cholestyramine to reduce high cholesterol rates in a group of male test subjects, the study reported a lower death rate for those on the drug than for subjects who took a placebo. Did this mean that cholesterol was to blame for heart disease, after all? Some observers, including Ahrens, cautioned that the average cholesterol level of the American public was far lower than that of the test group taking cholestyramine, meaning that there was nothing in the study to suggest that a nationwide effort to change citizens’ diets would make much difference in public health. But the press seemed to prefer a narrative that made diet a major cause of heart attacks. A 1984 Time cover story about cholesterol showed a dinner plate turned into an unhappy face, with two sunny-side-up eggs the frazzled-looking eyes above a frowning strip of bacon.

The scientific controversy grew more intense. In 1992, an authoritative review of 19 cholesterol studies worldwide found that, while men with cholesterol levels above 240 were disproportionately likely to suffer heart attacks, men with cholesterol levels below 160 were disproportionately likely to die from all causes, including lung cancer, respiratory disease, and digestive disease—an outcome that suggested a relationship between lowcholesterol levels and disease, something that scientists had never considered. The study also showed no difference in mortality rates for men with cholesterol levels between 160 and 240, even though the guidelines advised keeping levels below 200. Perhaps most surprisingly, the study also found that cholesterol levels made no difference at all in death rates among women. There was little doubt that some public-health researchers wished such research would go away. “Some people don’t want to talk about it,” said Michael Criqui, an epidemiologist at the University of California at San Diego and an associate editor of Circulation, which published the review. “They think it is going to impede public-health measures.”

More recent research has further undermined the cholesterol-as-bad-guy hypothesis. Scientific American summed up the disturbing state of the evidence in April 2010. The magazine cited a meta-analysis—that is, a combination of data from several large studies—of the dietary habits of 350,000 people worldwide, published in The American Journal of Clinical Nutrition, which found no association between the consumption of saturated fats and heart disease. Another recent study noted by Scientific American, by Harvard nutrition and epidemiology professor Meir Stampfer and associates and published in The New England Journal of Medicine, tracked 322 moderately obese people, each following one of three diets: a low-fat, calorie-restricted diet of the sort that the American Heart Association recommends; a so-called Mediterranean diet, rich in vegetables and low in red meat; and a low-carbohydrate diet without any calorie restrictions. Not only did the low-carb dieters lose the most weight, the study found; they also had the healthiest ratio of HDL (so-called good) cholesterol to LDL (bad) cholesterol.

The latest nutritional thinking has indeed zeroed in on carbohydrates as a likely cause of heart disease. Easily digestible carbs, in particular—starches like potatoes, white rice, and bread from processed flour, as well as refined sugar—make it hard to burn fat and also increase inflammations that can cause heart attacks, several studies have concluded. A 2007 Dutch study of 15,000 women found that those who ate foods with the highest “glycemic load,” a measure of portion sizes and of how easily digestible a food is, had the greatest risk of heart disease.

Looking at such evidence, several top medical scientists have concluded that the government’s carb-heavy guidelines may actually have harmed public health. In 2008, three researchers from the Albert Einstein School of Medicine—including the associate dean of clinical research, Paul Marantz, and a former president of the International Hypertension Society, Michael Alderman—observed in The American Journal of Preventive Medicine that since 1977, Americans have largely followed the government’s advice, doubtless as conveyed by the doctors they consulted. Men, for instance, cut their fat intake from 37 percent of their daily calories to 32 percent and increased their carbohydrate intake from 42 percent to 49 percent. Yet over the same three decades, the fraction of American men who were overweight or obese increased from 53 percent of the population to about 69 percent. The doctors wondered whether this correlation was an unintended consequence of telling the entire population to change its eating patterns. “In general,” the doctors wrote, “weak evidentiary support has been accepted as adequate justification for [the U.S. dietary] guidelines. This low standard of evidence is based on several misconceptions, most importantly the belief that such guidelines could not cause harm.” But, they concluded, “it now seems that the U.S. dietary guidelines recommending fat restriction might have worsened rather than helped the obesity epidemic and, by so doing, possibly laid the groundwork for a future increase in CVD,” cardiovascular disease.

It’s true that the particular kind of carbohydrates that the government has always recommended are carbs rich in fiber, which aren’t as quickly digested as those starches implicated by the latest research. But it’s difficult to tell an entire population to change its dietary habits without sowing confusion about such fine points. Further, as an October 2010 article in Nutrition points out, the government’s definition of what constitutes a fiber-rich grain is so broad as to include many foods that might actually promote heart disease because they are too easily digestible. “At a minimum,” says one of the authors of the Nutrition piece, SUNY Downstate Medical Center biologist Richard Feinman, “if you have an area of controversy or ambiguity in the science, you shouldn’t be issuing guidelines to the entire population.”

The guidelines themselves quietly acknowledge that they may have worsened public health. The 2000 version eliminated the recommendation to reduce intake of overall fat in favor of carbs, noting “the possibility that overconsumption of carbohydrates may contribute to obesity.” But that was as far as the government would go. It retained the advice to limit consumption of saturated fat and to keep intake of cholesterol to 300 milligrams per day, for example, even though dietary cholesterol—that is, the cholesterol we ingest by eating animal products—has been discounted by many researchers as a source of plaque buildup. (It was this advice about dietary cholesterol that led doctors, starting in the 1970s, to counsel patients to avoid eggs. Subsequent studies have concluded that any restrictions on eating them are “unwarranted for the majority of people and are not supported by scientific data,” as a 2004 article in The Journal of Nutrition put it.)

Supporters of the guidelines have increasingly resorted to ad hoc, even political, justifications for them. In a 2008 American Journal of Preventive Medicine article, for example, two influential nutritionists, Marion Nestle of New York University and Steven Woolf of the Virginia Commonwealth University Medical Center, admit that “whether the evidence is good enough to recommend population-based dietary changes comes down to a matter of subjective judgment.” But developing dietary recommendations is still a crucial government responsibility, they argue, in part because the government is already heavily involved in food policies. “Dietary guidelines have implications at every level of government, from federal agencies such as the U.S. Department of Agriculture (USDA) to the local school board,” they write, and without clear guidelines, big food industries and special interests could lobby political leaders and shape policy in unhealthy ways. But this argument makes sense only if you assume that the government’s guidelines will be any healthier.

Nestle and Woolf also argue that government’s success in persuading people to stop smoking justifies its efforts to change American eating habits. “If it was paternalistic for the government to advise people how to eat,” they ask rhetorically, “was it equally paternalistic . . . to alert the public about the hazards of tobacco use and to recommend in 1964 that smokers give up cigarette smoking?” But the major scientific dissenters from government dietary policy don’t accuse it of paternalism, though that’s a legitimate argument; they dissent because they find the government’s evidence inadequate and its recommendations potentially harmful.

The government’s response to the growing controversy has been to keep issuing the guidelines—and call for more research. Asked last year about whether the 2010 update would reflect the latest studies challenging previous recommendations, a USDA spokesperson merely suggested that the controversial areas be “put on the list of things to do with regard to more research.” In other words, more research is needed to overturn or withdraw the current recommendations, even though they were based on inconclusive evidence from the start.

Mike Bloomberg, Food Cop

In his 2005 book Prescription for a Healthy Nation, Dr. Thomas Farley wondered why “Americans behave in such an unhealthy way” and concluded that we don’t have the will to overcome the temptations around us, like easy access to junk food. Publishers Weekly noted that the book had “a pervasive tone of puritanical disapproval” and that Farley and his coauthor, Dr. Deborah Cohen, sounded like a “pair of scolds.” In 2008, New York City mayor Michael Bloomberg made the first of the scolds the city’s health commissioner, and Farley hasn’t disappointed. He’s carried on a tradition—begun by Bloomberg and his first health commissioner, Thomas Frieden, now director of the U.S. Centers for Disease Control and Prevention—of pursuing population-wide dietary policies that are aggressive, questionable, and born from the belief that government should regulate Americans’ indulgent behavior.

Early in his tenure, Frieden had his hands full, confronting everything from a SARS scare to a drug-resistant form of HIV. He nonetheless expanded his portfolio beyond these classic public-health threats into the more controversial area of lifestyle recommendations. In 2005, he began a campaign against man-made trans fats of the sort that appear in partially hydrogenated oils, which ultimately led to a ban on them—the first of its kind—in the city’s restaurants. Then the city mandated that chain restaurants include calorie counts in their menus. In 2009, Frieden and then Farley began a nationwide campaign to get food companies to use less sodium. Most recently, the Department of Health and Mental Hygiene has waged a campaign against sugared soda pop, aggressively linking it to human fat in a cause-and-effect relationship that even the city’s own scientific advisors have rejected.

Bloomberg’s interest in the field predates his mayoralty; the nation’s largest school of public-health medicine, at Johns Hopkins University, is named after him in gratitude for his large donations. But the health campaigns that he has run from city hall have made him a national influence. The Harvard School of Public Health Medicine awarded him its top honor in 2007. The 2009 federal health-care legislation commonly known as Obamacare, following New York’s lead, will require chain restaurants nationwide to post calorie counts, while the dietary guidelines issued by the United States Department of Agriculture, picking up where New York left off, are now targeting sodium as a health risk, too.

Among scientists, however, the Bloomberg administration’s crusades are controversial, because they push the limits of what research has shown and ignore the potential unintended consequences of advising entire populations on how to change their diets. In an article titled “The Panic Du Jour: Trans Fats,” New York Times nutrition columnist Gina Kolata noted that studies show trans fats to be no more dangerous than saturated fats. The authors of a 2008 article in The American Journal of Preventive Medicine, “A Call for Higher Standards of Evidence for Dietary Guidelines,” worried that “the net of effects of legal restrictions” on trans fats was unknown and that a ban could backfire if it led Americans to assume that foods without trans fats were healthful. Meanwhile, a study by researchers at New York University of the city’s calorie-posting law found that it had no effect on what consumers bought. Though a quarter of parents in the study claimed to consult the calorie postings before buying food for their kids, their receipts showed no calorie difference between the meals that they’d purchased and those purchased by parents who didn’t look at the postings.

Apparently, none of these controversies gave the Bloomberg administration pause. The city undertook its 2009 campaign against salt despite decades of scientific uncertainty about whether lowering sodium consumption matters to anyone except those few Americans highly sensitive to salt.

The mayor and his health commissioners have exaggerated the potential health benefits of their campaigns, in the process raising fundamental questions about the point at which a government should intervene in public-health matters. Bloomberg himself estimated, for instance, that the trans-fat ban would save “a couple of hundred lives a year in New York City,” a completely untested statement that failed to take into account a host of possibilities—for example, that people might start to consume an alternative to trans fats that would do just as much harm. But such exaggerations are increasingly common in public-health medicine because overstating the beneficial effects of policies helps get them enacted.

As if all this weren’t troubling enough, the USDA, again with uncertain scientific warrant, is now targeting sodium as a public-health menace. Following the lead of New York City’s health department, which is prodding food manufacturers to make their products less salty, the 2010 guidelines recommend that sodium consumption fall as low as 1,500 milligrams a day for those over 51, more than a one-third reduction from the amount that the previous version of the guidelines suggested.

For the general population of healthy Americans, however, that advice may be pointless or, again, even harmful. Decades of research have yielded continuing controversy over the benefits of lowering salt consumption. The science remains so inconclusive that Alderman recently described calls to reduce sodium intake as merely “opinion or common practice,” not science.  Experts like the authors of the October 2010 Nutrition article argue that people with particular health problems, such as hypertension, may indeed suffer from excessive sodium intake. But that’s a far cry from saying that everybody should cut down on salt. Alderman, an expert on hypertension, worries that the war on salt may have unintended consequences; diets that reduce salt intake produce a host of physiological changes, including decreased insulin sensitivity, which can raise the risk of heart disease. None of these concerns has stopped the Center for Science in the Public Interest from waging a zealous public-health crusade denouncing salt as “the deadly white powder you already snort.”

It’s all the more important to understand the problems with the dietary guidelines as the federal government embarks on its new campaign against obesity, which research and clinical experience have shown to be a major factor in ailments like diabetes and chronic heart disease. When the White House announced late last year that First Lady Michelle Obama would lead the fight against childhood obesity and she observed that “we can’t just leave it up to parents,” some prominent conservatives, including columnist Michelle Malkin and former vice presidential candidate Sarah Palin, accused the administration of entering an arena where parents, not the government, should be making decisions.

Opponents of the administration’s plans, however, shouldn’t just debate the government’s proper role in people’s health; they should also point out that its population-wide diet advice goes well beyond what science has established. “Some people in this field act more like zealots with a passion for a cause than scientists waiting for the evidence to support their conclusions,” complains California Polytechnic public-health economist Michael Marlow. As Marlow notes, America’s obesity rate was far lower back when nutrition was largely a parental responsibility, before government became widely involved in the diet-advice business.

The best thing government can encourage Americans to do on the health front may well be to develop their own diet and exercise programs, based on their individual circumstances, in consultation with health-care professionals. Otherwise, public-health medicine risks violating the central principle of medical ethics: First, do no harm.

Steven Malanga is the senior editor of  City Journal  and a senior fellow at the Manhattan Institute.  He is the author of  Shakedown: The Continuing Conspiracy Against the American Taxpayer.  

City Journal  offers a stimulating mix of hard-headed practicality and cutting-edge theory, with articles on everything from school financing, policing strategy, and welfare policy to urban architecture, family policy, and the latest theorizing emanating from the law schools, the charitable foundations, even the schools of public health. Since urban policy encompasses almost all domestic policy questions, as well as the largest issues of our culture and society, the magazine views its canvas as very broad indeed. The magazine holds itself to the highest intellectual, journalistic, and literary standards, aiming to produce intelligent and absorbing reading for intelligent and discerning readers.  

Journal Nutrition  Article: In the face of contradictory evidence: Report of the Dietary Guidelines for Americans Committee

The phrase “Evidence-based Medicine” (EBM) guarantees its proponents a certain degree of protection. After all, who would be against medicine that is based on the data, on hard facts rather than opinion. On the other hand, a study that needs to cloak itself in such a self-aggrandizing phrase must raise a few eyebrows; as usual, the Dietary Guidelines, moves to the top of the list in that category but there are many examples.  Martin Tobin, professor of Medicine at Loyola College provided an excellent deconstruction of evidence based medicine [1]. Some of his points were that the grading system has divorced itself from basic science.  For example, he points out that:

  • “ homeopathy uses drugs in which less than one molecule of active agent is present. … A meta-analysis of 89 placebo-controlled trials revealed a combined odds of 2.45 in favor of homeopathy. EBM grades meta-analysis as level 1 evidence but completely ignores scientific theory. There is nothing necessarily wrong with this particular meta-analysis, but the example illustrates how a system that grades findings of all meta-analyses as level 1 evidence is inherently flawed.  A grading system that ranks homeopathy as sounder evidence than centuries of pharmacologic science commits the reductio ad absurdum fallacy in logic.” [1]

Among the things that we found in our critique of the USDA dietary guidelines Report [2] was that the cited evidence did not meet their own standards. They were critical of low-carbohydrate diets on the basis of studies that their own analysis gave a “neutral” quality rating, even those that took dietary assessment at baseline and then assessed  cardiovascular mortality up to 12 years later.

But it is really the idea that there is some set of systematic definitions of science that everybody agrees on. My last post mentioned, by analogy with courts of law, the Frye standard which accepts as evidence, opinions supported by  “general acceptance’ in the scientific community.  While still accepted in some state courts, the federal courts have tried to go beyond trust in such narrow descriptions of science. In 1975, Congress established Federal Rules of Evidence.  The rules are quite general and the major impact is to broaden the range of evidence that could be considered.  Rule 401, defined relevance as  “evidence having any tendency to make the existence of any fact that is of consequence to the determination of the action more probable or less probable than it would be without the evidence,”  in other words, whatever works.  In a future post, I will discuss Daubert v. Merrell Dow, Inc. (pr. Dow-burt as in English), an outgrowth of the Rules of Evidence and generally considered the key judgment in the modern interaction of science and the law.  In the real world of jurisprudence, ideas on what constitutes scientific evidence have become problematical and Daubert may have had the paradoxical effect of restricting admissible data but, in the analogy with evidence in medicine, the Federal Rules of Evidence and Daubert have better captured the real quality of science in recognizing the need for flexibility. The kinds of absolute criteria — association does not imply causality, random controlled trials are a “gold standard,” etc. are at least different from the spirit of Daubert.

More important, nobody in any physical science would recognize the tables of levels of evidence.  A random controlled trial may be good for one kind of experiment but not for another and EBM is critical of “observational studies” but all of astronomy is observational.  In the end, most scientists would agree with the physicist Steven Weinberg, echoing Judge Potter Stewart’s famous take on pornography:

  •  “There is no logical formula that establishes a sharp dividing line between a beautiful explanatory theory and a mere list of data, but we know the difference when we see it — we demand a simplicity and rigidity in our principles before we are willing to take them seriously [3].”

So where do these arbitrary guidelines in EBM come from?  They were set up by the  medical community, a community that is stereotyped as being untrained in science. I hate stereotypes, especially medical stereotypes since I think of myself as coming from a medical family (my father and oldest daughter are physicians) but stereotypes come from someplace and, of course, it is well known that physicians never study nutrition.  In the end, it makes me think of the undoubtedly apocryphal story about Mozart.

  • A man comes to Mozart and wants to become a composer.  Mozart says that they have to study theory for  a couple of years, they should study orchestration and become proficient  at the piano, and goes on like this.  Finally, the man says “but you wrote your first symphony when you were 8 years old.”  Mozart says “Yes, but I didn’t ask anybody.”

Bibliography 

1. Tobin MJ: Counterpoint: evidence-based medicine lacks a sound scientific base. Chest 2008, 133(5):1071-1074; discussion 1074-1077.

2. Hite AH, Feinman RD, Guzman GE, Satin M, Schoenfeld PA, Wood RJ: In the face of contradictory evidence: report of the Dietary Guidelines for Americans Committee. Nutrition 2010, 26(10):915-924.

3. Weinberg S: Dreams of a final theory, 1st edn. New York: Pantheon Books; 1992.

According to the Journal of the American Medical Association (JAMA), the principle of “evidence-based medicine (EBM),” arose in the 1990s [1]. It is widely invoked in the medical literature as a kind of certification that the conclusions of the author are not mere opinions but are backed up by compelling information in biomedical science.  It sounds good. Or does it? It is certainly self-serving and a little bit suspicious, somewhat like Nixon assuring us that he was not a crook.  Evidence based medicine?  What were we doing before?  How was Pasteur able to function in the absence of such an idea?  One thing to think about is that evidence is what is introduced into courts of law.  But not all evidence is admissible. A judge decides what is admissible and there are many precedents, in particular, on what constitutes scientific evidence in a legal proceeding.

EBM relies on a hierarchy of levels of evidence (e.g. Table 1) with the random controlled trial (RCT) as the highest and expert opinion as the lowest.  Recommendations from health agencies and awarding of research grants are frequently justified on conformity to EBM or at least on their placing primary importance on RCTs.

Evidence from the USDA

Early in 2011, the USDA released its 2010 Dietary Guidelines for Americans [2].  With the dates suggesting the backward-looking nature of the Guidelines, they were nonetheless based on the Report of a prestigious committee (DGAC) [3] who, in turn, made much of their reliance on a new Nutrition Evidence Library (NEL). I and my colleagues were invited to submit a critique of the Report by the journal Nutrition. The editor, Michael Meguid indicated that the journal wanted a balanced report, pros and cons.  I called Dr. Meguid:

RDF: You know, the report is not particularly balanced. I’m not sure how you write a balanced review of an unbalanced report.

MM:  You can make the critique as strong as you like as long as you carefully document everything. But what’s your main problem with the Report?

RDF: Well, it makes very strong recommendations in the face of contradictory evidence.

MM: Make that the title of your article.

So we wrote an article called “In the face of contradictory evidence: Report of the Dietary Guidelines for Americans Committee” [4]. The journal was kind enough to make it an open access article and it’s available on this blog. In the end, on titles, we were one-upped by Steven Malanga, whose article in the New York Post was called “Fed’s Food Fog.”

For sure, both the Report and the final Guidelines were the proverbial camel-like production of a committee, tedious, repetitive and stylistic dreadful.  But what about the NEL?  What about the evidence?  Style aside, wasn’t this evidence based medicine?

Where do these guidelines come from? The assumption is that evidence follows its etymologic roots, stuff that is visible, stuff that comes from the sensible and true avouch of our own eyes. In fact, it is most often applied, as in the case of the DGAC, to the most controversial and contentious subjects. Calling something evidence is not enough. So what happens in courts of law? In a court of law, a judge decides on whether the jury can hear the evidence.  Who decides admissibility of the evidence in EBM?

Conflict resolution in science.

Science is a human activity. Conflict, controversy and a resistance to new ideas are well known even in the so-called hard, that is, more mathematical, sciences, and even where there are no outside forces as there was in the case of Galileo.  In the twentieth century, conflicts do not generally impede progress for long. Especially in the physical sciences, there is usually agreement on basic assumptions and on the rules of logic, allowing ultimate acceptance of strong evidence. Competing theories may coexist and supporters of both are likely to admit that they are awaiting reconciliation.

What happens when the spontaneous process of conflict resolution in science breaks down?  What happens in conditions where scientific disagreement is strong and a majority position becomes so dominant that it controls the funding and publication of scientific work and can ignore or repress contradictory evidence and repress exposition of alternative theories.  In essence, how do we deal with a recapitulation of the case of Galileo?

There is no system to decide on the admissibility in the cases considered by EBM.   I am not the first person to point out that EBM is largely the position of experts on one side of a scientific conflict [5], the lowest level of evidence on traditional EBM scales (e.g. “Level III: Opinions of respected authorities… of the US Preventive Services Task Force Systems,” Table 1).  EBM is sustained by those who want to use its particular criteria but these have never been subjected to outside affirmation.

In this situation, where science cannot police itself, we have to look for some outside guidance.  What do the courts do?  As one would expect there, is a long and extensive history of the legal system’s  attempt to deal with what constitutes scientific evidence.  On the chance that the legal perspective may help, I will discuss some of the issues.

Frye and the need for rules.

A key decision in the history of science in the courts is Frye v. United States.  In 1923, a Federal Appeals court ruled that the opinions of experts have to be supported by a scientific community. Frye had been convicted of second-degree murder but appealed on the grounds that he had successfully passed a lie-detector test.  At that time the device was a simple blood pressure machine and an expert witness testified as to the results. The court ruled that the lie-detector test “has not yet gained such standing and scientific recognition among physiological and psychological authorities as would justify the courts in admitting expert testimony‚” affirming the judgment of the lower court.

The ruling in Frye gave rise to the idea of “general acceptance,” and, by analogy, this appears to be the main principle in the admissibility of evidence in the nutrition world.  Sufficiently well established that it could be included in a biochemistry text is the idea that “consumption of saturated fats is positively associated with high levels of total plasma cholesterol and LDL cholesterol and an increased risk of CHD”[6] Known to students as “the Lippincott Book,” Harvey and Ferrier is the best selling biochemistry book in the world and it is correct when it states “Most experts strongly advise limiting intake of saturated fats.”

Most, but not all.  A small but not insignificant minority hold otherwise and whereas they agree that dietary saturated fat may raise blood cholesterol, they can provide overwhelming evidence that it is not associated with cardiovascular disease. This has been demonstrated in almost every large trial.

The problem is described in Marcia Angell’s Science on Trial [7].  Angell explains that Frye was not without its critics ([7], page 126).  Opponents, she wrote,

“claimed somewhat improbably, that it would tend to exclude novel, far-sighted testimony by modern-day Galileos. There is no record of this happening once, let alone often.  Furthermore, even if a modern-day Galileo did not make into court at first, that fact should not stop him from prevailing in the scientific community.  Courts do not determine scientific acceptance, as implied by the argument that we need to keep our courts open to the hidden Galileos in our midst.”

But isn’t this exactly what has happened in nutrition and maybe, in general, in the medical community?  The “experts” control editorial boards, granting agencies and academic departments and are as powerful as the Catholic Church in repressing dissent.  They have prestige and, in many cases, undisputed accomplishments, but does science run on general acceptance? Does majority (of experts) rule?

One of the problems with Frye that lawyers have addressed is a question of identifying the field of academic or scientific field in which the general acceptance is to be considered.  Different disciplines hold to different standards.  In the case at hand, many ideas in nutrition would be dismissed out of hand by biochemists. Many methodologies would be considered absurd by physical scientists: Intention-to-treat is perhaps the most absurd.  It has been pointed out that the question of who is an expert might have applied to the techniques in the original Frye case, at least as it might be implemented today: “If polygraph examiners are selected as the relevant field, polygraph results would be admissible.” (http://law.jrank.org/pages/2006/Scientific-Evidence-Frye-v-United-States.html).

The epidemic of obesity and diabetes stands as a testament to the failure of the experts.  A small library can be assembled of books attacking establishment medical nutrition. Uffe Ravnskov’s classic Cholesterol Myths is updated in Ignore the Awkward. Gary Taubes’s recent Good Calories, Bad Calories is the most compelling and James Le Fanu’s Rise and Fall of Modern Medicine, the most succinct but just sitting at my desk now I can see a dozen others on the book shelf.  Surprisingly, there has been only one rebuttal, Steinberg’s Cholesterol Wars, the subject of the next post .

Table 1.  Examples of Levels of Evidence from Various Sources. 

US Preventive Services Task Force Systems for ranking evidence about the effectiveness of treatments or screening:

Level I: Evidence obtained from at least one properly designed randomized controlled trial.

Level II-1: Evidence obtained from well-designed controlled trials without randomization.

Level II-2: Evidence obtained from well-designed cohort or case-control analytic studies, preferably from more than one center or research group.

Level II-3: Evidence obtained from multiple time series with or without the intervention. Dramatic results in uncontrolled trials might also be regarded as this type of evidence.

Level III: Opinions of respected authorities, based on clinical experience, descriptive studies, or reports of expert committees.

Bibliography

1. Torpy JM, Lynm C, Glass RM: JAMA patient page. Evidence-based medicine. JAMA 2009, 301(8):900.

2. Dietary Guidelines for Americans, 2010 [http://www.dietaryguidelines.gov.]

3. US Department of Agriculture and US Department of Health and Human Services: Report of the Dietary Guidelines Advisory Committee on the dietary guidelines for Americans, 2010. June 15, 2010. In.; 2010.

4. Hite AH, Feinman RD, Guzman GE, Satin M, Schoenfeld PA, Wood RJ: In the face of contradictory evidence: report of the Dietary Guidelines for Americans Committee. Nutrition 2010, 26(10):915-924.

5. Marantz P, Bird E, Alderman M: A Call for Higher Standards of Evidence for Dietary Guidelines. Am J Prev Med 2008, 34(3):234-239.

6. Harvey R, Ferrier D: Biochemistry, 5th edn. Baltimore and Philadelphia: Lippincott Williams & Wilkins; 2011.

7. Angell M: Science on Trial. New York: W. W. Norton & Co.; 1996.

Coincidence.  My daughter Mimi’s school was debating whether chocolate milk, skim or otherwise, is really a good thing — whole milk of course is not a choice.  At the same time, her first grade class was studying states of matter, specifically solids and liquids.  According to Mimi, they learned that solids do not change shape and liquids flow.  These two topics come together in the bizarre world of the USDA Dietary Guidelines.  Whole milk, it seems, is not a choice because it may be a solid.

Stepping back, the logic of the USDA recommendations is that

1) we now know that  total dietary fat poses no risk for cardiovascular disease (CVD), but

2) saturated fat  is a risk for CVD and therefore Americans must

3) reduce total dietary fat intake and drink skim-milk or fat-reduced milk.  If you are confused about this syllogism, not to worry.  The Guidelines have made it simple. The Advisory Committee has always had a concern for the ability of the public to think.  Dr. Xavier Pi-Sunyer (pr. Pea-soon-yay), a member of the committee wrote an editorial about a scientific report that showed no difference in the effect of different diets (never mind that, in practice, what people ate was about the same, regardless of the “diet” to which they were assigned.) He suggested that it “seems unwise at this point to burden type 2 diabetes patients with trying to pick and choose among different high- and low-GI foods.”  Also, in unveiling the new Guidelines, secretary of Agriculture Tom Vilsack said that he had not read the previous versions.  So, any simplification is welcome. The Dietary Guidelines has, as one of its latest innovations, the introduction of the term SoFAS, which stands for Solid Fats and Added Sugar and was in line for a SAY! Award (Stupidest Acronym of the Year).  So, we simply have to understand what they mean by solid.

If there is anything in science that is well understood and should not be open to conjecture, it is the states of matter.  Outside of the extreme conditions that produce plasmas and other exotica, the idea of solids, liquids and gasses seems pretty much, well, solid.  It turns out, though, that since the term saturated fat is, after all, a chemical term, it is necessary to make things simple for the consumer who is now advised to look for something “solid” instead.  To do this, the DGAC explains: “To determine whether foods contain oils or solid fats, consumers can read the ingredients list to make sure that fats in the foods are oils containing primarily unsaturated fatty acids and that solid fats are not one of the first few ingredients…. Examples of solid fats that may be used in the ingredients list are provided in Table A4-1.” But, of course, labels don’t list “solid fats.”  Got it? To find out if foods contain solid fats, see if it says solid fats on the ingredient list, but since labels don’t list “solid fats” you only have to use Table A4-1 which is shown below.  The table, however, has some surprising entries.  The presence of cream and vegetable oils certainly seems a little strange.  Not to worry.  It is easy to understand if you remember that those oils “are high in saturated fat…therefore, for nutritional purposes, these oils are considered solid fats.”  In other words: when is a solid not a solid?  When the USDA says so.  Not to milk this idea too much, we have to figure out how the consumer is to deal with the fact that “fat in fluid milk also is considered to be solid fat; milk fat (butter) is solid at room temperature but is suspended in fluid milk by the process of homogenization (p. 27).”  Apparently, the simple process of distilling the fluid from milk is all that is required for the consumer to find out whether the fat is solid.

For Mimi, this is not a problem because, the fat has been reduced or removed from the milk that she is offered at school and if there is some added sugar  (8 oz. Nesquik® has 29 g.) at least we know that removal of the fat will make the milk more nutrient dense (p. 5).  In a future post, I will explain how removing nutrients can make food more nutrient dense. For the moment, I think understanding that sometimes a liquid is a solid is enough new information.  Now, for the evidence.

“That which is not”

In one of the countries visited in Gulliver’s Travels, the inhabitants referred to “That which is not” because their language did not have a word for lying.  Now the Guidelines prides itself on transparency and evidence-based conclusions so, at the meeting on saturated fat in Detroit, I asked the representative from the USDA where the evidence was on solid fat.  The literature is large but as far as I knew, there were no studies at all that specifically looked at solid fat. He said he would get back to me on that. That was a while ago.

So in the first grade, we learned that solids hold their shape.  My question: are the USDA Dietary Guidelines solid science?

Nutrition & Metabolism Society

“Dost thou think, because thou art virtuous, there shall be no more cakes and ale?”

— William Shakespeare, Twelfth Night.

Experts on nutrition are like experts on sexuality.  No matter how professional they are in general, in some way they are always trying to justify their own lifestyle.  They share a tendency to think that their own lifestyle is the one that everybody else should follow and they are always eager to save us from our own sins, sexual or dietary. The new puritans want to save us from red meat. It is unknown whether Michael Pollan’s In Defense of Food was reporting the news or making the news but it’s coupling of not eating too much and not eating meat is common.  More magazine’s take on saturated fat was very sympathetic to my own point of view and I probably shouldn’t complain that tacked on at the end was the conclusion that “most physicians will probably wait for more research before giving you carte blanche to order juicy porterhouse steaks.” I’m not sure that my physician knows about the research that already exists or that I am waiting for his permission on a zaftig steak.

Daily Red Meat Raises Chances Of Dying Early” was the headline in the Washington Post last year. This scary story was accompanied by the photo below. The gloved hand slicing roast beef with a scalpel-like instrument was probably intended to evoke CSI autopsy scenes, although, to me, the beef still looked pretty good if slightly over-cooked.  I don’t know the reporter, Rob Stein, but I can’t help feeling that we’re not talking Woodward and Bernstein here.  For those too young to remember Watergate, the reporters from the Post were encouraged to “follow the money” by Deep Throat, their anonymous whistle-blower. A similar character, claiming to be an insider and  identifying himself or herself as “Fat Throat,” has been sending intermittent emails to bloggers, suggesting that they “follow the data.”

The Post story was based on a research report “Meat Intake and Mortality” published in the medical journal, Archives of Internal Medicine by Sinha and coauthors.  It got a lot of press and had some influence and recently re-surfaced in the Harvard Men’s Health Watch in a two part article called, incredibly enough, “Meat or beans: What will you have?” (The Health Watch does admit that “red meat is a good source of iron and protein and…beans can trigger intestinal gas” and that they are “very different foods”) but somehow it is assumed that we can substitute one for the other.

Let me focus on Dr. Sinha’s article and try to explain what it really says.  My conclusion will be that there is no reason to think that any danger of red meat has been demonstrated and I will try to point out some general ways in which one can deal with these kinds of reports of scientific information.

A few points to remember first.  During the forty years that we describe as the obesity and diabetes epidemic, protein intake has been relatively constant; almost all of the increase in calories has been due to an increase in carbohydrates; fat, if anything, went down. During this period, consumption of almost everything increased.  Wheat and corn, of course went up.  So did fruits and vegetables and beans.  The two things whose consumption went down were red meat and eggs.  In other words there is some a priori reason to think that red meat is not a health risk and that the burden of proof should be on demonstrating harm.  Looking ahead, the paper, like analysis of the population data, will rely entirely on associations.

The conclusion of the study was that “Red and processed meat intakes were associated with modest increases in total mortality, cancer mortality, and cardiovascular disease mortality.”  Now, modest increase in mortality is a fairly big step down from “Dying Early,” and surely a step-down from the editorial quoted in the Washington Post.  Written by Barry Popkin, professor of global nutrition at the University of North Carolina it said: “This is a slam-dunk to say that, ‘Yes, indeed, if people want to be healthy and live longer, consume less red and processed meat.'” Now, I thought that the phrase “slam-dunk” was pretty much out after George Tenet, then head of the CIA, told President Bush that the Weapons of Mass Destruction in Iraq was a slam-dunk.  (I found an interview with Tenet after his resignation quite disturbing; when the director of the CIA can’t lie convincingly, we are in big trouble).  And quoting Barry Popkin is like getting a second opinion from a member of the “administration.” It’s definitely different from investigative reporting like, you know, reading the article.

So what does the research article really say?  As I mentioned in my blog on eggs, when I read a scientific paper, I look for the pictures. The figures in a scientific paper usually make clear to the reader what is going on — that is the goal of scientific communication.  But there are no figures.  With no figures, Dr. Sinha’s research paper has to be analyzed for what it does have: a lot of statistics.  Many scientists share Mark Twain’s suspicion of statistics, so it is important to understand how it is applied.  A good statistics book will have an introduction that says something like “what we do in statistics, is try to put a number on our intuition.”  In other words, it is not really, by itself, science.  It is, or should be, a tool for the experimenter’s use. The problem is that many authors of papers in the medical literature allow statistics to become their master rather than their servant: numbers are plugged into a statistical program and the results are interpreted in a cut-and-dried fashion with no intervention of insight or common sense. On the other hand, many medical researchers see this as an impartial approach. So let it be with Sinha.

What were the outcomes? The study population of 322, 263 men and 223, 390 women was broken up into five groups (quintiles) according to meat consumption, the highest taking in about 7 times as much as the lower group (big differences).  The Harvard News Letter says that the men who ate the most red meat had a 31 % higher death rate than the men who ate the least meat.  This sounds serious but does it tell you what you want to know? In the media, scientific results are almost universally reported this way but it is entirely misleading.  (Bob has 30 % more money than Alice but they may both be on welfare). To be fair, the Abstract of the paper itself reported this as a hazard ratio of 1.31 which, while still misleading, is less prejudicial. Hazard ratio is a little bit complicated but, in the end, it is similar to odds ratio or risk ratio which is pretty much what you think: an odds ratio of 2 means you’re twice as likely to win with one strategy as compared to the other.  A moment’s thought tells you that this is not good information because you can get an odds ratio of 2, that is, you can double your chances of winning the lottery, by buying two tickets instead of one.  You need to know the actual odds of each strategy.  Taking the ratio hides information.  Do reporters not know this?  Some have told me they do but that their editors are trying to gain market share and don’t care.  Let me explain it in detail.  If you already understand, you can skip the next paragraph.

A trip to Las Vegas

Taking the hazard ratio as more or less the same as odds ratio or risk ratio, let’s consider applying odds (in the current case, they are very similar).  So, we are in Las Vegas and it turns out that there are two black-jack tables and, for some reason (different number of decks or something), the odds are different at the two tables (odds are ways of winning divided by ways of not winning).  Table 1 pays out on average once every 100 hands.  Table 2 pays out once in 67 hands. The odds are 1/99 or close to one in a hundred at the first table and 1/66 at the second.  The odds ratio is, obviously the ratio of the two odds or 1/66 divided by 1/99 or about 1.55.  (The odds ratio would be 1 if there were no difference between the two tables).

Right off, something is wrong: if you were just given the odds ratio you would have lost some important  information.  The odds ratio tells you that one gambling table is definitely better than the other but you need additional information to find out that the odds aren’t particularly good at either table: technically, information about the absolute risk was lost.

So knowing the odds ratio by itself is not much help.  But since we know the absolute risk of each table, does that help you decide which table to play?  Well, it depends who you are. For the guy who is at the blackjack table when you go up to your hotel room to go to sleep and who is still sitting there when you come down for the breakfast buffet, things are going to be much better off at the second table.  He will play hundreds of hands and the better odds ratio of 1.5 will pay off in the long run.  Suppose, however, that you are somebody who will take the advice of my cousin the statistician who says to just go and play one hand for the fun of it, just to see if the universe really loves you (that’s what gamblers are really trying to find out).  You’re going to play the hand and then, win or lose, you are going to go do something else.  Does it matter which table you play at?  Obviously it doesn’t.  The odds ratio doesn’t tell you anything useful because you know that your chances of winning are pretty slim either way.

Now going over to the Red Meat article the hazard ratio (again, roughly the odds ratio) between high and low red meat intakes for all-cause mortality for men, for example, is 1.31 or, as they like to report in the media 31 % higher risk of dying which sounds pretty scary.  But what is the absolute risk?  To find that we have to find the actual number of people who died in the high red meat quintile and the low end quintile.  This is easy for the low end: 6,437 people died from the group of  64,452, so the probability (probability is ways of winning divided by total possibilities) of dying are 6,437/64,452 or just about 0.10 or 10 %.  It’s a little trickier for the high red meat consumers.  There, 13,350 died.  Again,  dividing that by the number in that group, we find an absolute risk of 0.21 or 21 % which seems pretty high and the absolute difference in risk is an increase of 10 % which still seems pretty significant.  Or is it?  In these kinds of studies, you have to ask about confounders, variables that might bias the results.  Well, here, it is not hard to find.  Table 1 reveals that the high red meat group had 3 times the number of smokers. (Not 31 % more but 3 times more).  So the authors corrected the data for this and other effects (education, family history of cancer, BMI, etc.) which is how the final a value of 1.31 was obtained.  Since we know the absolute value of risk in the lowest red meat group, 0.1 we can calculate the risk in the highest red meat group which will be 0.131.  The absolute increase in risk from eating red meat, a lot more red meat, is then 0.131 – 0.10 = 0.031 or 3.1 % which is quite a bit less than we thought.

Now, we can see that the odds ratio of 1.31 is not telling us much — and remember this is for big changes, like 6 or 7 times as much meat; doubling red meat intake (quintiles 1 and 2) leads to a hazard ratio of 1.07.  What is a meaningful odds ratio?  For comparison, the odds ratio for smoking vs not smoking for incidence of lung disease is about 22.

Well, 3.1 % is not much but it’s something.  Are we sure?  Remember that this is a statistical outcome and that means that some people in the high red meat group had lower risk, not higher risk.  In other words, this is what is called statistically two-tailed, that is, the statistics reflect changes that go both ways.  What is the danger in reducing meat intake.  The data don’t really tell you that.  Unlike cigarettes, where there is little reason to believe that anybody’s lungs really benefit from cigarette smoke (and the statistics are due to random variation), we know that there are many benefits to protein especially if it replaces carbohydrate in the diet, that is, the variation may be telling us something real.  With odds ratios around 1.31 — again, a value of 1 means that there is no difference — you are almost as likely to benefit from adding red meat as you are reducing it.  The odds still favor things getting worse but it really is a risk in both directions. You are at the gaming tables.  You don’t get your chips back. If reducing red meat does not reduce your risk, it may increase it.  So much for the slam dunk.

What about public health? Many people would say that for a single person, red meat might not make a difference but if the population reduced meat by half, we would save thousands of lives.  The authors do want to do this.  At this point, before you and your family take part in a big experiment to save health statistics in the country, you have to ask how strong the relations are.  To understand the quality of the data, you must look for things that would not be expected to have a correlation.  “There was an increased risk associated with death from injuries and sudden death with higher consumption of red meat in men but not in women.”  The authors dismiss this because the numbers were smaller (343 deaths) but the whole study is about small differences and it sounds like we are dealing with a good deal of randomness.  Finally, the authors set out from the start to investigate red meat.  To be fair, they also studied white meat which was slightly beneficial. But what are we to compare the meat results to? Why red meat?  What about potatoes?  Cupcakes?   Breakfast cereal?  Are these completely neutral? If we ran these through the same computer, what would we see?  And finally there is the elephant in the room: carbohydrate. Basic biochemistry suggests that a roast beef sandwich may have a different effect than roast beef in a lettuce wrap.

So I’ve given you the perspective of a biochemistry professor.  This was a single paper and surely not the worst, but I think it’s not really about science.  It’s about sin.

*

Nutrition & Metabolism Society