Archive for the ‘The Nutrition Story’ Category

Intention-to-treat II: Foster, et al: Weight and Metabolic Outcomes After 2 Years

Posted: August 28, 2011 in Evidence Based Medicine, Intention to Treat, low-carbohydrate diet, The Nutrition Story
Tags: , , , , , ,
8

“Doctors prefer large studies that are bad to small studies that are good.”

— anon.

The paper by Foster and coworkers entitled Weight and Metabolic Outcomes After 2 Years on a Low-Carbohydrate Versus Low-Fat Diet, published in 2010, had a surprisingly limited impact, especially given the effect of their first paper in 2003 on a one-year study.  I have described the first low carbohydrate revolution as taking place around that time and, if Gary Taubes’s article in the New York Times Magazine was the analog of Thomas Paine’s Common Sense, Foster’s 2003 paper was the shot hear ’round the world.

The paper showed that the widely accepted idea that the Atkins diet, admittedly good for weight loss, was a risk for cardiovascular disease, was not true.  The 2003 Abstract said “The low-carbohydrate diet was associated with a greater improvement in some risk factors for coronary heart disease.” The publication generated an explosive popularity of the Atkins diet, ironic in that Foster had said publicly that he undertook the study in order to “once and for all,” get rid of the Atkins diet.  The 2010 paper by extending the study to 2 years would seem to be very newsworthy.  So what was wrong?  Why is the new paper more or less forgotten?  Two things.  First, the paper was highly biased and its methods were so obviously flawed — obvious even to the popular press — that it may have been a bit much even for the media. It remains to be seen whether it will really be cited but I will suggest here that it is a classic in misleading research and in the foolishness of intention-to-treat (ITT).

(more…)

Intention-to-treat. What it is and why you should care.

Posted: August 21, 2011 in Evidence Based Medicine, Intention to Treat, The Nutrition Story, vitamin E
Tags: , , , , ,
10

Asher Peres was a physicist, an expert in information theory who died in 2005 and was remembered for his scientific contributions as well as for his iconoclastic wit and ironic aphorisms. One of his witticisms was that “unperformed research has no results ”  Peres had undoubtedly never heard of intention-to-treat (ITT), the strange statistical method that has appeared recently, primarily in the medical literature.  According to ITT, the data from a subject assigned at random to an experimental group must be included in the reported outcome data for that group even if the subject does not follow the protocol, or even if they drop out of the experiment.  At first hearing, the idea is counter-intuitive if not completely idiotic  – why would you include people who are not in the experiment in your data? – suggesting that a substantial burden of proof rests with those who want to employ it.  No such obligation is usually met and particularly in nutrition studies, such as comparisons of isocaloric weight loss diets, ITT is frequently used with no justification and sometimes demanded by reviewers.   Not surprisingly, there is a good deal of controversy on this subject.  Physiologists or chemists, hearing this description usually walk away shaking their head or immediately come up with one or another obvious reductio ad absurdum, e.g. “You mean, if nobody takes the pill, you report whether or not they got better anyway?” That’s exactly what it means.

On the naive assumption that some people really didn’t understand what was wrong with ITT — I’ve been known to make a few elementary mistakes in my life — I wrote a paper on the subject.  It received negative, actually hostile. reviews from two public health journals — I include an amusing example at the end of this post.  I even got substantial grief from Nutrition & Metabolism, where I was the editor at the time, but where it was finally published.  The current post will be based on that paper and I will provide a couple of interesting cases from the medical literature.  In the next post I will discuss a quite remarkable new instance — Foster’s two year study of low carbohydrate diets — of the abuse of common sense that is the major alternative to ITT.

To put a moderate spin on the problem, there is nothing wrong with ITT, if you explicitly say what the method shows — the effect of assigning subjects to an experimental protocol; the title of my paper was Intention-to-treat.  What is the question? If you are very circumspect about that question, then there is little problem.  It is common, however, for the Abstract of a paper to correctly state that patients “were assigned to a diet” but by the time the Results are presented, the independent variable has become, not “assignment to the diet,” but “the diet” which most people would assume meant what people ate, rather than what they were told to eat. Caveat lector.  My paper was a kind of over-kill and I made several different arguments but the common sense argument gets to the heart of the problem in a practical way.  I’ll describe that argument and also give a couple of real examples.

Common sense argument against intention-to-treat

Consider an experimental comparison of two diets in which there is a simple, discrete outcome, e.g. a threshold amount of weight lost or remission of an identifiable symptom. Patients are randomly assigned to two different diets: diet group A or diet group B and a target of, say, 5 kg weight loss is considered success. As shown in the table above, in group A, half of the subject are able to stay on the diet but, for whatever reason, half are not. The half of the patients in group A who did stay on the diet, however, were all able to lose the target 5 kg.  In group B, on the other hand, everybody is able to stay on the diet but only half are able to lose the required amount of weight. An ITT analysis shows no difference in the two outcomes, while just looking at those people who followed the diet shows 100 % success.  This is one of the characteristics of ITT: it always makes the better diet look worse than it is.

         Diet A         Diet B
Compliance (of 100 patients)   50   100
Success (reached target)   50    50
ITT success   50/100 = 50%   50/100 = 50%
“per protocol” (followed diet) success   50/50 = 100%   50/100 = 50%

Now, you are the doctor.  With such data in hand should you advise a patient: “well, the diets are pretty much the same. It’s largely up to you which you choose,” or, looking at the raw data (both compliance and success), should the recommendation be: “Diet A is much more effective than diet B but people have trouble staying on it. If you can stay on diet A, it will be much better for you so I would encourage you to see if you could find a way to do so.” Which makes more sense? You’re the doctor.

I made several arguments trying to explain that there are two factors, only one of which (whether it works) is clearly due to the diet. The other (whether you follow the diet) is under control of other factors (whether WebMD tells you that one diet or the other will kill you, whether the evening news makes you lose your appetite, etc.)  I even dragged in a geometric argument because Newton had used one in the Principia: “a 2-dimensional outcome space where the length of a vector tells how every subject did…. ITT represents a projection of the vector onto one axis, in other words collapses a two dimensional vector to a one-dimensional vector, thereby losing part of the information.” Pretentious? Moi?

Why you should care.  Case I. Surgery or Medicine?

Does your doctor actually read these academic studies using ITT?  One can only hope not.  Consider the analysis by Newell  of the Coronary Artery Bypass Surgery (CABS) trial.  This paper is astounding for its blanket, tendentious insistence on what is correct without any logical argument.  Newell considers that the method of

 “the CABS research team was impeccable. They refused to do an ‘as treated’ analysis: ‘We have refrained from comparing all patients actually operated on with all not operated on: this does not provide a measure of the value of surgery.”

Translation: results of surgery do not provide a measure of the value of surgery.  So, in the CABS trial, patients were assigned to Medicine or Surgery. The actual method used and the outcomes are shown in the Table below. Intention-to-treat analysis was, as described by Newell, “used, correctly.” Looking at the table, you can see that a 7.8% mortality was found in those assigned to receive medical treatment (29 people out of 373 died), and a 5.3% mortality (21 deaths out of 371) for assignment to surgery.  If you look at the outcomes of each modality as actually used, it turns out that that medical treatment had a 9.5% (33/349) mortality rate compared with 4.1% (17/419) for surgery, an analysis that Newell says “would have wildly exaggerated the apparent value of surgery.”

Survivors and deaths after allocation to surgery or medical treatment
Allocated medicine Allocated surgery
  Received surgery     Received medicine   Received surgery     Received medicine
Survived 2 years   48   296   354   20
Died    2    27    15    6
Total   50   323   369   26

Common sense suggests that appearances are not deceiving. If you were one of the 33-17 = 16 people who were still alive, you would think that it was the potential report of your death that had been exaggerated.  The thing that is under the control of the patient and the physician, and which is not a feature of the particular modality, is getting the surgery implemented. Common sense dictates that a patient is interested in surgery, not the effect of being told that surgery is good.  The patient has a right to expect that if they comply, the physician would avoid conditions where, as stated by Hollis,  “most types of deviations from protocol would continue to occur in routine practice.” The idea that “Intention to treat analysis is … most suitable for pragmatic trials of effectiveness rather than for explanatory investigations of efficacy” assumes that practical considerations are the same everywhere and that any practitioner is locked into the same abilities or lack of abilities as the original experimenter.

What is the take home message.  One general piece of advice that I would give based on this discussion in the medical literature: don’t get sick.

Why you should care.  Case II. The effect of vitamin E supplementation

A clear cut case of how off-the-mark ITT can be is a report on the value of antioxidant supplements. The Abstract of the paper concluded that “there were no overall effects of ascorbic acid, vitamin E, or beta carotene on cardiovascular events among women at high risk for CVD.” The study was based on an ITT analysis but,on the fourth page of the paper, it turns out that removing subjects due to

“noncompliance led to a significant 13% reduction in the combined end point of CVD morbidity and mortality… with a 22% reduction in MI …, a 27% reduction in stroke …. a 23% reduction in the combination of MI, stroke, or CVD death (RR (risk ratio), 0.77; 95% CI, 0.64–0.92 [P = 005]).”

The media universally reported the conclusion from the Abstract, namely that there was no effect of vitamin E. This conclusion is correct if you think that you can measure the effect of vitamin E without taking the pill out of the bottle.  Does this mean that vitamin E is really of value? The data would certainly be accepted as valuable if the statistics were applied to a study of the value of replacing barbecued pork with whole grain cereal. Again, “no effect” was the answer to the question: “what happens if you are told to take vitamin E” but it still seems is reasonable that the effect of a vitamin means the effect of actually taking the vitamin.

The ITT controversy

Advocates of ITT see its principles as established and may dismiss a common sense approach as naïve. The issue is not easily resolved; statistics is not axiomatic: there is no F=ma, there is no zeroth law.  A good statistics book will tell you in the Introduction that what we do in statistics is to try to find a way to quantify our intuitions. If this is not appreciated, and you do not go back to consideration of exactly what the question is that you are asking, it is easy to develop a dogmatic approach and insist on a particular statistic because it has become standard.

As I mentioned above, I had a good deal of trouble getting my original paper published and one  anonymous reviewer said that “the arguments presented by the author may have applied, maybe, ten or fifteen years ago.” This criticism reminded me of Molière’s Doctor in Spite of Himself:

Sganarelle is disguised as a doctor and spouts medical double-talk with phony Latin, Greek and Hebrew to impress the client, Geronte, who is pretty dumb and mostly falls for it but:

Geronte: …there is only one thing that bothers me: the location of the liver and the heart. It seemed to me that you had them in the wrong place: the heart is on the left side but the liver is on the right side.

Sgnarelle: Yes. That used to be true but we have changed all that and medicine uses an entirely new approach.

Geronte: I didn’t know that and I beg your pardon for my ignorance.

 In the end, it is reasonable that scientific knowledge be based on real observations. This has never before been thought to include data that was not actually in the experiment. I doubt that nous avons changé tout cela.

The Office of Research Integrity Conference and the Crisis in Nutrition.

Posted: August 15, 2011 in Lipophobes, low-carbohydrate diet, Research Integrity, The Nutrition Story
12

The headline in the BBC News is “Fat ‘disrupts sugar sensors causing type 2 diabetes’” The article does not attribute the quotation in the headline and the first sentence says “US researchers say they have identified how a high-fat diet can trigger type 2 diabetes, in experiments on mice and human tissue.”  Should “mice” or “tissue” have been in the headline?  Should the article itself point out the extent to which mice respond differently, sometimes, oppositely from humans, to high-fat diets?  How strong is the evidence in light of other work?  Is the article altogether prejudicing the reader against fat which is the official position of both private and government health agencies?  The article in question may have some sins of omission but it is certainly restrained if not actually circumspect. The general problem, of course, is whether we get accurate scientific information from the popular media.

Peter Farnham and I and a group of bloggers (Laura Dolszon, Tom Naughton and Jimmy Moore) will speak at the end of the month at a conference produced by the Office of Research Integrity (ORI)) where we will raise several issues in the ethical conduct of current nutritional research. The conference, in general, tries to explore a number of questions on the interaction of science and  society.  The goals are to “discuss the latest research on research integrity…education in the responsible conduct of research; responsible research practices.” While each presenter will have only 15 minutes, this is one of the first times thatd the practices in nutrition with regard to issues of integrity are being addressed.  There are four areas that we will discuss:

Crisis in Nutrition I: The Popular Media and Research Publications

Richard David Feinman, SUNY Downstate Medical Center

Crisis in Nutrition II: Research Integrity in Meeting the Challenge  of Carbohydrate Restriction

Richard David Feinman, SUNY Downstate  Medical Center

Crisis in Nutrition III:  Was the Government Standard Met by the  2010 Dietary Guidelines?

Peter Farnham, Nutrition and Metabolism  Society, Alexandria, VA

Crisis in Nutrition IV:  Vox Populi 

Tom Naughton, Jimmy Moore, Laura Dolson, Independent Writers Franklin TN

The abstract of my first talk is presented below. It is, of course, a tricky area. Within some legal limits, reporters can say what they like.  A researcher speaking in a public venue, personal blog, social media can similarly pretty much sound of as they choose.  Or can they? If they are identified as an expert or are have credentials based on a employment by a prestigious institution, don’t they have to clearly distinguish between opinion and fact? And does the headline have to say that, for example, the high-fat study was done in mice?  All of these are gray areas and motives are hard to discern.  I focussed on one area that seemed more clear cut.  If an experimental study is reported in the media or in a press release from an academic institution (sometimes the same thing), is there an obligation to be sure that any opinions attributed to the investigator derives from that research unless otherwise indicated?

 Crisis in nutrition: I. The popular media and research publications  

Objective:

The public relies on popular media for descriptions of nutrition research.  Of particular interest is carbohydrate-restricted diets, the major challenge to official recommendations.  The goal is to assess the extent to which statements to the media and press releases accurately represent the results of research.

Summary of findings or main points:

Nutrition is an area of great interest to the public but one where matters of scientific fact and policy are contentious. Authors of research papers should sensibly have great freedom in describing of the implications of their research, but have an important role in explaining to the public when that opinion does or does not follow directly from the publication.  Two examples are given of where this is a critical issue. In one, an animal study (Foo, et al. Proc Natl Acad Sci USA 2009, 106: 15418-15423), the accompanying press release implies that it was motivated by observations of patients in a hospital, observations which were purely anecdotal and unsubstantiated.  In a second example, a press release stated that carbohydrate-restricted diets (CRDs) were not included in a comparative study because of their low compliance (Sacks, et al. N Engl J Med 2009, 360: 859-873). No data were given to support this allegation and, it is, in fact not true.  The study concluded that the macronutrient composition of the diet was not important even though, as implemented, dietary intake was the same for the groups studied and, again, the CRD was not included in the study.  It seems likely that that this would have an inhibiting effect on individuals choosing a CRD and represents an important impact of research integrity issues on the community.

Conclusions & recommendations:

Practices where research directly affects the community should be evaluated and guidelines should be generated by academic societies, scientific journals and the popular media. What constitutes appropriate press description of published research should be defined. Reasonable principles are that only those specific conclusions that derive directly from the publication are relevant and authors make clear what is their personal opinion and what is the product of research data.

 Office of Research Integrity

It is important to emphasize that ORI is sponsor of this academic conference and is not related to is regulatory function.  ORI is charged with overseeing federally funded research and emphasizes its role as watchdog in detection and prevention of research misconduct, assisting the Office of the General Counsel (OGC), dealing with suspected retaliation against whistleblowers, and responding to Freedom of Information and Privacy Acts.  Its usual activity involves pinpointing specific fraud.  The website reports, for example, a final judgement against an Assistant Professor at the Boston University School of Medicine Cancer Research Center who published two papers in which he had fabricated data shown as figures in the paper.  He is required to retract the papers and not enter into contracts or sit on advisory panels for two years.

ORI, in general, has the same relation to the research community that Internal Affairs has to the police.  Of course, in research, although there are substantive rewards, blatant fraud is generally pathological: if the results you are falsifying are important, they will surely be repeated and the misbehavior discovered while, if they are not important, the rewards are not likely to be spectacular although, especially these days, keeping your job is desirable.  The suspicion about ORI is also bolstered by their behavior in the Baltimore case in which they were part of the mindless zeal and witch-hunting whose appearance in human interactions seems to have such a low activation energy.

David Baltimore, a Nobel laureate in Physiology or Medicine and, at the time, MIT professor, had co-authored a paper with an immunologist named  Thereza Imanishi-Kari who was accused by a postdoctoral fellow of fabricating data. In the end, nothing came of it but there was much sound and fury and many idiots participated in the tale including the Secret Service (who I was taught were only supposed to protect the President and prosecute counterfeiters). The details of the Baltimore case are well told in capsule form in the Wikipedia entry. Daniel Kevles wrote an outstanding book, at least judged by the first half — the content was too infuriating for me to keep reading.  In any case, after most of the furor had died down, the ORI persisted and found Imanishi-Kari guilty of research misconduct, a ruling overturned by an appeals panel of the Department of Health and Human Services (HHS) which “found that much of what ORI presented was irrelevant, had limited probative value, was internally inconsistent, lacked reliability or foundation, was not credible or not corroborated, or was based on unwarranted assumptions.”  the ORI reputation has probably not recovered from this but it remains one of the few oversight agencies which, at least in nutrition, is sorely needed.

David Baltimore wrote his own description of the events and emphasized that it raise many questions, in particular,  “Who should judge science?” and “How does one distinguish between error and fraud? And does science do an adequate job of policing itself?”  The conference and this blog will discuss these matters which the crisis in nutrition has made of critical importance.  Such philosophical questions were formerly what most of us would have preferred to simply gab about in Starbucks.

Fair and Foul on the Diabetes Scene. Diabetes Health beats Jay Leno but the ADA takes the Cake.

Posted: July 14, 2011 in American Diabetes Association, Dietary Fiber, The Nutrition Story
Tags: , , , , , ,
3

“Headlines” is one of Jay Leno’s routines on The Tonight Show. While low on production values, it provides amusing typos, odd juxtapositions of text and inappropriate couplings from real notices and newspapers. The headlines are frequently very funny since, like fiction in general, authored comedy has to be plausible. There have been many other versions of the same idea including items in the New Yorker but Jay Leno’s audience rapport adds to the impact. Expert as he is, though, Jay seemed a little off guard when nobody laughed at the headline: “The Diabetes Discussion Group will meet at 10 AM right after the pancake breakfast.” It’s probably generational. After 30 or so years having the American Diabetes Association tell you that sugar is Ok as long as you “cover it with insulin” and that diabetes, a disease of carbohydrate intolerance, is best treated by adding carbohydrate and reducing fat, who knows what anybody believes.

One of the headlines on a previous show that did get a laugh said: “To increase gas mileage, drive less.”  (If Jay only knew how much we spent to get the USDA committee to come up with the advice that if you want to lose weight, you should eat less).

“.. Have we eaten on the insane root,
That takes the reason prisoner?”
— William Shakespeare, Macbeth.

For tragic humor in the bizarre field of diabetes information, it is really hard to compete. About the same time as the headlines sequence on the Tonight Show, DiabetesHealth  an organization and website that is intended to “investigate, inform, inspire” produced an inspiring investigation from the literature. The story is entitled “Maple Syrup – A Sweet Surprise.”  You gotta’ read this:

 “Meet the latest superfood: maple syrup.  Wait a minute…maple syrup? The super-sugary stuff poured on pancakes and waffles and used to glaze hams? That maple syrup? That’s right. Researchers from the University of Rhode Island have discovered that the syrup-produced in the northeastern United States and Canada–contains numerous compounds with real health benefits.”

So how did people with diabetes fare on the maple syrup? Well, there were no people. Or animals. The researchers did not test the effect of consumed maple syrup but only chemically analyzed samples of the stuff.

“‘In our laboratory research, we found that several of these compounds possess anti-oxidant and anti-inflammatory properties, which have been shown to fight cancer, diabetes, and bacterial illnesses,’ said Navindra Seeram, an assistant professor of pharmacognosy (the study of medicines derived from natural sources) at the university and the study’s lead author”

“Pharmacognosy,” incidentally, is the only English word correctly pronounced through the nose.  The article indicates that “a paper describing their results will appear in the Journal of Functional Foods. Scientists hope that these discoveries could lead to innovative treatments as the beneficial substances are synthesized to create new kinds of medicine.”  The article, however, is nothing if not circumspect:

“You might want to pause for a moment before rushing out and buying jug after jug of Canada’s finest maple syrup, though. It still contains plenty of sugar,…” In fact, by far the major ingredient in maple syrup is sucrose which, again, only has to be “covered” with insulin. So, with all those beneficial compounds, we will need less insulin per gram of sucrose with maple syrup, right?    Would Jay Leno have gotten a laugh if the diabetes meeting followed the pancakes and maple syrup breakfast?  How about if they were whole grain pancakes?

“If you can look into the seeds of time,
And say which grain will grow, and which will not…”
— William Shakespeare, Macbeth.

Not to be outdone, the American Diabetes Association website offers the lowdown on just how good grain is. Fiber, in general, is so good for you that you should be careful not to snarf it up too fast. As they point out, it is “important that you increase your fiber intake gradually, to prevent stomach irritation, and that you increase your intake of water and other liquids, to prevent constipation.” Doesn’t really sound all that healthy but foods with fiber “have a wealth of nutrition, containing many important vitamins and minerals.” Now, vitamin deficiency has always seemed to me to be the least of our nutritional problems but there’s more: “In fact,” using fact in its non-traditional meaning, fiber “may contain nutrients that haven’t even been discovered yet!” (their exclamation point). Not to belabor all the metaphors here, the ADA, long telling us that people with diabetes deserve to have their carbs, are surely offering pie in the sky.

The Warshaw-Diabetes Health Thing – Invitation to Discuss.

Posted: July 8, 2011 in low-carbohydrate diet, The Nutrition Story, USDA Dietary Guidelines
8

The whacko suggestion by Hope Warshaw on DiabetesHealth that people with diabetes should increase their carbohydrate intake — I don’t know whether she was serious or just trying to infuriate — obviously generated a rather large response, especially on the DiabetesHealth website itself.  I was writing my own post on the issue when the editor Nadia Al-Samarrie published a piece which seems to have added to the discord. I decided to bypass the argument and I posted the following letter to her suggesting a way to introduce more information and fewer bad vibes.

==============================================================

Dear Nadia,

I understand that publishing a popular site requires one to be provocative and I think you can see that many people had a strong response to Hope Warshaw’s article and your response.  I think you will agree however that this is a serious matter and I want to suggest a mechanism for bringing the science out for the general public.  I am suggesting a discussion between opposing points of view, less a debate that than a presentation of facts although one implementation might be to have a kind of jury of impartial scientists to present summaries.  I would suggest that you and I be organizers and if DiabetesHealth would be one of the sponsors, I feel sure that I would be able to provide other sponsors. It would, of course, be imperative for the American Diabetes Association and the USDA Advisory committee to participate (send or endorse discussants) to establish that recommendations for people with diabetes conform to some kind of “sunshine law.”

The details of such a meeting could be worked out but as a starting point, I would suggest something along the lines of the following.

There would be two panels, one who maintains that a low-carbohydrate diet (definitions to be agreed upon in advance) is the default diet, that is, the one to try first, for both type 1 and type 2 diabetes and metabolic syndrome.  The other would conform to the very restricted view on such diets (only for weight loss, concerns about heart disease or kidney disease or whatever).

There would be, say, four representatives on each panel endorsed, again, by the ADA and USDA, DiabetesHealth and by the Nutrition and Metabolism Society.

Because of the voluminous literature, each side would specify ten papers in the literature, popular writings or book sections (max 30 pages each).  Discussion would be restricted to these sources.

Participants would meet before hand to set up preliminary procedures to avoid a free-for-all or any “defenestration.”

Variations might include a second day in which both panels took questions from the public or press.

I feel sure that such a meeting would go a long way towards reducing the palpable bad feelings and I am sure you agree that the enemy is diabetes and related diseases and not people with other opinions.  I would be glad to discuss, on the phone, how we can get started.

Best Regards,

Richard David Feinman

No Sunshine Down Under: Comments on Larsen, et al.

Posted: June 28, 2011 in Association and Causality, Evidence Based Medicine, Intention to Treat, Lipophobes, low-carbohydrate diet, Observational Studies, The Nutrition Story
10

“These results suggest that there is no superior long-term metabolic benefit of a high-protein diet over a high-carbohydrate in the management of type 2 diabetes.”  The conclusion is from a paper by Larsen, et al. [1] which, based on that statement in the Abstract, I would not normally bother to read; it is good that you have to register trials and report failures but from a broader perspective, finding nothing is not great news and just because Larsen couldn’t do it, doesn’t mean it can’t be done.  However, in this case, I received an email from International Diabetes published bilingually in Beijing: “Each month we run a monthly column where choose a hot-topic article… and invite expert commentary opinion about that article” so I agreed to write an opinion. The following is my commentary:

“…no superior long-term metabolic benefit of a high-protein diet over a high-carbohydrate ….” A slightly more positive conclusion might have been that “a high-protein diet is as good as a high carbohydrate diet.”  After all, equal is equal. The article is, according to the authors, about “high-protein, low-carbohydrate” so rather than describing a comparison of apples and pears, the conclusion should emphasize low carbohydrate vs high carbohydrate.   It is carbohydrate, not protein, that is the key question in diabetes but clarity was probably not the idea. The paper by Larsen, et al. [1] represents a kind of classic example of the numerous studies in the literature whose goal is to discourage people with diabetes from trying a diet based on carbohydrate restriction, despite its intuitive sense (diabetes is a disease of carbohydrate intolerance) and despite its established efficacy and foundations in basic biochemistry.  The paper is characterized by blatant bias, poor experimental design and mind-numbing statistics rather than clear graphic presentation of the data. I usually try to take a collegial approach in these things but this article does have a unique and surprising feature, a “smoking gun” that suggests that the authors were actually aware of the correct way to perform the experiment or at least to report the data.

Right off, the title tells you that we are in trouble. “The effect of high-protein, low-carbohydrate diets in the treatment…” implying that all such diets are the same even though  there are several different versions, some of which (by virtue of better design) will turn out to have had much better performance than the diet studied here and, almost all of which are not “high protein.” Protein is one of the more stable features of most diets — the controls in this experiment, for example, did not substantially lower their protein even though advised to do so –and most low-carbohydrate diets advise only carbohydrate restriction.  While low-carbohydrate diets do not counsel against increased protein, they do not necessarily recommend it.  In practice, most carbohydrate-restricted diets are hypocaloric and the actual behavior of dieters shows that they generally do not add back either protein or fat, an observation first made by LaRosa in 1980.

Atkins-bashing is not as easy as it used to be when there was less data and one could run on “concerns.” As low-fat diets continue to fail at both long-term and short-term trials — think Women’s Health Initiative [2] — and carbohydrate restriction continues to show success and continues to bear out the predictions from the basic biochemistry of the insulin-glucose axis  [3], it becomes harder to find fault.  One strategy is to take advantage of the lack of formal definitions of low-carbohydrate diets to set up a straw man.  The trick is to test a moderately high carbohydrate diet and show that, on average, as here, there is no difference in hemoglobin A1c, triglycerides and total cholesterol, etc. when compared to a higher carbohydrate diet as control —  the implication is that in a draw, the higher carbohydrate diet wins.  So, Larsen’s low carbohydrate diet contains 40 % of energy as carbohydrate.  Now, none of the researchers who have demonstrated the potential of carbohydrate restriction would consider 40 % carbohydrate, as used in this study, to be a low-carbohydrate diet. In fact, 40 % is close to what the American population consumed before the epidemic of obesity and diabetes. Were we all on a low carbohydrate diet before Ancel Keys?

What happened?  As you might guess, there weren’t notable differences on most outcomes but like other such studies in the literature, the authors report only group statistics so you don’t really know who ate what and they use an intention-to-treat (ITT) analysis. According to ITT, a research report should include data from those subjects that dropped out of the study (here, about 19 % of each group). You read that correctly.  The idea is based on the assumption (insofar as it has any justification at all) that compliance is an inherent feature of the diet (“without carbs, I get very dizzy”) rather than a consequence of bias transmitted from the experimenter, or distance from the hospital, or any of a thousand other things.  While ITT has been defended vehemently, the practice is totally counter-intuitive, and has been strongly attacked on any number of grounds, the most important of which is that, in diet experiments, it makes the better diet look worse.  Whatever the case that can be made, however, there is no justification for reporting only intention-to-treat data, especially since, in this paper, the authors consider as one of the “strengths of the study … the measurement of dietary compliance.”

The reason that this is all more than technical statistical detail, is that the actual reported data show great variability (technically, the 95 % confidence intervals are large).  To most people, a diet experiment is supposed to give a prospective dieter information about outcome.  Most patients would like to know: if I stay on this diet, how will I do.  It is not hard to understand that if you don’t stay on the diet, you can’t expect good results.  Nobody knows what 81 % staying on the diet could mean.  In the same way, nobody loses an average amount of weight. If you look at  the spread in performance and in what was consumed by individuals on this diet, you can see that there is big individual variation Also, being “on a diet”, or being “assigned to a diet” is very different than actually carrying out dieting behavior, that is, eating a particular collection of food.  When there is wide variation, a person in the low-carb group may be eating more carbs than some person in the high-carb group.  It may be worth testing the effect of having the doctor tell you to eat fewer carbs, but if you are trying to lose weight, you want them to test the effect of actually eating fewer carbs.

When I review papers like this for a journal I insist that the authors present individual data in graphic form.  The question in low-carbohydrate diets is the effect of amount of carbohydrate consumed on the outcomes.  Making a good case to the reader involves showing individual data.  As a reviewer, I would have had the authors plot each individual’s consumption of carbohydrate vs for example, individual changes in triglyceride and especially HbA1c.  Both of these are expected to be dependent on carbohydrate consumption.  In fact, this is the single most common criticism I make as reviewer or that I made when I was co-editor-in chief at Nutrition and Metabolism.

So what is the big deal?  This is not the best presentation of the data and it is really hard to tell what the real effect of carbohydrate restriction is. Everybody makes mistakes and few of my own papers are without some fault or other. But there’s something else here.  In reading a paper like this, unless you suspect that something wasn’t done correctly, you don’t tend to read the Statistical analysis section of the Methods very carefully (computers have usually done most of the work).  In this paper, however, the following remarkable paragraph jumps out at you.  A real smoking gun:

  • “As this study involved changes to a number of dietary variables (i.e. intakes of calories, protein and carbohydrate), subsidiary correlation analyses were performed to identify whether study endpoints were a function of the change in specific dietary variables. The regression analysis was performed for the per protocol population after pooling data from both groups. “

What?  This is exactly what I would have told them to do.  (I’m trying to think back. I don’t think I reviewed this paper).  The authors actually must have plotted the true independent variable, dietary intake — carbohydrate, calories, etc. — against the outcomes, leaving out the people who dropped out of the study.  So what’s the answer?

  • “These tests were interpreted marginally as there was no formal adjustment of the overall type 1 error rate and the p values serve principally to generate hypotheses for validation in future studies.”

Huh?  They’re not going to tell us?  “Interpreted marginally?”  What the hell does that mean?  A type 1 error refers to a false positive, that is, they must have found a correlation between diet and outcome in distinction to what the conclusion of the paper is.  They “did not formally adjust for” the main conclusion?  And “p values serve principally to generate hypotheses?”  This is the catch-phrase that physicians are taught to dismiss experimental results that they don’t like.  Whether it means anything or not, in this case there was a hypothesis, stated right at the beginning of the paper in the Abstract: “…to determine whether high-protein diets are superior to high-carbohydrate diets for improving glycaemic control in individuals with type 2 diabetes.”

So somebody — presumably a reviewer — told them what to do but they buried the results.  My experience as an editor was, in fact, that there are people in nutrition who think that they are beyond peer review and I had had many fights with authors.  In this case, it looks like the actual outcome of the experiment may have actually been the opposite of what they say in the paper.  How can we find out?  Like most countries, Australia has what are called “sunshine laws,” that require government agencies to explain their actions.  There is a Australian Federal Freedom of Information Act (1992) and one for the the state of Victoria (1982). One of the authors is supported by NHMRC (National Health and Medical Research Council)  Fellowship so it may be they are obligated to share this marginal information with us.  Somebody should drop the government a line.

Bibliography

1. Larsen RN, Mann NJ, Maclean E, Shaw JE: The effect of high-protein, low-carbohydrate diets in the treatment of type 2 diabetes: a 12 month randomised controlled trial. Diabetologia 2011, 54(4):731-740.

2. Tinker LF, Bonds DE, Margolis KL, Manson JE, Howard BV, Larson J, Perri MG, Beresford SA, Robinson JG, Rodriguez B et al: Low-fat dietary pattern and risk of treated diabetes mellitus in postmenopausal women: the Women’s Health Initiative randomized controlled dietary modification trial. Arch Intern Med 2008, 168(14):1500-1511.

3. Volek JS, Phinney SD, Forsythe CE, Quann EE, Wood RJ, Puglisi MJ, Kraemer WJ, Bibus DM, Fernandez ML, Feinman RD: Carbohydrate Restriction has a More Favorable Impact on the Metabolic Syndrome than a Low Fat Diet. Lipids 2009, 44(4):297-309.

Metabolic advantage, “a calorie is a calorie,” and why the First Law of Thermodynamics does not apply.

Posted: June 6, 2011 in low-carbohydrate diet, The Nutrition Story, thermodynamics
Tags: , , , ,
32


The big news in the low carb world is that Consumer Reports has published, for the first time, faint praise for the Atkins diet. However, the vision one might have of CR employees testing running shoes on treadmills doesn’t really apply here. They did not put anybody on a diet, even for a day. They didn’t have to. They have the standards from the government. Conform to the USDA Guidelines and CR will give you thumbs up. It probably doesn’t matter since, these days, most people buy a food processor by checking out the reviews on the internet — there are now many reviews online of what it’s like to actually be on a low-carbohydrate diet, so rather than follow CR’s imaginings of what it’s like, you can check out what users say — Jimmy Moore, Tom Naughton and Laura Dolson together get about 1.5 million posts per month with many tests and best buy recommendations. What caught my eye, though, is the ubiquitous Dean Ornish; the ratio of words written about the Ornish diet to the number of people who actually use it is probably closing in on a googol (as it was originally spelled). The article says: “to lose weight, you have to burn up more calories than you take in, no matter what kind of diet you’re on. ‘The first law of thermodynamics still applies,’ says Dean Ornish, M.D.

That’s how I got into this field. My colleague Gene Fine, and I published our first papers in nutrition on the subject of metabolic advantage and thermodynamics and we gave ourselves credit for reducing the number of people invoking laws of thermodynamics. “Metabolic advantage” refers to the idea that you can lose more weight, calorie-for-calorie on a particular diet, usually a low-carbohydrate diet. (The term was used in a paper by Browning to mean the benefits in lipid metabolism of a low-carbohydrate diet, but in nutrition you can re-define anything you want and you don’t have to cite anybody else’s work if you don’t want to). The idea of metabolic advantage stands in opposition to the idea that “a calorie is a calorie” which is, of course, the backbone of establishment nutrition and all our woe. As in the CR article, whenever the data show that a low-carbohydrate diet is more effective for weight loss, somebody always jumps in to say that it would violate the laws of thermodynamics. Those of us who have studied or use thermodynamics recognize that it is a rather difficult subject — somebody called it the physics of partial differential equations — and we’re amazed at how many experts have popped up in the nutrition field.

Finding the right diet doesn’t require knowing much thermodynamics but it is an interesting subject and so I’ll try to explain what it is about and how it’s used in biochemistry. The physics of heat, work and energy, thermodynamics was developed in the nineteenth century in the context of the industrial revolution — how efficiently you could make a steam engine operate was a big deal.  Described by Prigogine as the first revolutionary science, it has some interesting twists and intellectual connections. The key revolutionary concept is embodied tin the second law which describes the efficiency of physical processes.  It has broad philosophical meaning.  The primary concept, the entropy, is also used in communication and  the content of messages in information theory.  The entropy of a message is, in one context, how much a message has been garbled in transmission.  The history of thermodynamics also has some very strange characters, besides me and Gene, so I will try to describe them too.

First, we can settle the question of metabolic advantage, or more precisely, energy inefficiency. The question is whether all of the calories in food are available for weight gain or loss (or exercise) regardless of the composition of the diet. Right off, metabolic advantage is an inherent property of higher protein diets and low carbohydrate diets. In the first case, the thermic effect of feeding (TEF) is a measure of how many of the calories in food are wasted in the process of digestion, absorption, low-level chemical transformation, etc. TEF (old name: specific dynamic action) is well known and well studied. Nobody disputes that the TEF can be substantial for protein, typically 20 % of calories. It is much less for carbohydrate and still less for fat. So, substituting any protein for either of the other macronutrients will lead to energy inefficiency (the calories will be wasted as heat). A second unambiguous point is that in the case of low-carbohydrate diets, in order to maintain blood glucose, the process of gluconeogenesis is required. You learn in biochemistry courses that it requires a good deal of energy to convert protein (the major source for gluconeogenesis) into glucose.

So, right off, metabolic advantage or energy inefficiency is known and measurable. Critics of carb restriction as a strategy admit that it occurs but say that it is too small in a practical sense to be worth considering when you are trying to lose weight. These are usually the same people who tell you that the best way to lose weight is through accumulation of small changes in daily weight loss by reducing 100 kcal a day or something like that. In any case, there is a big difference between things that are not practical or have only small effects and things that are theoretically impossible. If metabolic advantage were really impossible theoretically, that would be it. We could stop looking for the best diet and only calories would count. Since we know energy inefficiency is possible and measurable, shouldn’t we be trying to maximize it.  But what is the story on thermodynamics? What is it? Why do people think that metabolic advantage violates thermodynamics? What is their mistake? More specifically, doesn’t the first law of thermodynamics say that calories are conserved? Well, there is more than one law of thermodynamics and even the first law has to be applied correctly. Let me explain. (Note in passing that the dietary calorie is a physical kilocalorie (kcal; 1000 calories).

There are four laws of thermodynamics. Two are technical. The zeroth law says, in essence, that if two bodies have the same temperature as a third, they have the same temperature as each other. This sounds obvious but, in fact, it is an observational law — it always turns out that way. The law is necessary to make sure everything else is for real. If anybody ever finds an experimental case where it is not true, the whole business will come crashing down. The third law describes what happens at the special condition known as the absolute zero of temperature. In essence, the zeroth and third laws, allow everything else to be calculated and practical thermodynamics like bioenergetics pretty much assumes it in the background.

The second law is what thermodynamics is really about — it was actually formulated before the first law — but since the first law is usually invoked in nutrition, let’s consider this first. The first law is the conservation of energy law. Here’s how it works: thermodynamics considers systems and surroundings. The thing that you are interested in — living system, a single cell, a machine, whatever, is called the system — everything outside is the surroundings or environment. The first law says that any energy lost by the system must be gained by the environment and any energy taken up by the system must have come from the environment. Its application to chemical systems, which is what applies to nutrition, is that we can attribute to chemical systems, a so-called internal energy, usually written with symbol U (so as not to confuse it with the electrical potential, E). In thermodynamics, you usually look at changes, and the first law says that you can calculate ΔU, the change in U of a system, by adding up the changes in heat added to the system and work done by the system (you can see the roots of thermo in heat machines: we add heat and get work). In chemical systems, the energy can also change due to chemical reactions. Still, if you add up all the changes in the system plus the environment, all the heat, work and chemical changes, the energy is neither created nor destroyed. It is conserved.

Now, why doesn’t the first law apply to nutrition the way Ornish thinks it does? To understand this, you have to know what is done in chemical thermodynamics and bioenergetics, (thermo applied to living systems). If you want to. In nutrition, you can make up your own stuff. But, if you want to do what is done in chemical thermodynamics, you focus on the system itself, not the system plus the environment. So, from the standpoint of chemical thermodynamics, the calories in food represent the heat generated by complete oxidation of food in a calorimeter.



In a calorimeter, the food is placed in a small container with oxygen under pressure and ignited. The heat generated is determined from the increase in temperature of the water bath. (Before the food measurement, we determine the heat capacity of the water bath, that is, how much heat it takes to raise the temperature). The heat is how we define the calories in the food. The box around the sample in the figure shows that we are measuring the heat produced by the system, not the system plus the environment, that is, not applying the first law. If you applied the first law, the calories associated with the food would be zero, because any heat lost in combustion of the food would show up in the water bath of the calorimeter. The calories per gram of carbohydrate would be 0 instead of 4, the calories per gram of fat would be 0 not 9, etc. So, in studying reactions in chemical thermodynamics, energy is not conserved, it is dissipated. When systems dissipate energy, the change is indicated with a minus sign, so for oxidation of food, generally: ΔU < 0. So, no, the first law does not apply. That’s one of the reasons that “a calorie is not a calorie.”
There is an additional point that we assumed in passing. In chemical thermodynamics, the energy goes with the reaction, not with the food. It is not like particle physics where we give the mass of a particle in electron-volts, a measure of energy, because of E=mc2. What this means, practically, is that the 4 kcal per gram of carbohydrate is for the reaction of complete oxidation. Do anything else, make DNA, make protein and all bets are off.
The bottom line is that, contrary to what is usually said, thermodynamics does not predict energy balance and we should not be surprised when one diet is more or less efficient than another. In fact, the question to be answered is why energy balance is ever found. “A calories is a calorie” is frequently what is observed (although there is always a question as to how we make the measurement). The answer is that insofar as there is energy balance, it is a question of the unique behavior of living systems, not physical laws. Two similar subjects of similar age and genetic make-up may, under the right conditions, respond to different diets so that most of what they do is oxidize food and the contributions of DNA or protein synthesis, growth, etc. may be similar and may cancel out so that the major contribution to energy exchange is the heat of combustion.
But thermodynamics is really not about the first law which, while its history is a little odd, it is not revolutionary. Intellectually, the first law is related to conservation of matter. Thermodynamics is about the second law. The second law says that there is a physical parameter, called the entropy, almost always written S, and the change in entropy, ΔS, in any real process, always increases. In ideal, theoretical processes, ΔS may be zero, but it never goes down. In other words, looking at the universe, (any system and its surroundings), energy is conserved but entropy increases. The first law is a conservation law but the second law is a dissipation law. We identify the entropy with the organization, order or information in a system. Systems proceed naturally to the most probable state. In one of the best popular introductions to the subjects, von Baeyer’s Warmth Disperses and Time Passes, entropy is described in terms of the evolution of the organization of his teenage daughter’s room.  To finish up on calorimeters, though, there is Lavoisier’s whole animal calorimeter.



One of Lavoisier’s great contributions was to show that combustion was due to a combination with oxygen rather than the release of a substance, then known as the phlogiston. Lavoisier had the insight that in an animal, the combination of oxygen with food to produce carbon dioxide was the same kind of process. The whole animal calorimeter was a clever way to show this. The animal is placed in the basket compartment f. The inner jacket, b, is packed with ice. The outer jacket, a, is also packed with ice to keep the inner jacket, cold. The heat generated by the animal melts the ice in the inner jacket which is collected in container, Fig 8. Lavoisier showed that the amount of carbon dioxide formed was proportional to the heat generated as it would be if an animal were carrying out the same chemical reactions that occur, for example, in burning of charcoal. “La vie est donc une combustion.” His collaborator in this experiment was the famous mathematician Laplace and people sometimes wonder how he got a serious mathematician like Laplace to work on what is, well, nutrition. It seems likely that it was because Laplace owed him a lot of money.

Evidence Based Medicine and Admissibility. IV. The Oslo Diet-Heart Study.

Posted: June 1, 2011 in Evidence Based Medicine, Lipophobes, The Nutrition Story
Tags: , , , , ,
15

“In the Viking era, they were already using skis…and over the centuries, the Norwegians have proved themselves good at little else.”

–John Cleese, Norway, Home of Giants.

With the 3-foot bookshelf of popular attacks on the low-fat-diet-heart idea it is pretty remarkable that there is only one defense.  Daniel Steinberg’s Cholesterol Wars. The Skeptics vs. The Preponderance of Evidence is probably more accurately called a witness for the prosecution since low-fat, in some way or other is still the law of the land.

The Skeptics vs. the Preponderance of Evidence

The Skeptics vs. the Preponderance of Evidence

The book is very informative, if biased, and it provides an historical perspective describing the difficulty of establishing the cholesterol hypothesis. Oddly, though,  it still appears to be very defensive for a witness for the prosecution.  In any case, Steinberg introduces into evidence the Oslo Diet-Heart Study [2] with a serious complaint:

“Here was a carefully conducted study reported in 1966 with a statistically significant reduction in reinfarction [recurrence of heart attack] rate.  Why did it not receive the attention it deserved?”

“The key element,” he says, “was a sharp reduction in saturated fat and cholesterol intake and an increase in polyunsaturated fat intake. In fact. each experimental subject had to consume a pint of soybean oil every week, adding it to salad dressing or using it in cooking or, if necessary, just gulping it down!”

Whatever it deserved, the Oslo Diet-Heart Study did receive a good deal of attention.  The Women’s Health Initiative (WHI), liked it.  The WHI was the most expensive failure to date. It found that “over a mean of 8.1 years, a dietary intervention that reduced total fat intake and increased intakes of vegetables, fruits, and grains did not significantly reduce the risk of CHD, stroke, or CVD in postmenopausal women.” [3]

The WHI, adopted a “win a few, lose a few” attitude, comparing its results to the literature, where some studies showed an effect of reducing dietary fat and some did not — this made me wonder: if the case is so clear, whey are there any failures.  Anyway, it cited the Oslo Diet-Heart Study as one of the winners and attributed the outcome to the substantial lowering of plasma cholesterol.

So, “cross-examination” would tell us why, if  “a statistically significant reduction in reinfarction  rate”  it did “not receive the attention it deserved?”

First, the effect of diet on cholesterol over five years:

The results look good although, since all the numbers are considered fairly high, and since the range of values is not shown, it is hard to tell just how impressive the results really are. But we will stipulate that you can lower cholesterol on a low-fat diet. But what about the payoff? What about the outcomes?

The results are shown in Table 5 of the original paper:   Steinberg described how in the first 5 years: “58 patients of the 206 in the control group (28%) had a second heart attack” (first 3 lines under first line of blue-highlighting) but only

“…  32 of the 206 in the diet (16%)…”  which does sound pretty good.

In the end, though, it’s really the total deaths from cardiac disease.  The second blue-highlighted line in Table 5 shows the two final outcome.  How should we compare these.

1. The odds ratio or relative risk is just the ratio of the two outcomes (since there are the same number of subjects) = CHD mortality (diet)/ CHD mortality control) = 94/79 =  1.19.  This seems strikingly close to 1.0, that is, flip of a coin.  These days the media, or the report itself, would report that there was a 19 % reduction in total CHD mortality.

2, If you look at the absolute values, however, the  mortality in the controls is 94/206 = 45.6 % but the diet group had reduced this  to 79/206 = 38.3 % so the change in absolute risk is  45.6 % – 38.3 % or only 7.3 % which is less impressive but still not too bad.

3. So for every 206 people, we save 94-79 = 15 lives, or dividing 206/15 = 14 people needed to treat to save one life. (Usually abbreviated NNT). That doesn’t sound too bad.  Not penicillin but could be beneficial. I think…

Smoke and mirrors.

It’s what comes next that is so distressing.  Table 10 pools the two groups, the diet and the control group and now compares  the effect of smoking: on the whole population,  the ratio of CHD deaths in smokers vs non-smokers is 119/54 = 2.2 (magenta highlight) which is somewhat more impressive than the 1.19 effect we just saw.  Now,

1. The absolute difference in risk is (119-54)/206 = 31.6 % which sounds like a meaningful number.

2. The number needed to treat is 206/64 = 3.17  or only about 3 people need to quit smoking to see one less death

In fact, in some sense, the Oslo Diet-Heart Study provides smoking-CHD risk as an example of a meaningful association that one can take seriously. If only such a significant change had actually been found for the diet effect.

So what do the authors make of this? Their conclusion is that “When combining data from both groups, a three-fold greater CHD mortality rate is demonstrable among the hypercholesterolemic, hypertensive smokers than among those in whom these factors were low or absent.”  Clever but sneaky. The “hypercholesterolemic, hypertensive” part is irrelevant since you combined the groups. In other words, what started out as a diet study has become a “lifestyle study.”  They might has well have said “When combining data from fish and birds a significant number of wings were evident.” Members of the jury are shaking their heads.

Logistic regression. What is it? Can it help?

So they have mixed up smoking and diet. Isn’t there a way to tell which was more important?  Well, of course, there are several ways.  By coincidence, while I was writing this post, April Smith posted on facebook, the following challenge “The first person to explain logistic regression to me wins admission to SUNY Downstate Medical School!” I won although I am already at Downstate.  Logistic regression is, in fact, a statistical method that asks what the relative contribution of different inputs would have to be to fit the outcome and this could have been done but in this case, I would use my favorite statistical method, the Eyeball Test.  Looking at the data in Tables 5 and 10 for CHD deaths, you can see immediately what’s going on. Smoking is a bigger risk than diet.

If you really want a number, we calculated relative risk above. Again, we found for mortality, CHD (diet)/ CHD (control) = 94/79 =  1.19. But what happens if you took up smoking: Figure 10 shows that your chance of dying of heart disease would be increased by 119/54 = 2.2  or more than twice the risk.  Bottom line: you decided to add saturated fat to your diet, your risk would be 1.19 what it was before which might be a chance you could take faced with authentic Foie Gras.

Daniel Steinberg’s question:

“Here was a carefully conducted study reported in 1966 with a statistically significant reduction in reinfarction  rate.  Why did it not receive the attention it deserved?”

Well, it did. This is not the first critique.  Uffe Ravnskov described how the confusion of smoking and diet led to a new Oslo Trial which reductions in both were specifically recommended and, again, outcomes made diet look bad [4].  Ravnskov gave it the attention it deserved. But what about researchers writing in the scientific literature. Why do they not give the study the attention it deserves. Why do they not point out its status as a classic case of a saturated fat risk study with no null hypothesis.  It certainly deserves attention for its devious style. Of course, putting that in print would guarantee that your grant is never funded and your papers will be hard to publish.  So, why do researchers not give the Oslo-Diet-Heart study the attention it deserves?  Good question, Dan.

Bibliography

1. Steinberg D: The cholesterol wars : the skeptics vs. the preponderance of evidence, 1st edn. San Diego, Calif.: Academic Press; 2007.

2. Leren P: The Oslo diet-heart study. Eleven-year report. Circulation 1970, 42(5):935-942.

3. Howard BV, Van Horn L, Hsia J, Manson JE, Stefanick ML, Wassertheil-Smoller S, Kuller LH, LaCroix AZ, Langer RD, Lasser NL et al: Low-fat dietary pattern and risk of cardiovascular disease: the Women’s Health Initiative Randomized Controlled Dietary Modification Trial. JAMA 2006, 295(6):655-666.

4. Ravnskov U: The Cholesterol Myths: Exposing the Fallacy that Cholesterol and Saturated Fat Cause Heart Disease. Washington, DC: NewTrends Publishing, Inc.; 2000.

Evidence Based Medicine and Admissibility. III. Steinberg for the Prosecution.

Posted: May 22, 2011 in Evidence Based Medicine, Lipophobes, The Nutrition Story
Tags: , , , , , , ,
0

In 1985 an NIH Consensus Conference was able to “establish beyond any reasonable doubt the close relationship between elevated blood cholesterol levels (as measured in serum or plasma) and coronary heart disease” (JAMA 1985, 253:2080-2086).

I have been making an analogy between scientific behavior and the activities of the legal system and following that idea, the wording of the conference conclusion suggests a criminal indictment. Since the time of the NIH conference, however, data on the role of cholesterol fractions, the so-called “good (HDL)” and “bad (LDL)” cholesterols and, most recently, the apparent differences in the atherogenicity of different LDL sub-fractions would seem to have provided some reasonable doubt. What has actually happened is that the nutrition establishment, the lipophobes as Michael Pollan calls them, has extended the indictment to include dietary fat, especially saturated fat at least as accessories on the grounds that, as the Illinois Criminal Code put it “before or during the commission of an offense, and with the intent to promote or facilitate such commission, … solicits, aids, abets, agrees or attempts to aid… in the planning or commission of the offense. . . ..”

A major strategy in the indictment of saturated fat has been guilt by association.  The American Heart Association (AHA), which had long recommended margarine (the major source of trans-fats), has gone all out in condemning saturated fatty acids by linking them with trans-fats.  The AHA website has a truly deranged cartoon film of the evil brothers: “They’re a charming pair, Sat and Trans.  But that doesn’t mean they make good friends.  Read on to learn how they clog arteries and break hearts — and how to limit your time with them by avoiding the foods they’re in.”. While the risk of trans-fats is probably exaggerated — they are a small part of the diet — they have no benefit and nobody wants to defend them; dietary saturated fat, however, is a normal part of the diet, is made in your body and is less important in providing saturated fatty acids in the blood, than dietary carbohydrate.  Guilt by association is a tricky business in courts of law — just having a roommate who sells marijuana can get you into a good deal of trouble — but it takes more than somebody saying that you and the perpetrator make a charming pair.

The failure of the diet-cholesterol-heart hypothesis in clinical trials as been documented by numerous scientific articles and especially in popular books that document the original scientific sources. It is unknown what the reaction of the public is to these books.  However, amazingly, there is only one book I know of that takes the side of the lipophobes and that is Daniel Steinberg’s Cholesterol Wars. The Skeptics vs. the Preponderance of Evidence. A serious book with careful if slightly biased documentation and an uncommon willingness to answer the critics,  it is worth reading.  I will try to discuss it in detail in this and future posts.  First, the title indicates a step down from criminal prosecution.  “Preponderance of the evidence” is the standard for conviction in a civil court and is obviously a far weaker criterion.  One has to wonder why it is that the skeptics have the preponderance of the popular publications — if the scientific evidence is there and health agencies are so determined that the public know about this, why are there so few —  maybe only this one — rebutting the critics.

The Skeptics vs. the Preponderance of Evidence

In any case, what is Steinberg’s case?  The indictment on page 1 is somewhat different than one would have thought.

“….the [lipid] hypothesis relates to blood lipids not dietary lipids as the putative directly causative factor. Although diet, especially dietary lipid is an important determinant of blood lipid levels, many other factors play important roles. Moreover, there is a great deal of variability in response of individuals to dietary manipulations. Thus, it is essential to distinguish between the indirect “diet-heart” connection and the direct “blood lipid — hard” connection failure to make this distinction has been a frequent source of confusion. (his italics)”

What?  Are we really supposed to believe that diet is an incidental part of the lipid hypothesis?  Are we supposed to believe that our cholesterol is just a question of the variability of our response to diet.  Has the message really been that diet is not critical and that heart-disease is just the luck of the draw (until we start taking statins)?  This is certainly the source of confusion in my mind.  Of course by page 5, we are confronted with this:

“In 1966, Paul Leren published his classic five-year study of 412 patients who had had a prior myocardial infarction. He showed that substitution of polyunsaturated fat and saturated fat-rich butter-cream-venison diet favored by the Norwegians reduced their blood cholesterol by about 17 per cent and kept it down.  The number of secondary current events in the treated group was reduced by about one-third and the result was significant at the p < 0.03 level.”

In a future post, I will describe Paul Leren’s classic five-year study which, by 1970, had a follow-up to eleven years and the results will turn out not to be as compelling as described by Steinberg.  For the moment, it is worth considering that, given the strong message, from the AHA, from the American Diabetes Association, from the NIH Guidelines for Americans, the criterion really should be beyond a reasonable doubt. There shouldn’t be even a single failure like the Framingham Study or the Women’s Health Initiative. In fact, the preponderance of the evidence when you add them all up, isn’t there.

Evidence-based Medicine. Who decides Admissibility. II. Mozart and the Levels of Evidence.

Posted: May 12, 2011 in Evidence Based Medicine, The Nutrition Story
Tags: , ,
0

The phrase “Evidence-based Medicine” (EBM) guarantees its proponents a certain degree of protection. After all, who would be against medicine that is based on the data, on hard facts rather than opinion. On the other hand, a study that needs to cloak itself in such a self-aggrandizing phrase must raise a few eyebrows; as usual, the Dietary Guidelines, moves to the top of the list in that category but there are many examples.  Martin Tobin, professor of Medicine at Loyola College provided an excellent deconstruction of evidence based medicine [1]. Some of his points were that the grading system has divorced itself from basic science.  For example, he points out that:

  • “ homeopathy uses drugs in which less than one molecule of active agent is present. … A meta-analysis of 89 placebo-controlled trials revealed a combined odds of 2.45 in favor of homeopathy. EBM grades meta-analysis as level 1 evidence but completely ignores scientific theory. There is nothing necessarily wrong with this particular meta-analysis, but the example illustrates how a system that grades findings of all meta-analyses as level 1 evidence is inherently flawed.  A grading system that ranks homeopathy as sounder evidence than centuries of pharmacologic science commits the reductio ad absurdum fallacy in logic.” [1]

Among the things that we found in our critique of the USDA dietary guidelines Report [2] was that the cited evidence did not meet their own standards. They were critical of low-carbohydrate diets on the basis of studies that their own analysis gave a “neutral” quality rating, even those that took dietary assessment at baseline and then assessed  cardiovascular mortality up to 12 years later.

But it is really the idea that there is some set of systematic definitions of science that everybody agrees on. My last post mentioned, by analogy with courts of law, the Frye standard which accepts as evidence, opinions supported by  “general acceptance’ in the scientific community.  While still accepted in some state courts, the federal courts have tried to go beyond trust in such narrow descriptions of science. In 1975, Congress established Federal Rules of Evidence.  The rules are quite general and the major impact is to broaden the range of evidence that could be considered.  Rule 401, defined relevance as  “evidence having any tendency to make the existence of any fact that is of consequence to the determination of the action more probable or less probable than it would be without the evidence,”  in other words, whatever works.  In a future post, I will discuss Daubert v. Merrell Dow, Inc. (pr. Dow-burt as in English), an outgrowth of the Rules of Evidence and generally considered the key judgment in the modern interaction of science and the law.  In the real world of jurisprudence, ideas on what constitutes scientific evidence have become problematical and Daubert may have had the paradoxical effect of restricting admissible data but, in the analogy with evidence in medicine, the Federal Rules of Evidence and Daubert have better captured the real quality of science in recognizing the need for flexibility. The kinds of absolute criteria — association does not imply causality, random controlled trials are a “gold standard,” etc. are at least different from the spirit of Daubert.

More important, nobody in any physical science would recognize the tables of levels of evidence.  A random controlled trial may be good for one kind of experiment but not for another and EBM is critical of “observational studies” but all of astronomy is observational.  In the end, most scientists would agree with the physicist Steven Weinberg, echoing Judge Potter Stewart’s famous take on pornography:

  •  “There is no logical formula that establishes a sharp dividing line between a beautiful explanatory theory and a mere list of data, but we know the difference when we see it — we demand a simplicity and rigidity in our principles before we are willing to take them seriously [3].”

So where do these arbitrary guidelines in EBM come from?  They were set up by the  medical community, a community that is stereotyped as being untrained in science. I hate stereotypes, especially medical stereotypes since I think of myself as coming from a medical family (my father and oldest daughter are physicians) but stereotypes come from someplace and, of course, it is well known that physicians never study nutrition.  In the end, it makes me think of the undoubtedly apocryphal story about Mozart.

  • A man comes to Mozart and wants to become a composer.  Mozart says that they have to study theory for  a couple of years, they should study orchestration and become proficient  at the piano, and goes on like this.  Finally, the man says “but you wrote your first symphony when you were 8 years old.”  Mozart says “Yes, but I didn’t ask anybody.”

Bibliography 

1. Tobin MJ: Counterpoint: evidence-based medicine lacks a sound scientific base. Chest 2008, 133(5):1071-1074; discussion 1074-1077.

2. Hite AH, Feinman RD, Guzman GE, Satin M, Schoenfeld PA, Wood RJ: In the face of contradictory evidence: report of the Dietary Guidelines for Americans Committee. Nutrition 2010, 26(10):915-924.

3. Weinberg S: Dreams of a final theory, 1st edn. New York: Pantheon Books; 1992.

Older Entries
Newer Entries