Archive for the ‘Lipophobes’ Category

First published in October of 2011, this post announced a Q&A on line with Harvard’s Eric Rimm to answer question about the School of Public Health’s new  “Healthy Eating Plate,” its own version of nutritional recommendations to compete with the USDA’s MyPlate. A rather  limited window of one hour  was allotted for the entire country to phone in our questions.  Unfortunately HSPH was not as good at telecommunications as it is at epidemiology and the connection did not start working for a while.  The questions that I wanted to ask, however, still stand and this post is a duplicate of the original with the notice about the Q&A removed.  Harvard has been invited to participate in a panel discussion at the Ancestral Health Symposium, and we will see how these questions can be answered.

— adopted from Pops (at Louder and Smarter), the anonymous brilliant artist and admitted ne’er do well.

One of the questions surrounding USDA Nutrition Guidelines for Americans was whether so-called “sunshine laws,” like the Freedom of Information Act, were adhered to. Whereas hearings were recorded, and input from the public was solicited, there is the sense that if the letter of the law was followed, the spirit was weak.  When I and colleagues testified at the USDA hearings, there was little evidence that their representatives were listening; there was no discussion. We said our piece and then were heard no more.  In fact, at the break, when I tried to speak to one of the panel, somebody came out from backstage, I believe unarmed, to tell me that I could not discuss anything with the committee.

Harvard School of Public Health, home of  “odds ratio = 1.22,” last month published their own implementation of the one size-fits-all approach to public nutrition, the”Healthy Eating Plate.”  Their advice is full of  “healthy,” “packed with” and other self-praise that makes this mostly an infomercial for HSPH’s point of view. Supposedly a correction of the errors in MyPlate from the USDA, it seems to be more similar than different. The major similarity is the disdain for the intelligence of the American public. Comparing the two plates (below), they have exchanged the positions of fruits and vegetables.  “Grains” on MyPlate is now called “Whole Grains,” and “Protein” has been brilliantly changed to “Healthy Proteins.”  How many NIH grants were required to think of this is unknown.  Harvard will, of course, tell you what “healthy” is:, no red meat and, of course watch out for the Seventh Egg.

 

 

 

 

 

 

 

So here are the  questions that I wanted to ask:

  1. Dr. Rimm, you are recommending a diet for all Americans but even within the pattern of general recommendations, I don’t know of any experimental trial that has tested it.  Aren’t you just recommending another grand experiment like the original USDA recommendations which you are supposedly improving on?
  2. Dr. Rimm, given that half the population is overweight or obese shouldn’t there be at least two plates?
  3. Dr. Rimm, I think the American public expects a scientific document.  Don’t you think continued use of the words “healthy,” “packed with nutrients,” makes the Plate more of  an informercial for your point of view?
  4. Dr. Rimm, the Plate site says “The contents of this Web site are not intended to offer personal medical advice,” but it seems that is exactly what it is doing. If you say that you are recommending a diet that will “Lower blood pressure; reduced risk of heart disease, stroke, and probably some cancers; lower risk of eye and digestive problems,” how is that not medical advice?  Are you disowning responsibility for the outcome in advance?
  5. Dr. Rimm, more generally, how will you judge if these recommendations are successful? Is there a null hypothesis? The USDA recommendations continue from year to year without any considerations of past successes or failures.
  6. Dr. Rimm, “healthy” implies general consensus but there are many scientists and physicians with good credentials and experience who hold to different opinions. Have you considered these opinions in formulating the plate? Is there any room for dissent or alternatives?
  7. Dr. Rimm, the major alternative point of view is that low-carbohydrate diets offer benefits for weight loss and maintenance and, obviously, for diabetes and metabolic syndrome. Although your recommendations continually refer to regulation of blood sugar, it is not incorporated in the Plate.
  8. Dr. Rimm, nutritionally, fruits have more sugar, more calories, less potassium, fewer antioxidants than vegetables.  Why are they lumped together? And how can you equate beans, nuts and meat as a source of protein?
  9. Dr. Rimm, looking at the comparison of MyPlate and your Plate, it seems that all that is changed is that “healthy” has been added to proteins and “whole” has been added to grains.  If people know what “healthy” is, why is there an obesity epidemic? Or are you blaming the patient?
  10. Dr. Rimm, you are famous for disagreeing on lipids with the DGAC committee yet your name is on their report as well as on this document is supposed to be an alternative.  Do we know where you stand?
  11. Dr. Rimm, the Healthy Plate “differences” page says “The Healthy Eating Plate is based exclusively on the best available science and was not subjected to political and commercial pressures from food industry lobbyists.” This implies that the USDA recommendations are subject to such pressures.  What is the evidence for this? You were a member of the USDA panel. What pressures were brought to bear on you and how did you deal with them
  12. Dr. Rimm, the Healthy Plate still limits saturated fat even though a study from your department showed that there was, in fact, no effect of dietary saturated fat on cardiovascular disease.  That study, moreover, was an analysis of numerous previous trials, the great majority of which individually showed no risk from saturated fat. What was wrong with that study that allows you to ignore it?

*Medicineball, (colloq) a game that derives from Moneyball, in which an “unscientific culture responds, or fails to respond, to the scientific method ” in order  to stay funded.

Baseball is like church. Many attend. Few understand.

— Leo Durocher.

The movie Moneyball provides an affirmative answer to an important question in literature and drama: can you present a scene and bring out the character of a subject that is boring while, at the same time, not make the presentation boring?  The movie, and  Michael Lewis’sbook that it is based on, are about baseball and statistics!  For fans, baseball is not boring so much as incredibly slow, providing a soothing effect like fishing, interspersed with an occasional big catch. The movie stars Brad Pitt as Billy Beane, the General Manager of the Oakland Athletics baseball team in the 1990s.  A remarkably talented high school athlete, Billy Beane, for unknown reasons, was never able to play out his potential as an MLB player but, in the end, he had a decisive effect on the game at the managerial level. The question is how the A’s, with one-third of the budget of the Yankees, could have been in the play-offs three years in a row and, in 2001, could win 102 games.  The movie is more or less faithful to the book and both are as much about organizations and psychology as about sports. The story was “an example of how an unscientific culture responds, or fails to respond, to the scientific method” and the science is substantially statistical.

In America, baseball is a metaphor for just about everything. Probably because it is an experience of childhood and adolescence, lessons learned from baseball stay with us. Baby-boomers who grew up in Brooklyn were taught by Bobby Thompson’s 1951 home-run, as by nothing later, that life isn’t fair. The talking heads in Ken Burns’s Baseball who found profound meaning in the sport are good examples. Former New York Governor Mario Cuomo’s comments were quite philosophical although he did add the observation that getting hit in the head with a pitched ball led him to go into politics.

One aspect of baseball that is surprising, especially when you consider the money involved, is the extent to which strategy and scouting practices have generally ignored hard scientific data in favor of tradition and lore. Moneyball tells us about group think, self-deception and adherence to habit in the face of science. For those of us who a trying to make sense of the field of nutrition, where people’s lives are at stake and where numerous professionals who must know better insist on dogma — low fat, no red meat — in the face of contradictory evidence, baseball provides some excellent analogies.

The real stars of the story are the statistics and the computer or, more precisely, the statistics and computer guys: Bill James an amateur analyzer of baseball statistics and Paul DePodesta, assistant General Manager of the A’s who provided information about the real nature of the game and how to use this information. James self-published a photocopied book called 1977 baseball abstract: featuring 18 categories of statistical information you just can’t find anywhere else. The book was not just about statistics but was in fact a critique of traditional statistics pointing out, for example, that the concept of an “error;” was antiquated, deriving from the early days of gloveless fielders and un-groomed playing fields of the 1850s. In modern baseball, “you have to do something right to get an error; even if the ball is hit right at you, and you were standing in the right place to begin with.” Evolving rapidly, the Abstracts became a fixture of baseball life and are currently the premium (and expensive) way to obtain baseball information.

It is the emphasis on statistics that made people doubt that Moneyball could be made into a movie and is probably why they stopped shooting the first time around a couple of years ago. Also, although Paul DePodesta (above) is handsome and athletic, Hollywood felt that they should cast him as an overweight geek type played by Jonah Hill. All of the characters in the film have the names of the real people except for DePodesta “for legal reasons,” he says. Paul must have no sense of humor.

The important analogy with nutrition research and the continuing thread in this blog, is that it is about the real meaning of statistics. Lewis recognized that the thing that James thought was wrong with the statistics was that they

“made sense only as numbers, not as a language. Language, not numbers, is what interested him. Words, and the meaning they were designed to convey. ‘When the numbers acquire the significance of language,’ he later wrote, ‘they acquire the power to do all the things which language can do: to become fiction and drama and poetry … . And it is not just baseball that these numbers through a fractured mirror, describe. It is character. It is psychology, it is history, it is power and it is grace, glory, consistency….’”

By analogy, it is the tedious comparison of quintiles from the Harvard School of Public Health proving that white rice will give you diabetes but brown rice won’t or red meat is bad but white meat is not, odds ratio = 1.32. It is the bloodless, mindless idea that if the computer says so, it must be true, regardless of what common sense tells you. What Bill James and Paul DePodesta brought to the problem was understanding that the computer will only give you a meaningful answer if you ask the right question; asking what behaviors accumulated runs and won ball games, not which physical characteristics — runs fast, looks muscular — that seem to go with being a ball player… the direct analog of “you are what you eat,” or the relative importance of lowering you cholesterol vs whether you actually live or die.

As early as the seventies, the computer had crunched baseball stats and come up with clear recommendations for strategy. The one I remember, since it was consistent with my own intuition, was that a sacrifice bunt was a poor play; sometimes it worked but you were much better off, statistically, having every batter simply try to get a hit. I remember my amazement at how little effect the computer results had on the frequency of sacrifice bunts in the game. Did science not count? What player or manager did not care whether you actually won or lost a baseball game. The themes that are played out in Moneyball, is that tradition dies hard and we don’t like to change our mind even for our own benefit. We invent ways to justify our stubbornness and we focus on superficial indicators rather than real performance and sometimes we are just not real smart.

Among the old ideas, still current, was that the batting average is the main indicator of a batter’s strength. The batting average is computed by considering that a base-on-balls is not an official at bat whereas a moments thought tells you that the ability to avoid bad pitches is an essential part of the batter’s skill. Early on, even before he was hired by Billy Beane, Paul DePodesta had run the statistics from every twentieth century baseball team. There were only two offensive statistics that were important for a winning team percentage: on-base percentage (which included walks) and slugging percentage. “Everything else was far less important.” These numbers are now part of baseball although I am not enough of a fan to know the extent to which they are still secondary to the batting average.

One of the early examples of the conflict between tradition and science was the scouts refusal to follow up on the computer’s recommendation to look at a fat, college kid named Kevin Youkilis who would soon have the second highest on-base percentage after Barry Bonds. “To Paul, he’d become Euclis: the Greek god of walks.”

The big question in nutrition is how the cholesterol-diet-heart paradigm can persist in the face of the consistent failures of experimental and clinical trials to provide support. The story of these failures and the usurpation of the general field by idealogues has been told many times. Gary Taubes’s Good Calories, Bad Calories is the most compelling and, as I pointed out in a previous post, there seems to have been only one rebuttal, Steinberg’s Cholesterol Wars. The Skeptics vs. the Preponderance of Evidence. At least within the past ten year, a small group have tried to introduce new ideas, in particular that it is excessive consumption of dietary carbohydrate, not dietary fat, that is the metabolic component of the problems in obesity, diabetes and heart disease and have provided extensive, if generally un-referenced, experimental support. An analogous group tried to influence baseball in the years before Billy Beane. Larry Lucchino, an executive of the San Diego Padres described the group in baseball as being perceived as something of a cult and therefore easily dismissed. “There was a profusion of new knowledge and it was ignored.” As described in Moneyball “you didn’t have to look at big-league baseball very closely to see its fierce unwillingness to rethink any it was as if it had been inoculated against outside ideas.”

“Grady Fuson, the A’s soon to be former head of scouting, had taken a high school pitcher named Jeremy Bonderman and the kid had a 94 mile-per-hour fastball, a clean delivery, and a body that looked as if it had been created to wear a baseball uniform. He was, in short, precisely the kind of pitcher Billy thought he had trained the scouting department to avoid…. Taking a high school pitcher in the first round — and spending 1.2 million bucks to sign — that was exactly this sort of thing that happened when you let scouts have their way. It defied the odds; it defied reason. Reason, even science, was what Billy Beane was intent on bringing to baseball.”

The analogy is to the deeply ingrained nutritional tradition, the continued insistence on cholesterol and dietary fat that are assumed to have evolved in human history in order to cause heart disease. The analogy is the persistence of the lipophobes, in the face of scientific results showing, at every turn, that these were bad ideas, that, in fact, dietary saturated fat does not cause heart disease. It leads, in the end, to things like Steinberg’s description of the Multiple risk factor intervention trial. (MRFIT; It’s better not to be too clever on acronyms lest the study really bombs out): “Mortality from CHD was 17.9 deaths per 1,000 in the [intervention] group and 19.3 per 1,000 in the [control] group, a statistically nonsignificant difference of 7.1%”). Steinberg’s take on MRFIT:

“The study failed to show a significant decrease in coronary heart disease and is often cited as a negative study that challenges the validity of the lipid hypothesis. However, the difference in cholesterol level between the controls and those on the lipid lowering die was only about 2 per cent. This was clearly not a meaningful test of the lipid hypothesis.”

In other words, cholesterol is more important than outcome or at least a “diet designed to lower cholesterol levels, along with advice to stop smoking and advice on exercise” may still be a good thing.

Similarly, the Framingham study which found a strong association between cholesterol and heart disease found no effect of dietary fat, saturated fat or cholesterol on cardiovascular disease.  Again, a marker for risk is more important than whether you get sick.  “Scouts” who continued to look for superficial signs and ignore seemingly counter-intuitive conclusions from the computer still hold sway on the nutritional team.

“Grady had no way of knowing how much Billy disapproved of Grady’s most deeply ingrained attitude — that Billy had come to believe that baseball scouting was at roughly the same stage of development in the twenty-first century as professional medicine had been in the eighteenth.”

Professional medicine? Maybe not the best example.

What is going on here? Physicians, like all of us, are subject to many reinforcers but for humans power and control are usually predominant and, in medicine, that plays out most clearly in curing the patient. Defeating disease shines through even the most cynical analysis of physician’s motivations. And who doesn’t play baseball to win. “The game itself is a ruthless competition. Unless you’re very good, you don’t survive in it.”

Moneyball describes a “stark contrast between the field of play and the uneasy space just off it, where the executives in the Scouts make their livings.” For the latter, read the expert panels of the American Heat Association and the Dietary Guidelines committee, the Robert Eckels who don’t even want to study low carbohydrate diets (unless it can be done in their own laboratory with NIH money). In this

“space just off the field of play there really is no level of incompetence that won’t be tolerated. There are many reasons for this, but the big one is that baseball has structured itself less as a business and as a social club. The club includes not only the people who manage the team but also in a kind of women’s auxiliary many of the writers and commentators to follow and purport to explain. The club is selective, but the criteria for admission and retention and it is there many ways to embarrass the club, but being bad at your job isn’t one of them. The greatest offense a club member can commit is not ineptitude but disloyalty.”

The vast NIH-USDA-AHA social club does not tolerate dissent. And the media, WebMD, Heart.org and all the networks from ABCNews to Huffington Post will be there to support the club. The Huffington Post, who will be down on the President of the United States in a moment, will toe the mark when it comes to a low carbohydrate story.

The lessons from money ball are primarily in providing yet another precedent for human error, stubbornness and, possibly even stupidity, even in an area where the stakes are high. In other words, the nutrition mess is not in our imagination. The positive message is that there is, as they say in political science, validator satisfaction. Science must win out. The current threat is that the nutritional establishment is, as I describe it, slouching toward low-carb, doing small experiments, and easing into a position where they will say that they never were opposed to the therapeutic value of carbohydrate restriction. A threat because they will try to get their friends funded to repeat, poorly, studies that have already been done well. But that is another story, part of the strange story of Medicineball.

The headline in the BBC News is “Fat ‘disrupts sugar sensors causing type 2 diabetes’” The article does not attribute the quotation in the headline and the first sentence says “US researchers say they have identified how a high-fat diet can trigger type 2 diabetes, in experiments on mice and human tissue.”  Should “mice” or “tissue” have been in the headline?  Should the article itself point out the extent to which mice respond differently, sometimes, oppositely from humans, to high-fat diets?  How strong is the evidence in light of other work?  Is the article altogether prejudicing the reader against fat which is the official position of both private and government health agencies?  The article in question may have some sins of omission but it is certainly restrained if not actually circumspect. The general problem, of course, is whether we get accurate scientific information from the popular media.

Peter Farnham and I and a group of bloggers (Laura Dolszon, Tom Naughton and Jimmy Moore) will speak at the end of the month at a conference produced by the Office of Research Integrity (ORI)) where we will raise several issues in the ethical conduct of current nutritional research. The conference, in general, tries to explore a number of questions on the interaction of science and  society.  The goals are to “discuss the latest research on research integrity…education in the responsible conduct of research; responsible research practices.” While each presenter will have only 15 minutes, this is one of the first times thatd the practices in nutrition with regard to issues of integrity are being addressed.  There are four areas that we will discuss:

Crisis in Nutrition I: The Popular Media and Research Publications

Richard David Feinman, SUNY Downstate Medical Center

Crisis in Nutrition II: Research Integrity in Meeting the Challenge  of Carbohydrate Restriction

Richard David Feinman, SUNY Downstate  Medical Center

Crisis in Nutrition III:  Was the Government Standard Met by the  2010 Dietary Guidelines?

Peter Farnham, Nutrition and Metabolism  Society, Alexandria, VA

Crisis in Nutrition IV:  Vox Populi 

Tom Naughton, Jimmy Moore, Laura Dolson, Independent Writers Franklin TN

The abstract of my first talk is presented below. It is, of course, a tricky area. Within some legal limits, reporters can say what they like.  A researcher speaking in a public venue, personal blog, social media can similarly pretty much sound of as they choose.  Or can they? If they are identified as an expert or are have credentials based on a employment by a prestigious institution, don’t they have to clearly distinguish between opinion and fact? And does the headline have to say that, for example, the high-fat study was done in mice?  All of these are gray areas and motives are hard to discern.  I focussed on one area that seemed more clear cut.  If an experimental study is reported in the media or in a press release from an academic institution (sometimes the same thing), is there an obligation to be sure that any opinions attributed to the investigator derives from that research unless otherwise indicated?

 Crisis in nutrition: I. The popular media and research publications  

Objective:

The public relies on popular media for descriptions of nutrition research.  Of particular interest is carbohydrate-restricted diets, the major challenge to official recommendations.  The goal is to assess the extent to which statements to the media and press releases accurately represent the results of research.

Summary of findings or main points:

Nutrition is an area of great interest to the public but one where matters of scientific fact and policy are contentious. Authors of research papers should sensibly have great freedom in describing of the implications of their research, but have an important role in explaining to the public when that opinion does or does not follow directly from the publication.  Two examples are given of where this is a critical issue. In one, an animal study (Foo, et al. Proc Natl Acad Sci USA 2009, 106: 15418-15423), the accompanying press release implies that it was motivated by observations of patients in a hospital, observations which were purely anecdotal and unsubstantiated.  In a second example, a press release stated that carbohydrate-restricted diets (CRDs) were not included in a comparative study because of their low compliance (Sacks, et al. N Engl J Med 2009, 360: 859-873). No data were given to support this allegation and, it is, in fact not true.  The study concluded that the macronutrient composition of the diet was not important even though, as implemented, dietary intake was the same for the groups studied and, again, the CRD was not included in the study.  It seems likely that that this would have an inhibiting effect on individuals choosing a CRD and represents an important impact of research integrity issues on the community.

Conclusions & recommendations:

Practices where research directly affects the community should be evaluated and guidelines should be generated by academic societies, scientific journals and the popular media. What constitutes appropriate press description of published research should be defined. Reasonable principles are that only those specific conclusions that derive directly from the publication are relevant and authors make clear what is their personal opinion and what is the product of research data.

 Office of Research Integrity

It is important to emphasize that ORI is sponsor of this academic conference and is not related to is regulatory function.  ORI is charged with overseeing federally funded research and emphasizes its role as watchdog in detection and prevention of research misconduct, assisting the Office of the General Counsel (OGC), dealing with suspected retaliation against whistleblowers, and responding to Freedom of Information and Privacy Acts.  Its usual activity involves pinpointing specific fraud.  The website reports, for example, a final judgement against an Assistant Professor at the Boston University School of Medicine Cancer Research Center who published two papers in which he had fabricated data shown as figures in the paper.  He is required to retract the papers and not enter into contracts or sit on advisory panels for two years.

ORI, in general, has the same relation to the research community that Internal Affairs has to the police.  Of course, in research, although there are substantive rewards, blatant fraud is generally pathological: if the results you are falsifying are important, they will surely be repeated and the misbehavior discovered while, if they are not important, the rewards are not likely to be spectacular although, especially these days, keeping your job is desirable.  The suspicion about ORI is also bolstered by their behavior in the Baltimore case in which they were part of the mindless zeal and witch-hunting whose appearance in human interactions seems to have such a low activation energy.

David Baltimore, a Nobel laureate in Physiology or Medicine and, at the time, MIT professor, had co-authored a paper with an immunologist named  Thereza Imanishi-Kari who was accused by a postdoctoral fellow of fabricating data. In the end, nothing came of it but there was much sound and fury and many idiots participated in the tale including the Secret Service (who I was taught were only supposed to protect the President and prosecute counterfeiters). The details of the Baltimore case are well told in capsule form in the Wikipedia entry. Daniel Kevles wrote an outstanding book, at least judged by the first half — the content was too infuriating for me to keep reading.  In any case, after most of the furor had died down, the ORI persisted and found Imanishi-Kari guilty of research misconduct, a ruling overturned by an appeals panel of the Department of Health and Human Services (HHS) which “found that much of what ORI presented was irrelevant, had limited probative value, was internally inconsistent, lacked reliability or foundation, was not credible or not corroborated, or was based on unwarranted assumptions.”  the ORI reputation has probably not recovered from this but it remains one of the few oversight agencies which, at least in nutrition, is sorely needed.

David Baltimore wrote his own description of the events and emphasized that it raise many questions, in particular,  “Who should judge science?” and “How does one distinguish between error and fraud? And does science do an adequate job of policing itself?”  The conference and this blog will discuss these matters which the crisis in nutrition has made of critical importance.  Such philosophical questions were formerly what most of us would have preferred to simply gab about in Starbucks.

“These results suggest that there is no superior long-term metabolic benefit of a high-protein diet over a high-carbohydrate in the management of type 2 diabetes.”  The conclusion is from a paper by Larsen, et al. [1] which, based on that statement in the Abstract, I would not normally bother to read; it is good that you have to register trials and report failures but from a broader perspective, finding nothing is not great news and just because Larsen couldn’t do it, doesn’t mean it can’t be done.  However, in this case, I received an email from International Diabetes published bilingually in Beijing: “Each month we run a monthly column where choose a hot-topic article… and invite expert commentary opinion about that article” so I agreed to write an opinion. The following is my commentary:

“…no superior long-term metabolic benefit of a high-protein diet over a high-carbohydrate ….” A slightly more positive conclusion might have been that “a high-protein diet is as good as a high carbohydrate diet.”  After all, equal is equal. The article is, according to the authors, about “high-protein, low-carbohydrate” so rather than describing a comparison of apples and pears, the conclusion should emphasize low carbohydrate vs high carbohydrate.   It is carbohydrate, not protein, that is the key question in diabetes but clarity was probably not the idea. The paper by Larsen, et al. [1] represents a kind of classic example of the numerous studies in the literature whose goal is to discourage people with diabetes from trying a diet based on carbohydrate restriction, despite its intuitive sense (diabetes is a disease of carbohydrate intolerance) and despite its established efficacy and foundations in basic biochemistry.  The paper is characterized by blatant bias, poor experimental design and mind-numbing statistics rather than clear graphic presentation of the data. I usually try to take a collegial approach in these things but this article does have a unique and surprising feature, a “smoking gun” that suggests that the authors were actually aware of the correct way to perform the experiment or at least to report the data.

Right off, the title tells you that we are in trouble. “The effect of high-protein, low-carbohydrate diets in the treatment…” implying that all such diets are the same even though  there are several different versions, some of which (by virtue of better design) will turn out to have had much better performance than the diet studied here and, almost all of which are not “high protein.” Protein is one of the more stable features of most diets — the controls in this experiment, for example, did not substantially lower their protein even though advised to do so –and most low-carbohydrate diets advise only carbohydrate restriction.  While low-carbohydrate diets do not counsel against increased protein, they do not necessarily recommend it.  In practice, most carbohydrate-restricted diets are hypocaloric and the actual behavior of dieters shows that they generally do not add back either protein or fat, an observation first made by LaRosa in 1980.

Atkins-bashing is not as easy as it used to be when there was less data and one could run on “concerns.” As low-fat diets continue to fail at both long-term and short-term trials — think Women’s Health Initiative [2] — and carbohydrate restriction continues to show success and continues to bear out the predictions from the basic biochemistry of the insulin-glucose axis  [3], it becomes harder to find fault.  One strategy is to take advantage of the lack of formal definitions of low-carbohydrate diets to set up a straw man.  The trick is to test a moderately high carbohydrate diet and show that, on average, as here, there is no difference in hemoglobin A1c, triglycerides and total cholesterol, etc. when compared to a higher carbohydrate diet as control —  the implication is that in a draw, the higher carbohydrate diet wins.  So, Larsen’s low carbohydrate diet contains 40 % of energy as carbohydrate.  Now, none of the researchers who have demonstrated the potential of carbohydrate restriction would consider 40 % carbohydrate, as used in this study, to be a low-carbohydrate diet. In fact, 40 % is close to what the American population consumed before the epidemic of obesity and diabetes. Were we all on a low carbohydrate diet before Ancel Keys?

What happened?  As you might guess, there weren’t notable differences on most outcomes but like other such studies in the literature, the authors report only group statistics so you don’t really know who ate what and they use an intention-to-treat (ITT) analysis. According to ITT, a research report should include data from those subjects that dropped out of the study (here, about 19 % of each group). You read that correctly.  The idea is based on the assumption (insofar as it has any justification at all) that compliance is an inherent feature of the diet (“without carbs, I get very dizzy”) rather than a consequence of bias transmitted from the experimenter, or distance from the hospital, or any of a thousand other things.  While ITT has been defended vehemently, the practice is totally counter-intuitive, and has been strongly attacked on any number of grounds, the most important of which is that, in diet experiments, it makes the better diet look worse.  Whatever the case that can be made, however, there is no justification for reporting only intention-to-treat data, especially since, in this paper, the authors consider as one of the “strengths of the study … the measurement of dietary compliance.”

The reason that this is all more than technical statistical detail, is that the actual reported data show great variability (technically, the 95 % confidence intervals are large).  To most people, a diet experiment is supposed to give a prospective dieter information about outcome.  Most patients would like to know: if I stay on this diet, how will I do.  It is not hard to understand that if you don’t stay on the diet, you can’t expect good results.  Nobody knows what 81 % staying on the diet could mean.  In the same way, nobody loses an average amount of weight. If you look at  the spread in performance and in what was consumed by individuals on this diet, you can see that there is big individual variation Also, being “on a diet”, or being “assigned to a diet” is very different than actually carrying out dieting behavior, that is, eating a particular collection of food.  When there is wide variation, a person in the low-carb group may be eating more carbs than some person in the high-carb group.  It may be worth testing the effect of having the doctor tell you to eat fewer carbs, but if you are trying to lose weight, you want them to test the effect of actually eating fewer carbs.

When I review papers like this for a journal I insist that the authors present individual data in graphic form.  The question in low-carbohydrate diets is the effect of amount of carbohydrate consumed on the outcomes.  Making a good case to the reader involves showing individual data.  As a reviewer, I would have had the authors plot each individual’s consumption of carbohydrate vs for example, individual changes in triglyceride and especially HbA1c.  Both of these are expected to be dependent on carbohydrate consumption.  In fact, this is the single most common criticism I make as reviewer or that I made when I was co-editor-in chief at Nutrition and Metabolism.

So what is the big deal?  This is not the best presentation of the data and it is really hard to tell what the real effect of carbohydrate restriction is. Everybody makes mistakes and few of my own papers are without some fault or other. But there’s something else here.  In reading a paper like this, unless you suspect that something wasn’t done correctly, you don’t tend to read the Statistical analysis section of the Methods very carefully (computers have usually done most of the work).  In this paper, however, the following remarkable paragraph jumps out at you.  A real smoking gun:

  • “As this study involved changes to a number of dietary variables (i.e. intakes of calories, protein and carbohydrate), subsidiary correlation analyses were performed to identify whether study endpoints were a function of the change in specific dietary variables. The regression analysis was performed for the per protocol population after pooling data from both groups. “

What?  This is exactly what I would have told them to do.  (I’m trying to think back. I don’t think I reviewed this paper).  The authors actually must have plotted the true independent variable, dietary intake — carbohydrate, calories, etc. — against the outcomes, leaving out the people who dropped out of the study.  So what’s the answer?

  • “These tests were interpreted marginally as there was no formal adjustment of the overall type 1 error rate and the p values serve principally to generate hypotheses for validation in future studies.”

Huh?  They’re not going to tell us?  “Interpreted marginally?”  What the hell does that mean?  A type 1 error refers to a false positive, that is, they must have found a correlation between diet and outcome in distinction to what the conclusion of the paper is.  They “did not formally adjust for” the main conclusion?  And “p values serve principally to generate hypotheses?”  This is the catch-phrase that physicians are taught to dismiss experimental results that they don’t like.  Whether it means anything or not, in this case there was a hypothesis, stated right at the beginning of the paper in the Abstract: “…to determine whether high-protein diets are superior to high-carbohydrate diets for improving glycaemic control in individuals with type 2 diabetes.”

So somebody — presumably a reviewer — told them what to do but they buried the results.  My experience as an editor was, in fact, that there are people in nutrition who think that they are beyond peer review and I had had many fights with authors.  In this case, it looks like the actual outcome of the experiment may have actually been the opposite of what they say in the paper.  How can we find out?  Like most countries, Australia has what are called “sunshine laws,” that require government agencies to explain their actions.  There is a Australian Federal Freedom of Information Act (1992) and one for the the state of Victoria (1982). One of the authors is supported by NHMRC (National Health and Medical Research Council)  Fellowship so it may be they are obligated to share this marginal information with us.  Somebody should drop the government a line.

Bibliography

1. Larsen RN, Mann NJ, Maclean E, Shaw JE: The effect of high-protein, low-carbohydrate diets in the treatment of type 2 diabetes: a 12 month randomised controlled trial. Diabetologia 2011, 54(4):731-740.

2. Tinker LF, Bonds DE, Margolis KL, Manson JE, Howard BV, Larson J, Perri MG, Beresford SA, Robinson JG, Rodriguez B et al: Low-fat dietary pattern and risk of treated diabetes mellitus in postmenopausal women: the Women’s Health Initiative randomized controlled dietary modification trial. Arch Intern Med 2008, 168(14):1500-1511.

3. Volek JS, Phinney SD, Forsythe CE, Quann EE, Wood RJ, Puglisi MJ, Kraemer WJ, Bibus DM, Fernandez ML, Feinman RD: Carbohydrate Restriction has a More Favorable Impact on the Metabolic Syndrome than a Low Fat Diet. Lipids 2009, 44(4):297-309.

“In the Viking era, they were already using skis…and over the centuries, the Norwegians have proved themselves good at little else.”

–John Cleese, Norway, Home of Giants.

With the 3-foot bookshelf of popular attacks on the low-fat-diet-heart idea it is pretty remarkable that there is only one defense.  Daniel Steinberg’s Cholesterol Wars. The Skeptics vs. The Preponderance of Evidence is probably more accurately called a witness for the prosecution since low-fat, in some way or other is still the law of the land.

The Skeptics vs. the Preponderance of Evidence

The Skeptics vs. the Preponderance of Evidence

The book is very informative, if biased, and it provides an historical perspective describing the difficulty of establishing the cholesterol hypothesis. Oddly, though,  it still appears to be very defensive for a witness for the prosecution.  In any case, Steinberg introduces into evidence the Oslo Diet-Heart Study [2] with a serious complaint:

“Here was a carefully conducted study reported in 1966 with a statistically significant reduction in reinfarction [recurrence of heart attack] rate.  Why did it not receive the attention it deserved?”

“The key element,” he says, “was a sharp reduction in saturated fat and cholesterol intake and an increase in polyunsaturated fat intake. In fact. each experimental subject had to consume a pint of soybean oil every week, adding it to salad dressing or using it in cooking or, if necessary, just gulping it down!”

Whatever it deserved, the Oslo Diet-Heart Study did receive a good deal of attention.  The Women’s Health Initiative (WHI), liked it.  The WHI was the most expensive failure to date. It found that “over a mean of 8.1 years, a dietary intervention that reduced total fat intake and increased intakes of vegetables, fruits, and grains did not significantly reduce the risk of CHD, stroke, or CVD in postmenopausal women.” [3]

The WHI, adopted a “win a few, lose a few” attitude, comparing its results to the literature, where some studies showed an effect of reducing dietary fat and some did not — this made me wonder: if the case is so clear, whey are there any failures.  Anyway, it cited the Oslo Diet-Heart Study as one of the winners and attributed the outcome to the substantial lowering of plasma cholesterol.

So, “cross-examination” would tell us why, if  “a statistically significant reduction in reinfarction  rate”  it did “not receive the attention it deserved?”

First, the effect of diet on cholesterol over five years:

The results look good although, since all the numbers are considered fairly high, and since the range of values is not shown, it is hard to tell just how impressive the results really are. But we will stipulate that you can lower cholesterol on a low-fat diet. But what about the payoff? What about the outcomes?

The results are shown in Table 5 of the original paper:   Steinberg described how in the first 5 years: “58 patients of the 206 in the control group (28%) had a second heart attack” (first 3 lines under first line of blue-highlighting) but only

“…  32 of the 206 in the diet (16%)…”  which does sound pretty good.

In the end, though, it’s really the total deaths from cardiac disease.  The second blue-highlighted line in Table 5 shows the two final outcome.  How should we compare these.

1. The odds ratio or relative risk is just the ratio of the two outcomes (since there are the same number of subjects) = CHD mortality (diet)/ CHD mortality control) = 94/79 =  1.19.  This seems strikingly close to 1.0, that is, flip of a coin.  These days the media, or the report itself, would report that there was a 19 % reduction in total CHD mortality.

2, If you look at the absolute values, however, the  mortality in the controls is 94/206 = 45.6 % but the diet group had reduced this  to 79/206 = 38.3 % so the change in absolute risk is  45.6 % – 38.3 % or only 7.3 % which is less impressive but still not too bad.

3. So for every 206 people, we save 94-79 = 15 lives, or dividing 206/15 = 14 people needed to treat to save one life. (Usually abbreviated NNT). That doesn’t sound too bad.  Not penicillin but could be beneficial. I think…

Smoke and mirrors.

It’s what comes next that is so distressing.  Table 10 pools the two groups, the diet and the control group and now compares  the effect of smoking: on the whole population,  the ratio of CHD deaths in smokers vs non-smokers is 119/54 = 2.2 (magenta highlight) which is somewhat more impressive than the 1.19 effect we just saw.  Now,

1. The absolute difference in risk is (119-54)/206 = 31.6 % which sounds like a meaningful number.

2. The number needed to treat is 206/64 = 3.17  or only about 3 people need to quit smoking to see one less death

In fact, in some sense, the Oslo Diet-Heart Study provides smoking-CHD risk as an example of a meaningful association that one can take seriously. If only such a significant change had actually been found for the diet effect.

So what do the authors make of this? Their conclusion is that “When combining data from both groups, a three-fold greater CHD mortality rate is demonstrable among the hypercholesterolemic, hypertensive smokers than among those in whom these factors were low or absent.”  Clever but sneaky. The “hypercholesterolemic, hypertensive” part is irrelevant since you combined the groups. In other words, what started out as a diet study has become a “lifestyle study.”  They might has well have said “When combining data from fish and birds a significant number of wings were evident.” Members of the jury are shaking their heads.

Logistic regression. What is it? Can it help?

So they have mixed up smoking and diet. Isn’t there a way to tell which was more important?  Well, of course, there are several ways.  By coincidence, while I was writing this post, April Smith posted on facebook, the following challenge “The first person to explain logistic regression to me wins admission to SUNY Downstate Medical School!” I won although I am already at Downstate.  Logistic regression is, in fact, a statistical method that asks what the relative contribution of different inputs would have to be to fit the outcome and this could have been done but in this case, I would use my favorite statistical method, the Eyeball Test.  Looking at the data in Tables 5 and 10 for CHD deaths, you can see immediately what’s going on. Smoking is a bigger risk than diet.

If you really want a number, we calculated relative risk above. Again, we found for mortality, CHD (diet)/ CHD (control) = 94/79 =  1.19. But what happens if you took up smoking: Figure 10 shows that your chance of dying of heart disease would be increased by 119/54 = 2.2  or more than twice the risk.  Bottom line: you decided to add saturated fat to your diet, your risk would be 1.19 what it was before which might be a chance you could take faced with authentic Foie Gras.

Daniel Steinberg’s question:

“Here was a carefully conducted study reported in 1966 with a statistically significant reduction in reinfarction  rate.  Why did it not receive the attention it deserved?”

Well, it did. This is not the first critique.  Uffe Ravnskov described how the confusion of smoking and diet led to a new Oslo Trial which reductions in both were specifically recommended and, again, outcomes made diet look bad [4].  Ravnskov gave it the attention it deserved. But what about researchers writing in the scientific literature. Why do they not give the study the attention it deserves. Why do they not point out its status as a classic case of a saturated fat risk study with no null hypothesis.  It certainly deserves attention for its devious style. Of course, putting that in print would guarantee that your grant is never funded and your papers will be hard to publish.  So, why do researchers not give the Oslo-Diet-Heart study the attention it deserves?  Good question, Dan.

Bibliography

1. Steinberg D: The cholesterol wars : the skeptics vs. the preponderance of evidence, 1st edn. San Diego, Calif.: Academic Press; 2007.

2. Leren P: The Oslo diet-heart study. Eleven-year report. Circulation 1970, 42(5):935-942.

3. Howard BV, Van Horn L, Hsia J, Manson JE, Stefanick ML, Wassertheil-Smoller S, Kuller LH, LaCroix AZ, Langer RD, Lasser NL et al: Low-fat dietary pattern and risk of cardiovascular disease: the Women’s Health Initiative Randomized Controlled Dietary Modification Trial. JAMA 2006, 295(6):655-666.

4. Ravnskov U: The Cholesterol Myths: Exposing the Fallacy that Cholesterol and Saturated Fat Cause Heart Disease. Washington, DC: NewTrends Publishing, Inc.; 2000.

In 1985 an NIH Consensus Conference was able to “establish beyond any reasonable doubt the close relationship between elevated blood cholesterol levels (as measured in serum or plasma) and coronary heart disease” (JAMA 1985, 253:2080-2086).

I have been making an analogy between scientific behavior and the activities of the legal system and following that idea, the wording of the conference conclusion suggests a criminal indictment. Since the time of the NIH conference, however, data on the role of cholesterol fractions, the so-called “good (HDL)” and “bad (LDL)” cholesterols and, most recently, the apparent differences in the atherogenicity of different LDL sub-fractions would seem to have provided some reasonable doubt. What has actually happened is that the nutrition establishment, the lipophobes as Michael Pollan calls them, has extended the indictment to include dietary fat, especially saturated fat at least as accessories on the grounds that, as the Illinois Criminal Code put it “before or during the commission of an offense, and with the intent to promote or facilitate such commission, … solicits, aids, abets, agrees or attempts to aid… in the planning or commission of the offense. . . ..”

A major strategy in the indictment of saturated fat has been guilt by association.  The American Heart Association (AHA), which had long recommended margarine (the major source of trans-fats), has gone all out in condemning saturated fatty acids by linking them with trans-fats.  The AHA website has a truly deranged cartoon film of the evil brothers: “They’re a charming pair, Sat and Trans.  But that doesn’t mean they make good friends.  Read on to learn how they clog arteries and break hearts — and how to limit your time with them by avoiding the foods they’re in.”. While the risk of trans-fats is probably exaggerated — they are a small part of the diet — they have no benefit and nobody wants to defend them; dietary saturated fat, however, is a normal part of the diet, is made in your body and is less important in providing saturated fatty acids in the blood, than dietary carbohydrate.  Guilt by association is a tricky business in courts of law — just having a roommate who sells marijuana can get you into a good deal of trouble — but it takes more than somebody saying that you and the perpetrator make a charming pair.

The failure of the diet-cholesterol-heart hypothesis in clinical trials as been documented by numerous scientific articles and especially in popular books that document the original scientific sources. It is unknown what the reaction of the public is to these books.  However, amazingly, there is only one book I know of that takes the side of the lipophobes and that is Daniel Steinberg’s Cholesterol Wars. The Skeptics vs. the Preponderance of Evidence. A serious book with careful if slightly biased documentation and an uncommon willingness to answer the critics,  it is worth reading.  I will try to discuss it in detail in this and future posts.  First, the title indicates a step down from criminal prosecution.  “Preponderance of the evidence” is the standard for conviction in a civil court and is obviously a far weaker criterion.  One has to wonder why it is that the skeptics have the preponderance of the popular publications — if the scientific evidence is there and health agencies are so determined that the public know about this, why are there so few —  maybe only this one — rebutting the critics.

The Skeptics vs. the Preponderance of Evidence

In any case, what is Steinberg’s case?  The indictment on page 1 is somewhat different than one would have thought.

“….the [lipid] hypothesis relates to blood lipids not dietary lipids as the putative directly causative factor. Although diet, especially dietary lipid is an important determinant of blood lipid levels, many other factors play important roles. Moreover, there is a great deal of variability in response of individuals to dietary manipulations. Thus, it is essential to distinguish between the indirect “diet-heart” connection and the direct “blood lipid — hard” connection failure to make this distinction has been a frequent source of confusion. (his italics)”

What?  Are we really supposed to believe that diet is an incidental part of the lipid hypothesis?  Are we supposed to believe that our cholesterol is just a question of the variability of our response to diet.  Has the message really been that diet is not critical and that heart-disease is just the luck of the draw (until we start taking statins)?  This is certainly the source of confusion in my mind.  Of course by page 5, we are confronted with this:

“In 1966, Paul Leren published his classic five-year study of 412 patients who had had a prior myocardial infarction. He showed that substitution of polyunsaturated fat and saturated fat-rich butter-cream-venison diet favored by the Norwegians reduced their blood cholesterol by about 17 per cent and kept it down.  The number of secondary current events in the treated group was reduced by about one-third and the result was significant at the p < 0.03 level.”

In a future post, I will describe Paul Leren’s classic five-year study which, by 1970, had a follow-up to eleven years and the results will turn out not to be as compelling as described by Steinberg.  For the moment, it is worth considering that, given the strong message, from the AHA, from the American Diabetes Association, from the NIH Guidelines for Americans, the criterion really should be beyond a reasonable doubt. There shouldn’t be even a single failure like the Framingham Study or the Women’s Health Initiative. In fact, the preponderance of the evidence when you add them all up, isn’t there.

The headline read “Vascular effects of a low-carbohydrate high-protein diet.”   The article, as anticipated, was trying to trash low carb diets. This is not uncommon. Those of us who work in the field are used to it.  Low carbohydrate diets are the thing that doctors and nutritionists love to hate.  Every junior faculty in a medicine department feels obligated to write a review showing how bad such diets are, frequently by saying that they don’t conform to the recommendations of the USDA, an example of what was the original meaning of “begging the question”, namely assuming the question in deducing the answer, that is: The USDA guidelines defines healthy diets. The Atkins diet says that the USDA guidelines are bad.  The Atkins diet does not conform to the USDA guidelines. The Atkins diet is unhealthy. QED.

The paper in question, however, turned out to be especially infuriating, claiming that a low-carbohydrate  diet will cause the build-up of plaque that is characterized as atherosclerosis but the experiment was absurd.   It reminded me of the kind of articles that William F. Buckley, Jr. used to write, the kind that made you wonder: does he really not see how illogical this stuff is.  By coincidence the low carb-atherosclerosis paper appeared about the same time as Gary Wills’s portrait of Buckley appeared in the Atlantic.  It was a very sympathetic review although Wills was not blind to Buckley’s faults.  I saw in Buckley’s personality this same kind of thing that was in the atherosclerosis paper. I understood pleasure in being infuriating but you can always do that while trying to get it right. I suddenly had some perspective on it all. I could see a mindset where the truth was not the main reinforcer as we say in behavioral psychology.  Buckley was simply motivated by something else. So first, I’ll tell you about the low carbohydrate diet paper.

The thing that drives the nutritional establishment crazy is not that low carbohydrate are effective for weight loss. Everybody knows that. The earliest writers on food, Brillat-Savarin for example, made the observation that the principles of fattening your pig for market by feeding her grains applied to humans as well. What gets people nuts is that the cardiovascular risk that was supposed to follow from the increase in fat did not materialize.  In fact, when actually studied, low carbohydrate diets reduced cardiovascular risk, dramatically lowering triglycerides (fat in the blood), increasing HDL (the so-called “good cholesterol”)  and improving other risk factors.  In fact, the paper by Foo, et al. in PNAS admitted “randomized trials suggest low-carbohydrate diets may accelerate weight loss with surprisingly little negative effect on serum markers of cardiac risk such as cholesterol,” but that didn’t stop the authors who were able to come up with new warnings about atherosclerosis from low-carbohydrate diets.

It wasn’t people, that developed atherosclerosis. It turns out that the low-carbohydrate diet was administered to mice.  Now this could be news since mice are not particularly susceptible to atherosclerosis possibly because they have high HDL (again, the “good cholesterol”). But this was not your ordinary mouse.  This was a mouse that was genetically engineered to be susceptible to cardiovascular disease.  This was an apoE-knockout mouse, a mouse  from which the genes for apolipoprotein E had been deleted ((ApoE-/-).  Apolipoprotein E (apoE) is one of the protein components of the cholesterol and fat-carrying particles known as lipoproteins (LP) that circulate in the blood. It is required for efficient clearance of some of these particles. One wit on the internet suggested that giving an apoE knock-out mouse atherosclerosis was the behavioral equivalent of teaching a cat to go to sleep.  Did the authors not know this was nonsense?  Did they really believe that a low carbohydrate diet had more biological importance than a difference in species and a mutation that predisposed to atherosclerosis? What were they thinking? At stake is the corruption of science and the the corruption of medicine. The coincidental appearance of Wills’s picture of Buckley was helpful:

“Bill was considered an elitist because he loved to use big words. But he did it not from hauteur but from impishness. This was part of his playfulness. He liked to play games in general, and word games were especially appealing to him. He used the big words for their own sake, even when he was not secure in their meaning. One of his most famous usages poisoned the general currency, especially among young conservatives trying to imitate him. They took oxymoron in the sense he gave it, though that was the opposite of its true meaning. He thought it was a fancier word for “contradiction,” so young imitators would say that “an intelligent liberal” was an oxymoron. But the Greek word means “something that is surprisingly true, a paradox,” as in a shrewd dumbness.”

That was it.  The Beth Israel doctors were just having fun.  But cardiology is a serious business and this couldn’t be a pattern but this was a context where getting it right counted.  Atkins was a threat and the truth wasn’t really part of the game.  Wills said that  Buckley “never considered himself an intellectual” as he was always a risk taker.  His debate-team style was not really about the truth.  He certainly gave the impression that he thought he was an intellectual but it turns out that, in the end, he was just screwing around.

So what do Dr. Foo and Dr. Rosenzweig actually believe?  The press release from Beth Israel said that Rosenzweig went off his Atkins diet because his resident Dr. Foo kvetched at him in the hospital cafeteria.  You have to wonder why he went on the diet in the first place.  A recent survey showed that physicians were more likely to recommend low fat diets to their patients while using low carbohydrate diets themselves, although maybe it is easy to talk them out of it. A survey of physicians’ knowledge was also informative. In fact most physicians did not know that low-fat diets increased triglycerides or that carbohydrates are most likely to increase them.  The exception was cardiologists who were aware of these facts.  So maybe it wasn’t surprising that Rosenzweig would have been on the low carbohydrate diet.  That he went off it does suggest that he is as cavalier with his own health as with that of his patients and the readers of PNAS.  In the end, though, this does real harm.  At least for people with diabetes, control of carbohydrates can be life-saving. Buckley at least functioned in the area of journalistic politics whose standards are ambiguous at best.  In the comparison to the Beth Israel doctors, William F. Buckley comes off as rather likable, surely an oxymoron.