Archive for the ‘The Nutrition Story’ Category

First published in October of 2011, this post announced a Q&A on line with Harvard’s Eric Rimm to answer question about the School of Public Health’s new  “Healthy Eating Plate,” its own version of nutritional recommendations to compete with the USDA’s MyPlate. A rather  limited window of one hour  was allotted for the entire country to phone in our questions.  Unfortunately HSPH was not as good at telecommunications as it is at epidemiology and the connection did not start working for a while.  The questions that I wanted to ask, however, still stand and this post is a duplicate of the original with the notice about the Q&A removed.  Harvard has been invited to participate in a panel discussion at the Ancestral Health Symposium, and we will see how these questions can be answered.

— adopted from Pops (at Louder and Smarter), the anonymous brilliant artist and admitted ne’er do well.

One of the questions surrounding USDA Nutrition Guidelines for Americans was whether so-called “sunshine laws,” like the Freedom of Information Act, were adhered to. Whereas hearings were recorded, and input from the public was solicited, there is the sense that if the letter of the law was followed, the spirit was weak.  When I and colleagues testified at the USDA hearings, there was little evidence that their representatives were listening; there was no discussion. We said our piece and then were heard no more.  In fact, at the break, when I tried to speak to one of the panel, somebody came out from backstage, I believe unarmed, to tell me that I could not discuss anything with the committee.

Harvard School of Public Health, home of  “odds ratio = 1.22,” last month published their own implementation of the one size-fits-all approach to public nutrition, the”Healthy Eating Plate.”  Their advice is full of  “healthy,” “packed with” and other self-praise that makes this mostly an infomercial for HSPH’s point of view. Supposedly a correction of the errors in MyPlate from the USDA, it seems to be more similar than different. The major similarity is the disdain for the intelligence of the American public. Comparing the two plates (below), they have exchanged the positions of fruits and vegetables.  “Grains” on MyPlate is now called “Whole Grains,” and “Protein” has been brilliantly changed to “Healthy Proteins.”  How many NIH grants were required to think of this is unknown.  Harvard will, of course, tell you what “healthy” is:, no red meat and, of course watch out for the Seventh Egg.

 

 

 

 

 

 

 

So here are the  questions that I wanted to ask:

  1. Dr. Rimm, you are recommending a diet for all Americans but even within the pattern of general recommendations, I don’t know of any experimental trial that has tested it.  Aren’t you just recommending another grand experiment like the original USDA recommendations which you are supposedly improving on?
  2. Dr. Rimm, given that half the population is overweight or obese shouldn’t there be at least two plates?
  3. Dr. Rimm, I think the American public expects a scientific document.  Don’t you think continued use of the words “healthy,” “packed with nutrients,” makes the Plate more of  an informercial for your point of view?
  4. Dr. Rimm, the Plate site says “The contents of this Web site are not intended to offer personal medical advice,” but it seems that is exactly what it is doing. If you say that you are recommending a diet that will “Lower blood pressure; reduced risk of heart disease, stroke, and probably some cancers; lower risk of eye and digestive problems,” how is that not medical advice?  Are you disowning responsibility for the outcome in advance?
  5. Dr. Rimm, more generally, how will you judge if these recommendations are successful? Is there a null hypothesis? The USDA recommendations continue from year to year without any considerations of past successes or failures.
  6. Dr. Rimm, “healthy” implies general consensus but there are many scientists and physicians with good credentials and experience who hold to different opinions. Have you considered these opinions in formulating the plate? Is there any room for dissent or alternatives?
  7. Dr. Rimm, the major alternative point of view is that low-carbohydrate diets offer benefits for weight loss and maintenance and, obviously, for diabetes and metabolic syndrome. Although your recommendations continually refer to regulation of blood sugar, it is not incorporated in the Plate.
  8. Dr. Rimm, nutritionally, fruits have more sugar, more calories, less potassium, fewer antioxidants than vegetables.  Why are they lumped together? And how can you equate beans, nuts and meat as a source of protein?
  9. Dr. Rimm, looking at the comparison of MyPlate and your Plate, it seems that all that is changed is that “healthy” has been added to proteins and “whole” has been added to grains.  If people know what “healthy” is, why is there an obesity epidemic? Or are you blaming the patient?
  10. Dr. Rimm, you are famous for disagreeing on lipids with the DGAC committee yet your name is on their report as well as on this document is supposed to be an alternative.  Do we know where you stand?
  11. Dr. Rimm, the Healthy Plate “differences” page says “The Healthy Eating Plate is based exclusively on the best available science and was not subjected to political and commercial pressures from food industry lobbyists.” This implies that the USDA recommendations are subject to such pressures.  What is the evidence for this? You were a member of the USDA panel. What pressures were brought to bear on you and how did you deal with them
  12. Dr. Rimm, the Healthy Plate still limits saturated fat even though a study from your department showed that there was, in fact, no effect of dietary saturated fat on cardiovascular disease.  That study, moreover, was an analysis of numerous previous trials, the great majority of which individually showed no risk from saturated fat. What was wrong with that study that allows you to ignore it?

*Medicineball, (colloq) a game that derives from Moneyball, in which an “unscientific culture responds, or fails to respond, to the scientific method ” in order  to stay funded.

Baseball is like church. Many attend. Few understand.

— Leo Durocher.

The movie Moneyball provides an affirmative answer to an important question in literature and drama: can you present a scene and bring out the character of a subject that is boring while, at the same time, not make the presentation boring?  The movie, and  Michael Lewis’sbook that it is based on, are about baseball and statistics!  For fans, baseball is not boring so much as incredibly slow, providing a soothing effect like fishing, interspersed with an occasional big catch. The movie stars Brad Pitt as Billy Beane, the General Manager of the Oakland Athletics baseball team in the 1990s.  A remarkably talented high school athlete, Billy Beane, for unknown reasons, was never able to play out his potential as an MLB player but, in the end, he had a decisive effect on the game at the managerial level. The question is how the A’s, with one-third of the budget of the Yankees, could have been in the play-offs three years in a row and, in 2001, could win 102 games.  The movie is more or less faithful to the book and both are as much about organizations and psychology as about sports. The story was “an example of how an unscientific culture responds, or fails to respond, to the scientific method” and the science is substantially statistical.

In America, baseball is a metaphor for just about everything. Probably because it is an experience of childhood and adolescence, lessons learned from baseball stay with us. Baby-boomers who grew up in Brooklyn were taught by Bobby Thompson’s 1951 home-run, as by nothing later, that life isn’t fair. The talking heads in Ken Burns’s Baseball who found profound meaning in the sport are good examples. Former New York Governor Mario Cuomo’s comments were quite philosophical although he did add the observation that getting hit in the head with a pitched ball led him to go into politics.

One aspect of baseball that is surprising, especially when you consider the money involved, is the extent to which strategy and scouting practices have generally ignored hard scientific data in favor of tradition and lore. Moneyball tells us about group think, self-deception and adherence to habit in the face of science. For those of us who a trying to make sense of the field of nutrition, where people’s lives are at stake and where numerous professionals who must know better insist on dogma — low fat, no red meat — in the face of contradictory evidence, baseball provides some excellent analogies.

The real stars of the story are the statistics and the computer or, more precisely, the statistics and computer guys: Bill James an amateur analyzer of baseball statistics and Paul DePodesta, assistant General Manager of the A’s who provided information about the real nature of the game and how to use this information. James self-published a photocopied book called 1977 baseball abstract: featuring 18 categories of statistical information you just can’t find anywhere else. The book was not just about statistics but was in fact a critique of traditional statistics pointing out, for example, that the concept of an “error;” was antiquated, deriving from the early days of gloveless fielders and un-groomed playing fields of the 1850s. In modern baseball, “you have to do something right to get an error; even if the ball is hit right at you, and you were standing in the right place to begin with.” Evolving rapidly, the Abstracts became a fixture of baseball life and are currently the premium (and expensive) way to obtain baseball information.

It is the emphasis on statistics that made people doubt that Moneyball could be made into a movie and is probably why they stopped shooting the first time around a couple of years ago. Also, although Paul DePodesta (above) is handsome and athletic, Hollywood felt that they should cast him as an overweight geek type played by Jonah Hill. All of the characters in the film have the names of the real people except for DePodesta “for legal reasons,” he says. Paul must have no sense of humor.

The important analogy with nutrition research and the continuing thread in this blog, is that it is about the real meaning of statistics. Lewis recognized that the thing that James thought was wrong with the statistics was that they

“made sense only as numbers, not as a language. Language, not numbers, is what interested him. Words, and the meaning they were designed to convey. ‘When the numbers acquire the significance of language,’ he later wrote, ‘they acquire the power to do all the things which language can do: to become fiction and drama and poetry … . And it is not just baseball that these numbers through a fractured mirror, describe. It is character. It is psychology, it is history, it is power and it is grace, glory, consistency….’”

By analogy, it is the tedious comparison of quintiles from the Harvard School of Public Health proving that white rice will give you diabetes but brown rice won’t or red meat is bad but white meat is not, odds ratio = 1.32. It is the bloodless, mindless idea that if the computer says so, it must be true, regardless of what common sense tells you. What Bill James and Paul DePodesta brought to the problem was understanding that the computer will only give you a meaningful answer if you ask the right question; asking what behaviors accumulated runs and won ball games, not which physical characteristics — runs fast, looks muscular — that seem to go with being a ball player… the direct analog of “you are what you eat,” or the relative importance of lowering you cholesterol vs whether you actually live or die.

As early as the seventies, the computer had crunched baseball stats and come up with clear recommendations for strategy. The one I remember, since it was consistent with my own intuition, was that a sacrifice bunt was a poor play; sometimes it worked but you were much better off, statistically, having every batter simply try to get a hit. I remember my amazement at how little effect the computer results had on the frequency of sacrifice bunts in the game. Did science not count? What player or manager did not care whether you actually won or lost a baseball game. The themes that are played out in Moneyball, is that tradition dies hard and we don’t like to change our mind even for our own benefit. We invent ways to justify our stubbornness and we focus on superficial indicators rather than real performance and sometimes we are just not real smart.

Among the old ideas, still current, was that the batting average is the main indicator of a batter’s strength. The batting average is computed by considering that a base-on-balls is not an official at bat whereas a moments thought tells you that the ability to avoid bad pitches is an essential part of the batter’s skill. Early on, even before he was hired by Billy Beane, Paul DePodesta had run the statistics from every twentieth century baseball team. There were only two offensive statistics that were important for a winning team percentage: on-base percentage (which included walks) and slugging percentage. “Everything else was far less important.” These numbers are now part of baseball although I am not enough of a fan to know the extent to which they are still secondary to the batting average.

One of the early examples of the conflict between tradition and science was the scouts refusal to follow up on the computer’s recommendation to look at a fat, college kid named Kevin Youkilis who would soon have the second highest on-base percentage after Barry Bonds. “To Paul, he’d become Euclis: the Greek god of walks.”

The big question in nutrition is how the cholesterol-diet-heart paradigm can persist in the face of the consistent failures of experimental and clinical trials to provide support. The story of these failures and the usurpation of the general field by idealogues has been told many times. Gary Taubes’s Good Calories, Bad Calories is the most compelling and, as I pointed out in a previous post, there seems to have been only one rebuttal, Steinberg’s Cholesterol Wars. The Skeptics vs. the Preponderance of Evidence. At least within the past ten year, a small group have tried to introduce new ideas, in particular that it is excessive consumption of dietary carbohydrate, not dietary fat, that is the metabolic component of the problems in obesity, diabetes and heart disease and have provided extensive, if generally un-referenced, experimental support. An analogous group tried to influence baseball in the years before Billy Beane. Larry Lucchino, an executive of the San Diego Padres described the group in baseball as being perceived as something of a cult and therefore easily dismissed. “There was a profusion of new knowledge and it was ignored.” As described in Moneyball “you didn’t have to look at big-league baseball very closely to see its fierce unwillingness to rethink any it was as if it had been inoculated against outside ideas.”

“Grady Fuson, the A’s soon to be former head of scouting, had taken a high school pitcher named Jeremy Bonderman and the kid had a 94 mile-per-hour fastball, a clean delivery, and a body that looked as if it had been created to wear a baseball uniform. He was, in short, precisely the kind of pitcher Billy thought he had trained the scouting department to avoid…. Taking a high school pitcher in the first round — and spending 1.2 million bucks to sign — that was exactly this sort of thing that happened when you let scouts have their way. It defied the odds; it defied reason. Reason, even science, was what Billy Beane was intent on bringing to baseball.”

The analogy is to the deeply ingrained nutritional tradition, the continued insistence on cholesterol and dietary fat that are assumed to have evolved in human history in order to cause heart disease. The analogy is the persistence of the lipophobes, in the face of scientific results showing, at every turn, that these were bad ideas, that, in fact, dietary saturated fat does not cause heart disease. It leads, in the end, to things like Steinberg’s description of the Multiple risk factor intervention trial. (MRFIT; It’s better not to be too clever on acronyms lest the study really bombs out): “Mortality from CHD was 17.9 deaths per 1,000 in the [intervention] group and 19.3 per 1,000 in the [control] group, a statistically nonsignificant difference of 7.1%”). Steinberg’s take on MRFIT:

“The study failed to show a significant decrease in coronary heart disease and is often cited as a negative study that challenges the validity of the lipid hypothesis. However, the difference in cholesterol level between the controls and those on the lipid lowering die was only about 2 per cent. This was clearly not a meaningful test of the lipid hypothesis.”

In other words, cholesterol is more important than outcome or at least a “diet designed to lower cholesterol levels, along with advice to stop smoking and advice on exercise” may still be a good thing.

Similarly, the Framingham study which found a strong association between cholesterol and heart disease found no effect of dietary fat, saturated fat or cholesterol on cardiovascular disease.  Again, a marker for risk is more important than whether you get sick.  “Scouts” who continued to look for superficial signs and ignore seemingly counter-intuitive conclusions from the computer still hold sway on the nutritional team.

“Grady had no way of knowing how much Billy disapproved of Grady’s most deeply ingrained attitude — that Billy had come to believe that baseball scouting was at roughly the same stage of development in the twenty-first century as professional medicine had been in the eighteenth.”

Professional medicine? Maybe not the best example.

What is going on here? Physicians, like all of us, are subject to many reinforcers but for humans power and control are usually predominant and, in medicine, that plays out most clearly in curing the patient. Defeating disease shines through even the most cynical analysis of physician’s motivations. And who doesn’t play baseball to win. “The game itself is a ruthless competition. Unless you’re very good, you don’t survive in it.”

Moneyball describes a “stark contrast between the field of play and the uneasy space just off it, where the executives in the Scouts make their livings.” For the latter, read the expert panels of the American Heat Association and the Dietary Guidelines committee, the Robert Eckels who don’t even want to study low carbohydrate diets (unless it can be done in their own laboratory with NIH money). In this

“space just off the field of play there really is no level of incompetence that won’t be tolerated. There are many reasons for this, but the big one is that baseball has structured itself less as a business and as a social club. The club includes not only the people who manage the team but also in a kind of women’s auxiliary many of the writers and commentators to follow and purport to explain. The club is selective, but the criteria for admission and retention and it is there many ways to embarrass the club, but being bad at your job isn’t one of them. The greatest offense a club member can commit is not ineptitude but disloyalty.”

The vast NIH-USDA-AHA social club does not tolerate dissent. And the media, WebMD, Heart.org and all the networks from ABCNews to Huffington Post will be there to support the club. The Huffington Post, who will be down on the President of the United States in a moment, will toe the mark when it comes to a low carbohydrate story.

The lessons from money ball are primarily in providing yet another precedent for human error, stubbornness and, possibly even stupidity, even in an area where the stakes are high. In other words, the nutrition mess is not in our imagination. The positive message is that there is, as they say in political science, validator satisfaction. Science must win out. The current threat is that the nutritional establishment is, as I describe it, slouching toward low-carb, doing small experiments, and easing into a position where they will say that they never were opposed to the therapeutic value of carbohydrate restriction. A threat because they will try to get their friends funded to repeat, poorly, studies that have already been done well. But that is another story, part of the strange story of Medicineball.

“Portion Control” is a popular buzz-word in nutrition. It has a serious and somewhat quantitative sound as if it were recently discovered and transcends what it really means which is, of course, self-control. Self-control has been around for long time and has a poor history as a dieting strategy.  Lip service is paid to how we no longer think that overeating means that you are a bad person but “portion control” is just the latest version of the moralistic approach to dieting; the sense of deprivation that accompanies traditional diets may be one of the greatest barriers to success. Getting away from this attitude is probably the main psychological benefit of low-carbohydrate diets.  “Eat all the meat you want” sounds scary to the blue-stockings at the USDA but most people who actually use such diets know that the emphasis is on “want” and by removing the nagging, people usually find that they have very little desire to clean their plate and don’t eat any more meat than they ever did.  Coupled with the satiety of fat and protein compared to carbohydrate, this is surely a major factor in the success of carbohydrate restriction.  In the big comparison trials, the low-fat trials are constrained to fix calories while the low-carbohydrate group is allowed to eat ad-libitum, and the two groups usually come out about the same total calories.

On the other hand, there is an obvious benefit to having a lean and hungry feel if not look and, as Woody Allen might have put it: eating less is good if only for caloric reasons.  So, one tactic in a low carbohydrate diet is to eat a small portion — say, one fried egg, a small hamburger — and then see if you are still hungry before having the second or third portion which while not forbidden, is also not required. The longer you wait between portions, the more satiety sets in.

(more…)

“Doctors prefer large studies that are bad to small studies that are good.”

— anon.

The paper by Foster and coworkers entitled Weight and Metabolic Outcomes After 2 Years on a Low-Carbohydrate Versus Low-Fat Diet, published in 2010, had a surprisingly limited impact, especially given the effect of their first paper in 2003 on a one-year study.  I have described the first low carbohydrate revolution as taking place around that time and, if Gary Taubes’s article in the New York Times Magazine was the analog of Thomas Paine’s Common Sense, Foster’s 2003 paper was the shot hear ’round the world.

The paper showed that the widely accepted idea that the Atkins diet, admittedly good for weight loss, was a risk for cardiovascular disease, was not true.  The 2003 Abstract said “The low-carbohydrate diet was associated with a greater improvement in some risk factors for coronary heart disease.” The publication generated an explosive popularity of the Atkins diet, ironic in that Foster had said publicly that he undertook the study in order to “once and for all,” get rid of the Atkins diet.  The 2010 paper by extending the study to 2 years would seem to be very newsworthy.  So what was wrong?  Why is the new paper more or less forgotten?  Two things.  First, the paper was highly biased and its methods were so obviously flawed — obvious even to the popular press — that it may have been a bit much even for the media. It remains to be seen whether it will really be cited but I will suggest here that it is a classic in misleading research and in the foolishness of intention-to-treat (ITT).

(more…)

Asher Peres was a physicist, an expert in information theory who died in 2005 and was remembered for his scientific contributions as well as for his iconoclastic wit and ironic aphorisms. One of his witticisms was that “unperformed research has no results ”  Peres had undoubtedly never heard of intention-to-treat (ITT), the strange statistical method that has appeared recently, primarily in the medical literature.  According to ITT, the data from a subject assigned at random to an experimental group must be included in the reported outcome data for that group even if the subject does not follow the protocol, or even if they drop out of the experiment.  At first hearing, the idea is counter-intuitive if not completely idiotic  – why would you include people who are not in the experiment in your data? – suggesting that a substantial burden of proof rests with those who want to employ it.  No such obligation is usually met and particularly in nutrition studies, such as comparisons of isocaloric weight loss diets, ITT is frequently used with no justification and sometimes demanded by reviewers.   Not surprisingly, there is a good deal of controversy on this subject.  Physiologists or chemists, hearing this description usually walk away shaking their head or immediately come up with one or another obvious reductio ad absurdum, e.g. “You mean, if nobody takes the pill, you report whether or not they got better anyway?” That’s exactly what it means.

On the naive assumption that some people really didn’t understand what was wrong with ITT — I’ve been known to make a few elementary mistakes in my life — I wrote a paper on the subject.  It received negative, actually hostile. reviews from two public health journals — I include an amusing example at the end of this post.  I even got substantial grief from Nutrition & Metabolism, where I was the editor at the time, but where it was finally published.  The current post will be based on that paper and I will provide a couple of interesting cases from the medical literature.  In the next post I will discuss a quite remarkable new instance — Foster’s two year study of low carbohydrate diets — of the abuse of common sense that is the major alternative to ITT.

To put a moderate spin on the problem, there is nothing wrong with ITT, if you explicitly say what the method shows — the effect of assigning subjects to an experimental protocol; the title of my paper was Intention-to-treat.  What is the question? If you are very circumspect about that question, then there is little problem.  It is common, however, for the Abstract of a paper to correctly state that patients “were assigned to a diet” but by the time the Results are presented, the independent variable has become, not “assignment to the diet,” but “the diet” which most people would assume meant what people ate, rather than what they were told to eat. Caveat lector.  My paper was a kind of over-kill and I made several different arguments but the common sense argument gets to the heart of the problem in a practical way.  I’ll describe that argument and also give a couple of real examples.

Common sense argument against intention-to-treat

Consider an experimental comparison of two diets in which there is a simple, discrete outcome, e.g. a threshold amount of weight lost or remission of an identifiable symptom. Patients are randomly assigned to two different diets: diet group A or diet group B and a target of, say, 5 kg weight loss is considered success. As shown in the table above, in group A, half of the subject are able to stay on the diet but, for whatever reason, half are not. The half of the patients in group A who did stay on the diet, however, were all able to lose the target 5 kg.  In group B, on the other hand, everybody is able to stay on the diet but only half are able to lose the required amount of weight. An ITT analysis shows no difference in the two outcomes, while just looking at those people who followed the diet shows 100 % success.  This is one of the characteristics of ITT: it always makes the better diet look worse than it is.

         Diet A         Diet B
Compliance (of 100 patients)   50   100
Success (reached target)   50    50
ITT success   50/100 = 50%   50/100 = 50%
“per protocol” (followed diet) success   50/50 = 100%   50/100 = 50%

Now, you are the doctor.  With such data in hand should you advise a patient: “well, the diets are pretty much the same. It’s largely up to you which you choose,” or, looking at the raw data (both compliance and success), should the recommendation be: “Diet A is much more effective than diet B but people have trouble staying on it. If you can stay on diet A, it will be much better for you so I would encourage you to see if you could find a way to do so.” Which makes more sense? You’re the doctor.

I made several arguments trying to explain that there are two factors, only one of which (whether it works) is clearly due to the diet. The other (whether you follow the diet) is under control of other factors (whether WebMD tells you that one diet or the other will kill you, whether the evening news makes you lose your appetite, etc.)  I even dragged in a geometric argument because Newton had used one in the Principia: “a 2-dimensional outcome space where the length of a vector tells how every subject did…. ITT represents a projection of the vector onto one axis, in other words collapses a two dimensional vector to a one-dimensional vector, thereby losing part of the information.” Pretentious? Moi?

Why you should care.  Case I. Surgery or Medicine?

Does your doctor actually read these academic studies using ITT?  One can only hope not.  Consider the analysis by Newell  of the Coronary Artery Bypass Surgery (CABS) trial.  This paper is astounding for its blanket, tendentious insistence on what is correct without any logical argument.  Newell considers that the method of

 “the CABS research team was impeccable. They refused to do an ‘as treated’ analysis: ‘We have refrained from comparing all patients actually operated on with all not operated on: this does not provide a measure of the value of surgery.”

Translation: results of surgery do not provide a measure of the value of surgery.  So, in the CABS trial, patients were assigned to Medicine or Surgery. The actual method used and the outcomes are shown in the Table below. Intention-to-treat analysis was, as described by Newell, “used, correctly.” Looking at the table, you can see that a 7.8% mortality was found in those assigned to receive medical treatment (29 people out of 373 died), and a 5.3% mortality (21 deaths out of 371) for assignment to surgery.  If you look at the outcomes of each modality as actually used, it turns out that that medical treatment had a 9.5% (33/349) mortality rate compared with 4.1% (17/419) for surgery, an analysis that Newell says “would have wildly exaggerated the apparent value of surgery.”

Survivors and deaths after allocation to surgery or medical treatment
Allocated medicine Allocated surgery
  Received surgery     Received medicine   Received surgery     Received medicine
Survived 2 years   48   296   354   20
Died    2    27    15    6
Total   50   323   369   26

Common sense suggests that appearances are not deceiving. If you were one of the 33-17 = 16 people who were still alive, you would think that it was the potential report of your death that had been exaggerated.  The thing that is under the control of the patient and the physician, and which is not a feature of the particular modality, is getting the surgery implemented. Common sense dictates that a patient is interested in surgery, not the effect of being told that surgery is good.  The patient has a right to expect that if they comply, the physician would avoid conditions where, as stated by Hollis,  “most types of deviations from protocol would continue to occur in routine practice.” The idea that “Intention to treat analysis is … most suitable for pragmatic trials of effectiveness rather than for explanatory investigations of efficacy” assumes that practical considerations are the same everywhere and that any practitioner is locked into the same abilities or lack of abilities as the original experimenter.

What is the take home message.  One general piece of advice that I would give based on this discussion in the medical literature: don’t get sick.

Why you should care.  Case II. The effect of vitamin E supplementation

A clear cut case of how off-the-mark ITT can be is a report on the value of antioxidant supplements. The Abstract of the paper concluded that “there were no overall effects of ascorbic acid, vitamin E, or beta carotene on cardiovascular events among women at high risk for CVD.” The study was based on an ITT analysis but,on the fourth page of the paper, it turns out that removing subjects due to

“noncompliance led to a significant 13% reduction in the combined end point of CVD morbidity and mortality… with a 22% reduction in MI …, a 27% reduction in stroke …. a 23% reduction in the combination of MI, stroke, or CVD death (RR (risk ratio), 0.77; 95% CI, 0.64–0.92 [P = 005]).”

The media universally reported the conclusion from the Abstract, namely that there was no effect of vitamin E. This conclusion is correct if you think that you can measure the effect of vitamin E without taking the pill out of the bottle.  Does this mean that vitamin E is really of value? The data would certainly be accepted as valuable if the statistics were applied to a study of the value of replacing barbecued pork with whole grain cereal. Again, “no effect” was the answer to the question: “what happens if you are told to take vitamin E” but it still seems is reasonable that the effect of a vitamin means the effect of actually taking the vitamin.

The ITT controversy

Advocates of ITT see its principles as established and may dismiss a common sense approach as naïve. The issue is not easily resolved; statistics is not axiomatic: there is no F=ma, there is no zeroth law.  A good statistics book will tell you in the Introduction that what we do in statistics is to try to find a way to quantify our intuitions. If this is not appreciated, and you do not go back to consideration of exactly what the question is that you are asking, it is easy to develop a dogmatic approach and insist on a particular statistic because it has become standard.

As I mentioned above, I had a good deal of trouble getting my original paper published and one  anonymous reviewer said that “the arguments presented by the author may have applied, maybe, ten or fifteen years ago.” This criticism reminded me of Molière’s Doctor in Spite of Himself:

Sganarelle is disguised as a doctor and spouts medical double-talk with phony Latin, Greek and Hebrew to impress the client, Geronte, who is pretty dumb and mostly falls for it but:

Geronte: …there is only one thing that bothers me: the location of the liver and the heart. It seemed to me that you had them in the wrong place: the heart is on the left side but the liver is on the right side.

Sgnarelle: Yes. That used to be true but we have changed all that and medicine uses an entirely new approach.

Geronte: I didn’t know that and I beg your pardon for my ignorance.

 In the end, it is reasonable that scientific knowledge be based on real observations. This has never before been thought to include data that was not actually in the experiment. I doubt that nous avons changé tout cela.

The headline in the BBC News is “Fat ‘disrupts sugar sensors causing type 2 diabetes’” The article does not attribute the quotation in the headline and the first sentence says “US researchers say they have identified how a high-fat diet can trigger type 2 diabetes, in experiments on mice and human tissue.”  Should “mice” or “tissue” have been in the headline?  Should the article itself point out the extent to which mice respond differently, sometimes, oppositely from humans, to high-fat diets?  How strong is the evidence in light of other work?  Is the article altogether prejudicing the reader against fat which is the official position of both private and government health agencies?  The article in question may have some sins of omission but it is certainly restrained if not actually circumspect. The general problem, of course, is whether we get accurate scientific information from the popular media.

Peter Farnham and I and a group of bloggers (Laura Dolszon, Tom Naughton and Jimmy Moore) will speak at the end of the month at a conference produced by the Office of Research Integrity (ORI)) where we will raise several issues in the ethical conduct of current nutritional research. The conference, in general, tries to explore a number of questions on the interaction of science and  society.  The goals are to “discuss the latest research on research integrity…education in the responsible conduct of research; responsible research practices.” While each presenter will have only 15 minutes, this is one of the first times thatd the practices in nutrition with regard to issues of integrity are being addressed.  There are four areas that we will discuss:

Crisis in Nutrition I: The Popular Media and Research Publications

Richard David Feinman, SUNY Downstate Medical Center

Crisis in Nutrition II: Research Integrity in Meeting the Challenge  of Carbohydrate Restriction

Richard David Feinman, SUNY Downstate  Medical Center

Crisis in Nutrition III:  Was the Government Standard Met by the  2010 Dietary Guidelines?

Peter Farnham, Nutrition and Metabolism  Society, Alexandria, VA

Crisis in Nutrition IV:  Vox Populi 

Tom Naughton, Jimmy Moore, Laura Dolson, Independent Writers Franklin TN

The abstract of my first talk is presented below. It is, of course, a tricky area. Within some legal limits, reporters can say what they like.  A researcher speaking in a public venue, personal blog, social media can similarly pretty much sound of as they choose.  Or can they? If they are identified as an expert or are have credentials based on a employment by a prestigious institution, don’t they have to clearly distinguish between opinion and fact? And does the headline have to say that, for example, the high-fat study was done in mice?  All of these are gray areas and motives are hard to discern.  I focussed on one area that seemed more clear cut.  If an experimental study is reported in the media or in a press release from an academic institution (sometimes the same thing), is there an obligation to be sure that any opinions attributed to the investigator derives from that research unless otherwise indicated?

 Crisis in nutrition: I. The popular media and research publications  

Objective:

The public relies on popular media for descriptions of nutrition research.  Of particular interest is carbohydrate-restricted diets, the major challenge to official recommendations.  The goal is to assess the extent to which statements to the media and press releases accurately represent the results of research.

Summary of findings or main points:

Nutrition is an area of great interest to the public but one where matters of scientific fact and policy are contentious. Authors of research papers should sensibly have great freedom in describing of the implications of their research, but have an important role in explaining to the public when that opinion does or does not follow directly from the publication.  Two examples are given of where this is a critical issue. In one, an animal study (Foo, et al. Proc Natl Acad Sci USA 2009, 106: 15418-15423), the accompanying press release implies that it was motivated by observations of patients in a hospital, observations which were purely anecdotal and unsubstantiated.  In a second example, a press release stated that carbohydrate-restricted diets (CRDs) were not included in a comparative study because of their low compliance (Sacks, et al. N Engl J Med 2009, 360: 859-873). No data were given to support this allegation and, it is, in fact not true.  The study concluded that the macronutrient composition of the diet was not important even though, as implemented, dietary intake was the same for the groups studied and, again, the CRD was not included in the study.  It seems likely that that this would have an inhibiting effect on individuals choosing a CRD and represents an important impact of research integrity issues on the community.

Conclusions & recommendations:

Practices where research directly affects the community should be evaluated and guidelines should be generated by academic societies, scientific journals and the popular media. What constitutes appropriate press description of published research should be defined. Reasonable principles are that only those specific conclusions that derive directly from the publication are relevant and authors make clear what is their personal opinion and what is the product of research data.

 Office of Research Integrity

It is important to emphasize that ORI is sponsor of this academic conference and is not related to is regulatory function.  ORI is charged with overseeing federally funded research and emphasizes its role as watchdog in detection and prevention of research misconduct, assisting the Office of the General Counsel (OGC), dealing with suspected retaliation against whistleblowers, and responding to Freedom of Information and Privacy Acts.  Its usual activity involves pinpointing specific fraud.  The website reports, for example, a final judgement against an Assistant Professor at the Boston University School of Medicine Cancer Research Center who published two papers in which he had fabricated data shown as figures in the paper.  He is required to retract the papers and not enter into contracts or sit on advisory panels for two years.

ORI, in general, has the same relation to the research community that Internal Affairs has to the police.  Of course, in research, although there are substantive rewards, blatant fraud is generally pathological: if the results you are falsifying are important, they will surely be repeated and the misbehavior discovered while, if they are not important, the rewards are not likely to be spectacular although, especially these days, keeping your job is desirable.  The suspicion about ORI is also bolstered by their behavior in the Baltimore case in which they were part of the mindless zeal and witch-hunting whose appearance in human interactions seems to have such a low activation energy.

David Baltimore, a Nobel laureate in Physiology or Medicine and, at the time, MIT professor, had co-authored a paper with an immunologist named  Thereza Imanishi-Kari who was accused by a postdoctoral fellow of fabricating data. In the end, nothing came of it but there was much sound and fury and many idiots participated in the tale including the Secret Service (who I was taught were only supposed to protect the President and prosecute counterfeiters). The details of the Baltimore case are well told in capsule form in the Wikipedia entry. Daniel Kevles wrote an outstanding book, at least judged by the first half — the content was too infuriating for me to keep reading.  In any case, after most of the furor had died down, the ORI persisted and found Imanishi-Kari guilty of research misconduct, a ruling overturned by an appeals panel of the Department of Health and Human Services (HHS) which “found that much of what ORI presented was irrelevant, had limited probative value, was internally inconsistent, lacked reliability or foundation, was not credible or not corroborated, or was based on unwarranted assumptions.”  the ORI reputation has probably not recovered from this but it remains one of the few oversight agencies which, at least in nutrition, is sorely needed.

David Baltimore wrote his own description of the events and emphasized that it raise many questions, in particular,  “Who should judge science?” and “How does one distinguish between error and fraud? And does science do an adequate job of policing itself?”  The conference and this blog will discuss these matters which the crisis in nutrition has made of critical importance.  Such philosophical questions were formerly what most of us would have preferred to simply gab about in Starbucks.

“Headlines” is one of Jay Leno’s routines on The Tonight Show. While low on production values, it provides amusing typos, odd juxtapositions of text and inappropriate couplings from real notices and newspapers. The headlines are frequently very funny since, like fiction in general, authored comedy has to be plausible. There have been many other versions of the same idea including items in the New Yorker but Jay Leno’s audience rapport adds to the impact. Expert as he is, though, Jay seemed a little off guard when nobody laughed at the headline: “The Diabetes Discussion Group will meet at 10 AM right after the pancake breakfast.” It’s probably generational. After 30 or so years having the American Diabetes Association tell you that sugar is Ok as long as you “cover it with insulin” and that diabetes, a disease of carbohydrate intolerance, is best treated by adding carbohydrate and reducing fat, who knows what anybody believes.

One of the headlines on a previous show that did get a laugh said: “To increase gas mileage, drive less.”  (If Jay only knew how much we spent to get the USDA committee to come up with the advice that if you want to lose weight, you should eat less).

“.. Have we eaten on the insane root,
That takes the reason prisoner?”
— William Shakespeare, Macbeth.

For tragic humor in the bizarre field of diabetes information, it is really hard to compete. About the same time as the headlines sequence on the Tonight Show, DiabetesHealth  an organization and website that is intended to “investigate, inform, inspire” produced an inspiring investigation from the literature. The story is entitled “Maple Syrup – A Sweet Surprise.”  You gotta’ read this:

 “Meet the latest superfood: maple syrup.  Wait a minute…maple syrup? The super-sugary stuff poured on pancakes and waffles and used to glaze hams? That maple syrup? That’s right. Researchers from the University of Rhode Island have discovered that the syrup-produced in the northeastern United States and Canada–contains numerous compounds with real health benefits.”

So how did people with diabetes fare on the maple syrup? Well, there were no people. Or animals. The researchers did not test the effect of consumed maple syrup but only chemically analyzed samples of the stuff.

“‘In our laboratory research, we found that several of these compounds possess anti-oxidant and anti-inflammatory properties, which have been shown to fight cancer, diabetes, and bacterial illnesses,’ said Navindra Seeram, an assistant professor of pharmacognosy (the study of medicines derived from natural sources) at the university and the study’s lead author”

“Pharmacognosy,” incidentally, is the only English word correctly pronounced through the nose.  The article indicates that “a paper describing their results will appear in the Journal of Functional Foods. Scientists hope that these discoveries could lead to innovative treatments as the beneficial substances are synthesized to create new kinds of medicine.”  The article, however, is nothing if not circumspect:

“You might want to pause for a moment before rushing out and buying jug after jug of Canada’s finest maple syrup, though. It still contains plenty of sugar,…” In fact, by far the major ingredient in maple syrup is sucrose which, again, only has to be “covered” with insulin. So, with all those beneficial compounds, we will need less insulin per gram of sucrose with maple syrup, right?    Would Jay Leno have gotten a laugh if the diabetes meeting followed the pancakes and maple syrup breakfast?  How about if they were whole grain pancakes?

“If you can look into the seeds of time,
And say which grain will grow, and which will not…”
— William Shakespeare, Macbeth.

Not to be outdone, the American Diabetes Association website offers the lowdown on just how good grain is. Fiber, in general, is so good for you that you should be careful not to snarf it up too fast. As they point out, it is “important that you increase your fiber intake gradually, to prevent stomach irritation, and that you increase your intake of water and other liquids, to prevent constipation.” Doesn’t really sound all that healthy but foods with fiber “have a wealth of nutrition, containing many important vitamins and minerals.” Now, vitamin deficiency has always seemed to me to be the least of our nutritional problems but there’s more: “In fact,” using fact in its non-traditional meaning, fiber “may contain nutrients that haven’t even been discovered yet!” (their exclamation point). Not to belabor all the metaphors here, the ADA, long telling us that people with diabetes deserve to have their carbs, are surely offering pie in the sky.