Posts Tagged ‘In the face of contradictory evidence’

Confronting the Crisis in Nutrition. The Office of Research Integrity Conference on Quest for Research Excellence.

Posted: March 12, 2012 in Crisis in Nutrition, low-carbohydrate diet, Research Integrity, The Nutrition Story, Volek-Westman principle
Tags: , , , , ,
20

The Office of Research Integrity is hosting a conference on the Quest for Research Excellence and, for the first time, there is session that directly confronts policy and The Crises in Nutrition. The Speakers will delineate the problem — the two worlds of establishment nutrition and the major challenge of low carbohydrate diets, the growing problems of childhood obesity and our failure to deal with it, the confusion in the popular press on scientific issues, and finally, the voice of the patient, the failure to listen to the people who are dissatisfied with official guidelines and who have found workable solutions themselves. Three specific goals are recommended: 1) open hearings in which all researchers are represented, 2) funding research in which all people in low carbohydrate research work with others and finally, 3) a new oversight agency from NSF or Office of Research and Technology Policy.

The three goals may be a useful crystallizing point for moving forward. What can you do?

  1. Contact your elected officials and copy one of the authors from the conference. Use the Abstracts below as a basis for your own version of what needs to be done. The three goals can be more narrowly focussed for your own interests.
  2. Encourage local media to cover the meeting. Information is at http://ori.hhs.gov and the speakers can be contacted directly.
  3. Publicize your version of the three goals on your blog, your facebook page or other social media.

2011 Office of Research Integrity Conference Washington DC

Quest for Research Excellence, March 15, 2012.

Session on Crisis in Nutrition.

Chair: Richard David Feinman Contact Information: feinman@mac.com (917) 554-7794

Introduction and Abstracts.

The interest in nutrition for general health and for the prevention and treatment of disease is probably greater than at any time in history. A fairly large research community has grown up to provide information on the subject but the excellence of the results and their ability to inform the general public is highly questionable. The prospect for the future quality of research is similarly discouraging. This session focusses on a crisis in nutrition: the confusion in the public’s mind and the lack of accountability of official agencies and their failure to consider minority points of view. Four areas are considered in this session: the need to consider work that has been done on carbohydrate restriction (the major alternative to current recommendations), the limitations of current media representations of research, the problem of childhood obesity, and finally, the failure to listen to the patients who have not been well served by current ideas and who have discovered alternatives for themselves. The public, athrough forums and comments to blogs and other social media, have expressed substantial dissatisfaction with the current state of medical nutrition.

Three approaches are suggested as first steps for resolving the current crises:

  1. First, we need hearings to be held by congress or HHS in which all major researchers in nutrition are represented. We have to have everybody in the game. The USDA guidelines committee, the American Health Association nutrition panels have to meet with their critics. In particular, researchers in dietary carbohydrate restriction should be able to meet and discuss issues with their critics. This is what the government can do. Better than taxation or other punitive measures, they can bring out the information. The NIH or congress should hold meaningful hearings where all sides are heard.
  1. Second, we need to fund a study in which researchers in dietary carbohydrate restriction and critics of such diets cooperate to design a long-term comparison of CRD and low-fat diets, Mediterranean diets or whatever. The groups agree on methods of procedure, make-up of the diets, how compliance will be effected, and what parameters will be measured. They “write the paper first, leaving room for the data,” that is, they agree in advance on what the possible outcomes are and what conclusions could be drawn from them. In this way, the public and other scientists will have a sense that the issues have adequately been addressed and the results reliably evaluated.
  1. Finally, what’s needed is the creation of a new oversight organization, possibly under the auspices of the National Science Foundation or the Office of Science and Technology Policy in which scientists with no personal stake in nutrition, assess bias in grant awards and publications. The scientific principles involved in nutrition are neither so technical nor so profound that accomplished scientists from other fields cannot evaluate them. Such organizations might make recommendations (or indicate the limitations in existing knowledge that prevent making recommendations) after hearing all credentialed experts.

In the end, we have to say whether there is really a problem or not. Is their really an epidemic of obesity and overweight? Is there a crisis in the incidence of diabetes, or not? Are our health problems, the rising cost, the patient suffering, real? If they’re real, we have to use everything we have. We have to have real science and we can’t continue with one expert committee after another making recommendations but taking no responsibility for outcomes and refusing to show any willingness to confront their critics.

Crisis in nutrition: I. Research Integrity and the Challenge of Carbohydrate Restriction.

Author: Richard David Feinman.

Objective: Research integrity extends beyond falsification of data and explicit misconduct. We assessed the extent to which established majority opinion recommending dietary fat and saturated fat reduction has failed to cite contradictory evidence, accepted undocumented conclusions and marginalized contributions of alternative points of view, specifically the role of dietary carbohydrate restriction, the major challenge to current recommendations..

Main points: Government and private health agencies have long recommended a reduction in dietary fat, particularly saturated fat, in the treatment or prevention of cardiovascular disease, obesity and diabetes. While there are many disclaimers, low-fat in some form remains the standard nutritional recommendation. Alternative strategies based on control of insulin fluctuations via carbohydrate restriction, while widely used by many in the community, have been discouraged if not actually attacked. This has contributed to a “two worlds” system that has increased confusion among scientists and the public. While there are many exceptions and some emerging acceptance of carbohydrate restriction — which frequently fails to cite earlier work — there is a perception of a majority opinion with pervasive control of the scientific infrastructure: editorial boards, study sections and health agency administration. Examples will be given of undocumented negative statements about low-carbohydrate diets, misrepresentation of data and extensive failure to cite relevant papers from the literature. Most troubling is the tendency to accept some of the conclusions previously demonstrated in low-carbohydrate research without, again, giving appropriate citations to that research. This has led a significant part of the population to distrust official recommendations and medical science.

There is a need to re-evaluate published data on carbohydrate restriction and to guarantee adequate peer review of future manuscripts and grant applications on macronutrient composition of the diet. More generally, better communication and cooperation between researchers and physicians with different opinions can only benefit science and society, a society that is admittedly not making good progress on obesity, diabetes and metabolic syndrome.

Conclusions & Recommendations:

Recommendations for better integration of different points of view include government-sponsored meetings where all scientific approaches can present their own opinions and address critics, representation on study sections and editorial boards of people with experience in carbohydrate restriction-insulin control diets and long term comparative trials that include PIs with experience and understanding of the role of the glucose-insulin axis in obesity, diabetes and metabolic syndrome. Agreement in advance between the “two worlds” as to the expected outcomes and interpretations would provide most benefit for the public and scientist-community interactions. Given the pervasiveness of the problem, in the end, intervention of new oversight agencies, e.g. from NSF or Office of Science and Technology, may be needed

Figure 1. Comparison of low-carbohydrate diets to low-GI diets and high cereal diets.

Crisis in nutrition: II. The popular media and research publications  

Author: Richard David Feinman.

Objective: The public relies on popular media for description of nutrition research. A major interest is the controversy over macronutrient composition of the diet and particularly the role carbohydrate-restriction, the major challenge to official recommendations. The goal is to assess the extent to which statements to the media and especially press releases from authors, author institutions and journals accurately represent the results of reported research. To determine the extent to which personal bias influences and is taken as fact by the media.

Main points: Authors of research papers should sensibly have great freedom in describing the implications of their research to the media, but it is important that the public be aware of when that opinion does or does not follow directly from the publication. Two examples are given. In one, an animal study (Foo, et al. Proc Natl Acad Sci USA 2009, 106: 15418-15423), the accompanying press release implied that it was motivated by observations of patients in a hospital which were not described, were unsubstantiated and would have been purely anecdotal. In a second example, a press release stated that carbohydrate-restricted diets (CRDs) were not included in a comparative study because of their low compliance (Sacks, et al. N Engl J Med 2009, 360: 859-873. No data were given to support this assertion and it is, in fact not true — CRDs have, on average, better compliance than other dietary interventions. The study concluded that the macronutrient composition of the diet was not important even though, as implemented, dietary intake was the same for the groups studied and, again, the CRD was not included in the study. It seems likely that that this would have an inhibiting effect on the willingness of individuals to choose a CRD, an outcome that was not justified by the published research.

Conclusions & recommendations: Practices should be evaluated and guidelines should be generated by academic societies, scientific journals and the popular media as to what constitutes appropriate press description of published research. Reasonable principle are that only those specific conclusions that derive directly from the publication. The generally accepted idea that authors make clear what is their personal opinion and what is the product of research should be the norm.

Biography: Richard David Feinman, PhD in Chemistry (University of Oregon) is Professor of Cell Biology at SUNY Downstate Medical Center. His current area of research is nutritional biochemistry and biochemical education especially as it relates to macronutrients and bioenergetics. He is founder of the Nutrition & Metabolism Society and former co-editor-in-chief of the journal Nutrition & Metabolism.

Figure 2. The world according to Reuters. Low-fat is good. It’s bad. It’s not as bad as we thought. Wait! Eat more fruits and vegetables. “The low-dat diet craze?” Is that what it’s been? Is?

Crisis in nutrition: III. Childhood Obesity: Prevention and Intervention  

Author: Wendy Knapp Pogozelski, Dept of Chemistry, SUNY Geneseo, Geneseo, NY 14454.

Objective: Almost one-third of American children aged 2-11 qualify as obese or overweight, with obesity-related diseases such as type 2 diabetes greatly on the rise in this population. Despite the labeling of the crisis as “epidemic,” funding to study childhood obesity has been limited and restricted to the traditional intervention strategies (to reduce calories, to reduce dietary fat and to exercise more) despite the fact that these efforts have been largely unsuccessful. The time has come for frank assessment of foundational beliefs about a) the causes of obesity in children and b) effective prevention and intervention strategies. This talk will discuss assumptions that are barriers to research and will compare results from traditional calorie-restriction programs with results from programs that have emphasized carbohydrate control and insulin reduction.

Main points: The current generation of children is predicted to be the first to experience a lower life expectancy than that of its parents. Children across the world are experiencing unparalleled rates of obesity, heart disease and type 2 diabetes. Relatively little formal research has addressed the causes of childhood obesity, perhaps due to an assumption that the problem is already understood. Despite reluctance to use children as subjects in studies that depart from the traditional “eat less and exercise more” philosophies, it has been noted that the current efforts, dietary recommendations, educational programs and mandates of school lunch programs could be characterized as experiments. These experiments, like the numerous interventions based on traditional strategies, have had poor results but it has been very difficult to implement or fund those approaches that focus on carbohydrate control despite demonstrable success in this area. We will examine typical meals given in schools and at home, compare data from various obesity interventions and discuss causes of obesity on a molecular level

Conclusions & recommendations: The crisis warrants policy change. 1) Funding for childhood obesity should be increased. 2) A broader range of methods and principal investigators should be instituted, with greater accountability required of funded investigators. 3) The USDA nutritional recommendations, a “one size fits all” guide for school meal programs should be reevaluated and reformulated to take into account all strategies for obesity prevention and intervention. 4) Education for physicians, dietitians and health care professionals, as well as the general public, should be altered to include an understanding of the most positive results in obesity prevention.

Biography: Wendy Pogozelski, PhD in Chemistry (Johns Hopkins University) is Professor of Chemistry at SUNY Geneseo. Her research has been in radiation effects, DNA damage, and DNA computing. Current efforts are directed toward biochemical-based nutrition education for health professionals, educators and the general public. In addition to developing teaching materials that incorporate nutrition research, Dr. Pogozelski writes and lectures on diabetes and works with local and national organizations to improve nutrition education.

Figure 3. Before and After from James Bailes’s No More Fat Kids

Crisis in nutrition: IV. Vox Populi

Authors: Tom Naughton, Jimmy Moore, Laura Dolson

Objective: Blogs and other social media provide insights into how the public views the current state of nutrition science and the official dietary recommendations. We ask what can be learned from online discussions among people who dispute and distrust the official recommendations.

Main points: A growing share of the population no longer trusts the dietary advice offered by private and government health agencies. They believe the supposed benefits of the low-fat, grain-based diets promoted by those agencies are not based on solid science and that benefits of low-carbohydrate diets have been deliberately squelched. The following is typical of comments the authors (whose websites draw a combined 1.5 million visitors monthly) receive daily:

“The medical and pharmaceutical companies have no interest in us becoming healthy through nutrition. It is in their financial interest to keep us where we are so they can sell us medications.”

Similar distrust of the government’s dietary recommendations has been expressed by doctors and academics. The following comments, left by a physician on one of the authors’ blogs, are not unusual:

“You and Denise Minger should collaborate on a book about the shoddy analysis put out by hacks like the Dietary Guidelines Advisory Committee.”

“Sometimes I wonder if people making these statements even took a basic course in biochemistry and physiology.”

Many patients have given up on their health care professionals and turn to Internet sites for advice they trust. This is particularly true of people with diabetes who find that a low-fat, high-carbohydrate diet is not helping them control their blood glucose. As one woman wrote about her experience with a diabetes center:

“I was so frustrated, I quit going to the center for check ups.”

The data suggest a serious problem in science-community interactions which needs to be

explored.

Conclusions & recommendations: Our findings document a large number of such cases pointing to the need for public hearings and or conference. The community is not well served by an establishment that refuses to address its critics from within the general population as well as health professionals.

Figure 4. Some comments from the Active Low-Carber Forums (140, 660 members on March 12, 2012).

Biographies:

Tom Naughton is a former writer for a health magazine, a contributor to the Encylopedia Britannica’s Health and Medical Annual, a documentary filmmaker, and popular blogger who specializes in health and nutrition issues.

Jimmy Moore’s top-rated “Livin’ La Vida Low-Carb” blog has drawn more than 6 million visitors since 2005. His podcast show, “The Livin’ La Vida Low-Carb Show with Jimmy Moore” has featured interviews with hundreds of respected doctors and researcher. He has also authored two books.

Laura Dolson, MS is a writer and cancer support provider at Mediconsult.com, and hastaught health and nutrition classes at a junior high charter school in California. Her About.com nutrition website draws hundreds of thousands of visitors monthly.


The Nutrition Mess. Lessons from Moneyball.

Posted: September 23, 2011 in Evidence Based Medicine, Lipophobes, low-carbohydrate diet, The Nutrition Story
Tags: , , , , , , ,
10

Baseball is like church. Many attend. Few understand.

— Leo Durocher.

The movie Moneyball provides an affirmative answer to an important question in literature and drama: can you present a scene and bring out the character of a subject that is boring while, at the same time, not make the presentation boring?  The movie, and  Michael Lewis’sbook that it is based on, are about baseball and statistics!  For fans, baseball is not boring so much as incredibly slow, providing a soothing effect like fishing, interspersed with an occasional big catch. The movie stars Brad Pitt as Billy Beane, the General Manager of the Oakland Athletics baseball team in the 1990s.  A remarkably talented high school athlete, Billy Beane, for unknown reasons, was never able to play out his potential as an MLB player but, in the end, he had a decisive effect on the game at the managerial level. The question is how the A’s, with one-third of the budget of the Yankees, could have been in the play-offs three years in a row and, in 2001, could win 102 games.  The movie is more or less faithful to the book and both are as much about organizations and psychology as about sports. The story was “an example of how an unscientific culture responds, or fails to respond, to the scientific method” and the science is substantially statistical.

In America, baseball is a metaphor for just about everything. Probably because it is an experience of childhood and adolescence, lessons learned from baseball stay with us. Baby-boomers who grew up in Brooklyn were taught by Bobby Thompson’s 1951 home-run, as by nothing later, that life isn’t fair. The talking heads in Ken Burns’s Baseball who found profound meaning in the sport are good examples. Former New York Governor Mario Cuomo’s comments were quite philosophical although he did add the observation that getting hit in the head with a pitched ball led him to go into politics.

One aspect of baseball that is surprising, especially when you consider the money involved, is the extent to which strategy and scouting practices have generally ignored hard scientific data in favor of tradition and lore. Moneyball tells us about group think, self-deception and adherence to habit in the face of science. For those of us who a trying to make sense of the field of nutrition, where people’s lives are at stake and where numerous professionals who must know better insist on dogma — low fat, no red meat — in the face of contradictory evidence, baseball provides some excellent analogies.

The real stars of the story are the statistics and the computer or, more precisely, the statistics and computer guys: Bill James an amateur analyzer of baseball statistics and Paul DePodesta, assistant General Manager of the A’s who provided information about the real nature of the game and how to use this information. James self-published a photocopied book called 1977 baseball abstract: featuring 18 categories of statistical information you just can’t find anywhere else. The book was not just about statistics but was in fact a critique of traditional statistics pointing out, for example, that the concept of an “error;” was antiquated, deriving from the early days of gloveless fielders and un-groomed playing fields of the 1850s. In modern baseball, “you have to do something right to get an error; even if the ball is hit right at you, and you were standing in the right place to begin with.” Evolving rapidly, the Abstracts became a fixture of baseball life and are currently the premium (and expensive) way to obtain baseball information.

It is the emphasis on statistics that made people doubt that Moneyball could be made into a movie and is probably why they stopped shooting the first time around a couple of years ago. Also, although Paul DePodesta (above) is handsome and athletic, Hollywood felt that they should cast him as an overweight geek type played by Jonah Hill. All of the characters in the film have the names of the real people except for DePodesta “for legal reasons,” he says. Paul must have no sense of humor.

The important analogy with nutrition research and the continuing thread in this blog, is that it is about the real meaning of statistics. Lewis recognized that the thing that James thought was wrong with the statistics was that they

“made sense only as numbers, not as a language. Language, not numbers, is what interested him. Words, and the meaning they were designed to convey. ‘When the numbers acquire the significance of language,’ he later wrote, ‘they acquire the power to do all the things which language can do: to become fiction and drama and poetry … . And it is not just baseball that these numbers through a fractured mirror, describe. It is character. It is psychology, it is history, it is power and it is grace, glory, consistency….’”

By analogy, it is the tedious comparison of quintiles from the Harvard School of Public Health proving that white rice will give you diabetes but brown rice won’t or red meat is bad but white meat is not, odds ratio = 1.32. It is the bloodless, mindless idea that if the computer says so, it must be true, regardless of what common sense tells you. What Bill James and Paul DePodesta brought to the problem was understanding that the computer will only give you a meaningful answer if you ask the right question; asking what behaviors accumulated runs and won ball games, not which physical characteristics — runs fast, looks muscular — that seem to go with being a ball player… the direct analog of “you are what you eat,” or the relative importance of lowering you cholesterol vs whether you actually live or die.

As early as the seventies, the computer had crunched baseball stats and come up with clear recommendations for strategy. The one I remember, since it was consistent with my own intuition, was that a sacrifice bunt was a poor play; sometimes it worked but you were much better off, statistically, having every batter simply try to get a hit. I remember my amazement at how little effect the computer results had on the frequency of sacrifice bunts in the game. Did science not count? What player or manager did not care whether you actually won or lost a baseball game. The themes that are played out in Moneyball, is that tradition dies hard and we don’t like to change our mind even for our own benefit. We invent ways to justify our stubbornness and we focus on superficial indicators rather than real performance and sometimes we are just not real smart.

Among the old ideas, still current, was that the batting average is the main indicator of a batter’s strength. The batting average is computed by considering that a base-on-balls is not an official at bat whereas a moments thought tells you that the ability to avoid bad pitches is an essential part of the batter’s skill. Early on, even before he was hired by Billy Beane, Paul DePodesta had run the statistics from every twentieth century baseball team. There were only two offensive statistics that were important for a winning team percentage: on-base percentage (which included walks) and slugging percentage. “Everything else was far less important.” These numbers are now part of baseball although I am not enough of a fan to know the extent to which they are still secondary to the batting average.

One of the early examples of the conflict between tradition and science was the scouts refusal to follow up on the computer’s recommendation to look at a fat, college kid named Kevin Youkilis who would soon have the second highest on-base percentage after Barry Bonds. “To Paul, he’d become Euclis: the Greek god of walks.”

The big question in nutrition is how the cholesterol-diet-heart paradigm can persist in the face of the consistent failures of experimental and clinical trials to provide support. The story of these failures and the usurpation of the general field by idealogues has been told many times. Gary Taubes’s Good Calories, Bad Calories is the most compelling and, as I pointed out in a previous post, there seems to have been only one rebuttal, Steinberg’s Cholesterol Wars. The Skeptics vs. the Preponderance of Evidence. At least within the past ten year, a small group have tried to introduce new ideas, in particular that it is excessive consumption of dietary carbohydrate, not dietary fat, that is the metabolic component of the problems in obesity, diabetes and heart disease and have provided extensive, if generally un-referenced, experimental support. An analogous group tried to influence baseball in the years before Billy Beane. Larry Lucchino, an executive of the San Diego Padres described the group in baseball as being perceived as something of a cult and therefore easily dismissed. “There was a profusion of new knowledge and it was ignored.” As described in Moneyball “you didn’t have to look at big-league baseball very closely to see its fierce unwillingness to rethink any it was as if it had been inoculated against outside ideas.”

“Grady Fuson, the A’s soon to be former head of scouting, had taken a high school pitcher named Jeremy Bonderman and the kid had a 94 mile-per-hour fastball, a clean delivery, and a body that looked as if it had been created to wear a baseball uniform. He was, in short, precisely the kind of pitcher Billy thought he had trained the scouting department to avoid…. Taking a high school pitcher in the first round — and spending 1.2 million bucks to sign — that was exactly this sort of thing that happened when you let scouts have their way. It defied the odds; it defied reason. Reason, even science, was what Billy Beane was intent on bringing to baseball.”

The analogy is to the deeply ingrained nutritional tradition, the continued insistence on cholesterol and dietary fat that are assumed to have evolved in human history in order to cause heart disease. The analogy is the persistence of the lipophobes, in the face of scientific results showing, at every turn, that these were bad ideas, that, in fact, dietary saturated fat does not cause heart disease. It leads, in the end, to things like Steinberg’s description of the Multiple risk factor intervention trial. (MRFIT; It’s better not to be too clever on acronyms lest the study really bombs out): “Mortality from CHD was 17.9 deaths per 1,000 in the [intervention] group and 19.3 per 1,000 in the [control] group, a statistically nonsignificant difference of 7.1%”). Steinberg’s take on MRFIT:

“The study failed to show a significant decrease in coronary heart disease and is often cited as a negative study that challenges the validity of the lipid hypothesis. However, the difference in cholesterol level between the controls and those on the lipid lowering die was only about 2 per cent. This was clearly not a meaningful test of the lipid hypothesis.”

In other words, cholesterol is more important than outcome or at least a “diet designed to lower cholesterol levels, along with advice to stop smoking and advice on exercise” may still be a good thing.

Similarly, the Framingham study which found a strong association between cholesterol and heart disease found no effect of dietary fat, saturated fat or cholesterol on cardiovascular disease.  Again, a marker for risk is more important than whether you get sick.  “Scouts” who continued to look for superficial signs and ignore seemingly counter-intuitive conclusions from the computer still hold sway on the nutritional team.

“Grady had no way of knowing how much Billy disapproved of Grady’s most deeply ingrained attitude — that Billy had come to believe that baseball scouting was at roughly the same stage of development in the twenty-first century as professional medicine had been in the eighteenth.”

Professional medicine? Maybe not the best example.

What is going on here? Physicians, like all of us, are subject to many reinforcers but for humans power and control are usually predominant and, in medicine, that plays out most clearly in curing the patient. Defeating disease shines through even the most cynical analysis of physician’s motivations. And who doesn’t play baseball to win. “The game itself is a ruthless competition. Unless you’re very good, you don’t survive in it.”

Moneyball describes a “stark contrast between the field of play and the uneasy space just off it, where the executives in the Scouts make their livings.” For the latter, read the expert panels of the American Heat Association and the Dietary Guidelines committee, the Robert Eckels who don’t even want to study low carbohydrate diets (unless it can be done in their own laboratory with NIH money). In this

“space just off the field of play there really is no level of incompetence that won’t be tolerated. There are many reasons for this, but the big one is that baseball has structured itself less as a business and as a social club. The club includes not only the people who manage the team but also in a kind of women’s auxiliary many of the writers and commentators to follow and purport to explain. The club is selective, but the criteria for admission and retention and it is there many ways to embarrass the club, but being bad at your job isn’t one of them. The greatest offense a club member can commit is not ineptitude but disloyalty.”

The vast NIH-USDA-AHA social club does not tolerate dissent. And the media, WebMD, Heart.org and all the networks from ABCNews to Huffington Post will be there to support the club. The Huffington Post, who will be down on the President of the United States in a moment, will toe the mark when it comes to a low carbohydrate story.

The lessons from money ball are primarily in providing yet another precedent for human error, stubbornness and, possibly even stupidity, even in an area where the stakes are high. In other words, the nutrition mess is not in our imagination. The positive message is that there is, as they say in political science, validator satisfaction. Science must win out. The current threat is that the nutritional establishment is, as I describe it, slouching toward low-carb, doing small experiments, and easing into a position where they will say that they never were opposed to the therapeutic value of carbohydrate restriction. A threat because they will try to get their friends funded to repeat, poorly, studies that have already been done well. But that is another story, part of the strange story of Medicineball.

Intention-to-treat II: Foster, et al: Weight and Metabolic Outcomes After 2 Years

Posted: August 28, 2011 in Evidence Based Medicine, Intention to Treat, low-carbohydrate diet, The Nutrition Story
Tags: , , , , , ,
8

“Doctors prefer large studies that are bad to small studies that are good.”

— anon.

The paper by Foster and coworkers entitled Weight and Metabolic Outcomes After 2 Years on a Low-Carbohydrate Versus Low-Fat Diet, published in 2010, had a surprisingly limited impact, especially given the effect of their first paper in 2003 on a one-year study.  I have described the first low carbohydrate revolution as taking place around that time and, if Gary Taubes’s article in the New York Times Magazine was the analog of Thomas Paine’s Common Sense, Foster’s 2003 paper was the shot hear ’round the world.

The paper showed that the widely accepted idea that the Atkins diet, admittedly good for weight loss, was a risk for cardiovascular disease, was not true.  The 2003 Abstract said “The low-carbohydrate diet was associated with a greater improvement in some risk factors for coronary heart disease.” The publication generated an explosive popularity of the Atkins diet, ironic in that Foster had said publicly that he undertook the study in order to “once and for all,” get rid of the Atkins diet.  The 2010 paper by extending the study to 2 years would seem to be very newsworthy.  So what was wrong?  Why is the new paper more or less forgotten?  Two things.  First, the paper was highly biased and its methods were so obviously flawed — obvious even to the popular press — that it may have been a bit much even for the media. It remains to be seen whether it will really be cited but I will suggest here that it is a classic in misleading research and in the foolishness of intention-to-treat (ITT).

(more…)

Evidence-based Medicine: Who Decides Admissibility? I. The Frye Standard

Posted: May 6, 2011 in Evidence Based Medicine, The Nutrition Story
Tags: , , , , ,
6

According to the Journal of the American Medical Association (JAMA), the principle of “evidence-based medicine (EBM),” arose in the 1990s [1]. It is widely invoked in the medical literature as a kind of certification that the conclusions of the author are not mere opinions but are backed up by compelling information in biomedical science.  It sounds good. Or does it? It is certainly self-serving and a little bit suspicious, somewhat like Nixon assuring us that he was not a crook.  Evidence based medicine?  What were we doing before?  How was Pasteur able to function in the absence of such an idea?  One thing to think about is that evidence is what is introduced into courts of law.  But not all evidence is admissible. A judge decides what is admissible and there are many precedents, in particular, on what constitutes scientific evidence in a legal proceeding.

EBM relies on a hierarchy of levels of evidence (e.g. Table 1) with the random controlled trial (RCT) as the highest and expert opinion as the lowest.  Recommendations from health agencies and awarding of research grants are frequently justified on conformity to EBM or at least on their placing primary importance on RCTs.

Evidence from the USDA

Early in 2011, the USDA released its 2010 Dietary Guidelines for Americans [2].  With the dates suggesting the backward-looking nature of the Guidelines, they were nonetheless based on the Report of a prestigious committee (DGAC) [3] who, in turn, made much of their reliance on a new Nutrition Evidence Library (NEL). I and my colleagues were invited to submit a critique of the Report by the journal Nutrition. The editor, Michael Meguid indicated that the journal wanted a balanced report, pros and cons.  I called Dr. Meguid:

RDF: You know, the report is not particularly balanced. I’m not sure how you write a balanced review of an unbalanced report.

MM:  You can make the critique as strong as you like as long as you carefully document everything. But what’s your main problem with the Report?

RDF: Well, it makes very strong recommendations in the face of contradictory evidence.

MM: Make that the title of your article.

So we wrote an article called “In the face of contradictory evidence: Report of the Dietary Guidelines for Americans Committee” [4]. The journal was kind enough to make it an open access article and it’s available on this blog. In the end, on titles, we were one-upped by Steven Malanga, whose article in the New York Post was called “Fed’s Food Fog.”

For sure, both the Report and the final Guidelines were the proverbial camel-like production of a committee, tedious, repetitive and stylistic dreadful.  But what about the NEL?  What about the evidence?  Style aside, wasn’t this evidence based medicine?

Where do these guidelines come from? The assumption is that evidence follows its etymologic roots, stuff that is visible, stuff that comes from the sensible and true avouch of our own eyes. In fact, it is most often applied, as in the case of the DGAC, to the most controversial and contentious subjects. Calling something evidence is not enough. So what happens in courts of law? In a court of law, a judge decides on whether the jury can hear the evidence.  Who decides admissibility of the evidence in EBM?

Conflict resolution in science.

Science is a human activity. Conflict, controversy and a resistance to new ideas are well known even in the so-called hard, that is, more mathematical, sciences, and even where there are no outside forces as there was in the case of Galileo.  In the twentieth century, conflicts do not generally impede progress for long. Especially in the physical sciences, there is usually agreement on basic assumptions and on the rules of logic, allowing ultimate acceptance of strong evidence. Competing theories may coexist and supporters of both are likely to admit that they are awaiting reconciliation.

What happens when the spontaneous process of conflict resolution in science breaks down?  What happens in conditions where scientific disagreement is strong and a majority position becomes so dominant that it controls the funding and publication of scientific work and can ignore or repress contradictory evidence and repress exposition of alternative theories.  In essence, how do we deal with a recapitulation of the case of Galileo?

There is no system to decide on the admissibility in the cases considered by EBM.   I am not the first person to point out that EBM is largely the position of experts on one side of a scientific conflict [5], the lowest level of evidence on traditional EBM scales (e.g. “Level III: Opinions of respected authorities… of the US Preventive Services Task Force Systems,” Table 1).  EBM is sustained by those who want to use its particular criteria but these have never been subjected to outside affirmation.

In this situation, where science cannot police itself, we have to look for some outside guidance.  What do the courts do?  As one would expect there, is a long and extensive history of the legal system’s  attempt to deal with what constitutes scientific evidence.  On the chance that the legal perspective may help, I will discuss some of the issues.

Frye and the need for rules.

A key decision in the history of science in the courts is Frye v. United States.  In 1923, a Federal Appeals court ruled that the opinions of experts have to be supported by a scientific community. Frye had been convicted of second-degree murder but appealed on the grounds that he had successfully passed a lie-detector test.  At that time the device was a simple blood pressure machine and an expert witness testified as to the results. The court ruled that the lie-detector test “has not yet gained such standing and scientific recognition among physiological and psychological authorities as would justify the courts in admitting expert testimony‚” affirming the judgment of the lower court.

The ruling in Frye gave rise to the idea of “general acceptance,” and, by analogy, this appears to be the main principle in the admissibility of evidence in the nutrition world.  Sufficiently well established that it could be included in a biochemistry text is the idea that “consumption of saturated fats is positively associated with high levels of total plasma cholesterol and LDL cholesterol and an increased risk of CHD”[6] Known to students as “the Lippincott Book,” Harvey and Ferrier is the best selling biochemistry book in the world and it is correct when it states “Most experts strongly advise limiting intake of saturated fats.”

Most, but not all.  A small but not insignificant minority hold otherwise and whereas they agree that dietary saturated fat may raise blood cholesterol, they can provide overwhelming evidence that it is not associated with cardiovascular disease. This has been demonstrated in almost every large trial.

The problem is described in Marcia Angell’s Science on Trial [7].  Angell explains that Frye was not without its critics ([7], page 126).  Opponents, she wrote,

“claimed somewhat improbably, that it would tend to exclude novel, far-sighted testimony by modern-day Galileos. There is no record of this happening once, let alone often.  Furthermore, even if a modern-day Galileo did not make into court at first, that fact should not stop him from prevailing in the scientific community.  Courts do not determine scientific acceptance, as implied by the argument that we need to keep our courts open to the hidden Galileos in our midst.”

But isn’t this exactly what has happened in nutrition and maybe, in general, in the medical community?  The “experts” control editorial boards, granting agencies and academic departments and are as powerful as the Catholic Church in repressing dissent.  They have prestige and, in many cases, undisputed accomplishments, but does science run on general acceptance? Does majority (of experts) rule?

One of the problems with Frye that lawyers have addressed is a question of identifying the field of academic or scientific field in which the general acceptance is to be considered.  Different disciplines hold to different standards.  In the case at hand, many ideas in nutrition would be dismissed out of hand by biochemists. Many methodologies would be considered absurd by physical scientists: Intention-to-treat is perhaps the most absurd.  It has been pointed out that the question of who is an expert might have applied to the techniques in the original Frye case, at least as it might be implemented today: “If polygraph examiners are selected as the relevant field, polygraph results would be admissible.” (http://law.jrank.org/pages/2006/Scientific-Evidence-Frye-v-United-States.html).

The epidemic of obesity and diabetes stands as a testament to the failure of the experts.  A small library can be assembled of books attacking establishment medical nutrition. Uffe Ravnskov’s classic Cholesterol Myths is updated in Ignore the Awkward. Gary Taubes’s recent Good Calories, Bad Calories is the most compelling and James Le Fanu’s Rise and Fall of Modern Medicine, the most succinct but just sitting at my desk now I can see a dozen others on the book shelf.  Surprisingly, there has been only one rebuttal, Steinberg’s Cholesterol Wars, the subject of the next post .

Table 1.  Examples of Levels of Evidence from Various Sources. 

US Preventive Services Task Force Systems for ranking evidence about the effectiveness of treatments or screening:

Level I: Evidence obtained from at least one properly designed randomized controlled trial.

Level II-1: Evidence obtained from well-designed controlled trials without randomization.

Level II-2: Evidence obtained from well-designed cohort or case-control analytic studies, preferably from more than one center or research group.

Level II-3: Evidence obtained from multiple time series with or without the intervention. Dramatic results in uncontrolled trials might also be regarded as this type of evidence.

Level III: Opinions of respected authorities, based on clinical experience, descriptive studies, or reports of expert committees.

Bibliography

1. Torpy JM, Lynm C, Glass RM: JAMA patient page. Evidence-based medicine. JAMA 2009, 301(8):900.

2. Dietary Guidelines for Americans, 2010 [http://www.dietaryguidelines.gov.]

3. US Department of Agriculture and US Department of Health and Human Services: Report of the Dietary Guidelines Advisory Committee on the dietary guidelines for Americans, 2010. June 15, 2010. In.; 2010.

4. Hite AH, Feinman RD, Guzman GE, Satin M, Schoenfeld PA, Wood RJ: In the face of contradictory evidence: report of the Dietary Guidelines for Americans Committee. Nutrition 2010, 26(10):915-924.

5. Marantz P, Bird E, Alderman M: A Call for Higher Standards of Evidence for Dietary Guidelines. Am J Prev Med 2008, 34(3):234-239.

6. Harvey R, Ferrier D: Biochemistry, 5th edn. Baltimore and Philadelphia: Lippincott Williams & Wilkins; 2011.

7. Angell M: Science on Trial. New York: W. W. Norton & Co.; 1996.