Posts Tagged ‘nutritional studies’

Breaking news: Seidelmann, et al. did not collude with EAT-Lancet. Obstruction of Science still unsettled.

Posted: March 31, 2019 in Crisis in Nutrition
Tags: , , , ,
16

Ferdinand I. Charm, Piltdown, UK

The bizarre behavior of the medical nutritional establishment in recent years has caused critics to see an analogy with the Federal government. The so-called “invasion” of low-carb researchers across the borders of traditional medicine has caused a strong backlash which, in turn, has led to  accusations of a breakdown of style and even financial misdeeds. A key issue was a paper by Sarah Seidelmann, MD, a cardiologist with no apparent experience in nutrition, and her coworkers at Harvard School of Public Health and the associated Brigham and Women’s hospital.  The article claimed that the increasingly popular low-carbohydrate diets were actually life-threatening. Apparent communication between the authors and a Norwegian-based vegetarian pressure group called EAT-Lancet raised questions about foreign influence and led to investigation by the so-called Moola Committee. The committee was concerned that Dr. Walter Willett of Harvard, one of the authors on Seidelmann, et al., was also a co-chairman of a recent EAT-Lancet commission and had been present at various meetings with foreign nutritional powers. It was revealed today, however, that the Moola Investigation did not find evidence of collusion with Norway or other nations.

Willett is a widely respected nutritionist and co-developer of the Hu-Willett  theorem: for any diet and any disease state, there exists at least one finite set of confounders such that the diet can be shown to cause the disease. 

Seidelmann, et al. was generally considered to be a reflection of the long-standing attempts of the medical establishment to build a wall to keep low-carbohydrate diets from patients with diabetes and obesity. The authors denied such hostile intent and emphasized the fundamental scientific importance of their publication. “This work provides the most comprehensive study of carbohydrate intake that has been done to date,” was the description by the principle author, another cardiologist, Scott Solomon, MD. Writing in social media in a style that has come to be called Presidential, Solomon explained: “It’s a really terrific study. You’re gonna’ love it. Nobody’s ever done a better study. It’s terrific.” The Moola committee considered that this was not accurate and the study was really obstructing investigation into the real science. 

Scott Solomon, MD. Nobody has ever done a more comprehensive study.

The issue was not carbohydrate intake — none was measured. In fact, no low-carbohydrate diets were studied at all. The data were extracted from a different study about atherosclerosis. There was, however, a comprehensive questionnaire about what subjects had been eating at two time points many years apart. The results were quite surprising in that low-carbohydrate diets and high-carbohydrate diets were a risk. It seems only the 50 %. level characteristic of the US population diet during the obesity epidemic was beneficial. This paradoxical recommendation, not to change anything, in combination with the failure to cite major benefits of carbohydrate restriction in the literature, was further evidence of obstruction, however passive.

The website says that “EAT is a bite-sized organization with an outsized appetite for impact,” reflecting the sense of humor for which Norway is famous. Among the insights: “Packing leftovers into lunch boxes, using them in new creative recipes or keeping them for future consumption is good for the planet.” The EAT-Lancet commission explains that it is flexible in its recommendations “There is something for everyone across price ranges, cultures, age groups and individual preferences” although it is important “to consume no more than 98 grams of red meat (pork, beef or lamb), 203 grams of poultry and 196 grams of fish per week.”

The highlighted section is about 98 grams.

The Moola investigators were concerned that Willett’s presence on the EAT-Lancet governing board might be a conflict of interest. That authors are charged $ 5,000 to publish in Lancet Public Health also raised questions about financial impropriety. Because the study was funded by the NIH, it seemed that American tax payers were giving money to a pressure group whose roots appear to be in foreign countries.

Walter Willett, MD. Looks like Wyatt Earp but not a straight-shooter.

In the end, the Investigation knew some bucks were involved but couldn’t figure out what had happened at all but decided it wasn’t collusion. They couldn’t understand, though, how the Lancet, supposedly an independent medical journal could be tied to a political lobby group and worried that this might represent obstruction of science. This was the general reaction in the scientific community and wits on the internet suggested the journal change its name to Slants-it. 

Dr. Richard Feinman of SUNY Downstate Medical Center, author of the expose Nutrition in Crisis, was one of many critics who claimed that the Seidelmann paper could not have received peer review. He was told by Dr. Audrey Ceschia, editor of Lancet Public Health that he simply did not understand the contemporary concept of Editor-Based-Medicine. 

“These kinds of epidemiological studies with such low relative risk values are, of course, meaningless,” Feinman said. The relative risk for “low-carb” vs. control was so close to 50-50 that, as he further explained: “it means just what fifty per cent chance of rain, means. It means you know just what you knew before you turned on the radio.” Oddly, Feinman took a positive point of view of things: “overall the low-carb-deniers have done a good job. Science is a human activity and so there are no precise definitions, but a good start: ‘What you do in science is you make a hypothesis and then try to shoot yourself down.’ So the obsessive compulsive mission of the medical establishment to find something, anything, wrong with low-carb diets and their failure to come up with any risk, anything at all, means that low-carb diets are about the healthiest change you can make in your life-style.” Of course, there are always individual exceptions. Recently, after fifty years of scientific studies of low-carb diets without significant side-effects, a couple of unnamed, unidentified women were reported to have said — not a scientific study — that their low-carb diet gave them vaginal odor. The low-carb-deniers grabbed their story and it was blown all over the media including serious outlets like Forbes. (No women were identified and the story appears to have been invented by Edelman, a public relations firm although their client remains unknown).

The Moola Investigation is reportedly considering other potential obstruction of science cases. There is a consistent pattern, according to the investigators. to bring back “concerns” about health which have been widely discredited. Most recently, and most charming for its retro style, eggs and cholesterol, have come out of retirement following reports from yet another 50 % chance of rain study. Other likely attempts are to bring back the food pyramid which will now be used to store ever more whole grain. 

In a story in The Guardian, Seidelmann said “the team had published a substantial body of work ‘to thoroughly answer a question and not simply provide just one piece of the picture.’” Feinman admitted that it was the most substantial body of work on low-carbohydrate diets that did not study any low-carbohydrate diets.

All are eager to see the final Moola report.


Uses and misuses of group statistics. Bill Gates walks into a bar…

Posted: January 28, 2013 in Fructose, misuse of statistics
Tags: , , ,
20

Everybody has their favorite example of how averages don’t really tell you what you want to know or how they are inappropriate for some situations. Most of these are funny because they apply averages to cases where single events are important. I’ll list a couple in the text boxes in this post. From the title:

If Bill Gates walks into a bar, on average, everybody in the bar is a millionaire.

Technically speaking, averages start with the assumption that deviations are due to random error, that is, that there is a kind of “true” value if we could only control things well — if there were no wind resistance and all balls were absolutely uniform, they would always fall in the same place; any spread in values is random rather than systematic.
Standard_deviation_diagram.svg
(more…)

Flawed studies II. Occam’s Razor and How to Reduce Fructose Consumption.

Posted: July 27, 2012 in Crisis in Nutrition, Fructose, lipid metabolism, low-carbohydrate diet, Lustig, Occam's Razor, Reduce Fructose Consumption
Tags: , , ,
79

“I may have killed a dozen men but I never stole a horse.”

— last words of outlaw in the American West before being hanged.

The principle known as Occam’s Razor is usually understood as a statement that a simple explanation is preferable to one that is more complicated. The principle has many variations. It might be interpreted as saying that you have to have a sense of priorities. Occam’s Razor is not exactly a scientific idea so much as a principle of aesthetics expressing the value of elegance in scientific explanations. Named for William of Ockham (c. 1285–1349) — it is also referred to as Ockham’s Razor — the idea can be described mathematically by saying that if the outcome, Y, of an experiment can be expressed with a rough sort of equation: Y = A + B + C +… and if A explains Y, then you don’t want to drag in B, C, etc unless you absolutely have to. (A more compelling description might be to consider the principle in terms of a power series and if you are inclined to mathematics, Wikipedia has excellent description and animation).

Where we’re going. The bottom line on this post is that for obesity, diabetes and general health, the predominant effect of diet, the major contribution to the outcome — A in the equation above — is provided by substituting fat (any fat) for carbohydrate (any carbohydrate). That’s what the science says. That will give you the best effect. The B contribution (type of fat, type of carbohydrate) is strictly secondary. The practical consequence: if for some reason, you want to reduce fructose in the diet, the best advice is to reduce carbohydrate across the board. You can then add the additional advice “preferably sugar and high fructose corn syrup” but even if B doesn’t kick in, you will surely get a benefit. Most of all, if you take out Pepsi® and put in Pepperidge Farm® Whole Wheat Bread, you may not accomplish much.

In practical terms, confronted with a phenomenon that has many controlling variables, make sure you can’t do with one before you bring in the others. In nutrition, when people say that the phenomenon is very complicated, they frequently mean that they don’t want to look at a simple explanation. On its practical side, if a patients in a dietary experiment responds to the level of carbohydrate, you have to assume that carbohydrate across the board is the controlling variable. If, however, you think that it is specifically the fructose in the diet that caused the effect, or if you think that it was an additional effect of fructose beyond its role as carbohydrate, then that is something that you have to show separately. Until you do, the fructose effect is sliced off by Occam’s Razor. In terms of policy, you don’t want to go after fructose unless you are sure that it is not primarily the role of fructose acting as a carbohydrate.

So, there is a logical question surrounding recommendations against sugar and especially against fructose. What we know well in nutrition is that if you replace carbohydrate with fat, as in Krauss’s experiment described in the previous post, things improve and this is why we suggest low-carbohydrate diets as the “default diet,” the one to try first for diabetes and metabolic syndrome and probably for cardiovascular risk. I have, however, received at least two emails from well-known nutritionists saying that “the type of carbohydrate is more important than how much carbohydrate” and, of course, Rob Lustig is everywhere telling us how toxic sugar is but never suggesting that a low carbohydrate diet is any kind of ideal. On the face of it, the idea doesn’t make much sense. Fructose is a carbohydrate so the amount and type are not easily separable.

There are all kinds of strange things in nutrition. People actually say that the type of diet you are on is less important than whether you stay on the diet. While true, it is like saying that if you are baseball player, whether you get a hit depends less on who’s pitching than whether you remember to show up for the game. But anyway, I decided to ask the question about relative importance of type and amount of carbohydrate on facebook and on a couple of blogs where things like Hizzona’ Michael Bloomberg’s Big Bottle Ban or related questions was being discussed. Here’s how I put it.

For general health, should you change the type of carbohydrate or replace the carbohydrate with fat (any natural fat, no trans-fat)? It’s a thought experiment (not real world situation with subtleties). You only get three choices: For general health (no change in calories):

1. Change type of carbohydrate
2. Replace carbohydrate with fat
3. It doesn’t matter

Strangely enough, I did not get very many answers. I think that people didn’t like the question and even when they voted, they wanted to put in disclaimers:

ANS: 2. Replace carbohydrate with fat But I want to add; not replacing ALL the carbs. Only the worst ones. You know; Sugar, grains (bread and pasta) potatoes and rice.

RDF: You can do that in a real case but the question is about first-order strategies. You only get 3 choices.

ANS: okej 2. Replace carbohydrate with fat.

And James Krieger jumped in:

“Feinman, your ‘thought experiment’ is essentially a false trichotomy…same thing as a false dichotomy except you’ve arbitrarily limited it to 3 choices rather than 2, when in fact there are many more. This is why you aren’t getting answers…because you’re committing a common logical fallacy.”

I explained that

“It’s called Occam’s Razor…. I’m simply asking: if you could theoretically do only one thing, 1. or 2., which would be better? There are many other choices but in a thought experiment you imagine these to be held constant or to be the higher order terms in a power series.”

But, of course, Krieger was right. The question is not really answerable. Not because it is false so much as because it is confused. Fructose is a carbohydrate and whatever its unique contribution, it is hard to say it is more important than the contribution of the fructose as a carbohydrate. It is a screwy idea but, again, that’s how it was phrased to me in emails and probably in print someplace. Researchers in this field say: “it is not carbohydrate per se (or glycemic index/load) that is involved in adverse metabolic effects of dietary carbohydrates, but rather the type of carbohydrate,…” The kind of evidence that is used to support such an idea, the kind of result that is used to support fructophobia is in the paper by Stanhope, et al.

Stanhope, et al. measured the effects of chronic consumption of either glucose- or fructose-sweetened beverages providing 25% of energy requirements for 10 weeks in overweight and obese subjects. The figure below shows the superimposed outcomes in the response of triglycerides in the course of a day (red lines = fructose, blue = glucose). It is obvious that there is a difference — people consuming fructose had higher triglyceride responses (although fasting levels were not different). Looking at the figure, though, there is big variation in the data and it is not clear that everybody showed big differences between the glucose and fructose curves: the error bars represent standard error of the mean (SEM) which, while it shows you that there may be a statistically significant difference between the trials, doesn’t display very well the spread of the individual values, that is, whether a few individuals biased the grouped data. To convert to standard deviation, which gives you a better feel for the variation, you multiply, in this case, by about 4. In other words, there must have been big overlap between the fructose people and the glucose people.

So there is an effect of type of carbohydrate. But what to compare it to? The study of Krauss in the previous post showed much bigger changes when you substituted fat for carbohydrate and, in fact, those were fasting triglycerides which, in the fructose experiment, didn’t change at all but this is a different kind of experiment. So for comparison, we can look at a study from Jeff Volek’s lab where carbohydrate was replaced with fat in the carbohydrate restricted diet (CRD) in comparison to a low-fat diet (LFD). I described this study previously because it showed how carbohydrate, rather than dietary saturated fat, was actually controlling saturated fat in the blood. Here is what the responses to meals as seen in plasma triglycerides:

Maybe it’s the Fructose.

The fructose experiments can be shaved with Occam’s razor — insofar as we can tell, reducing carbohydrate across the board is more effective than changing type of carbohydrate. But how do we know that the effect of reducing carbohydrates wasn’t due to removing fructose — fructose is a carbohydrate so carbohydrate restriction may be due to the de facto removal of the fructose? Well, we don’t. It’s unlikely but possible. Where does this leave us? Wikipedia cites Bertrand Russell’s variation of Occam’s Razor: “Whenever possible, substitute constructions out of known entities for inferences to unknown entities.” This is a pompous way of saying: “don’t make things up.”

Another way of looking at Stanhope’s experiment is to recognize that it does not show, as the title says, “Consuming fructose-sweetened, not glucose-sweetened, beverages increases visceral adiposity and lipids… in overweight/obese humans.” What the paper really is about is “Consuming fructose-sweetened, not glucose-sweetened, beverages as part a high carbohydrate diet (55 % of energy) increases….” In other words, you don’t know whether you would get any benefit in changing from fructose to glucose if the total carbohydrate were lower.  In terms of our Occam’s Razor equation, you can’t say that you have proved that your results are due to A  (the major controlling variable (carbohydrate)) when all you have studied is A with the specific change in  the B term (secondary effect of the type of carbohydrate). Stanhope’s experiment shows: if you are on a high carbohydrate diet, replacing glucose with fructose will make things worse but that’s different than saying that fructose is toxic. From a practical point of view, if you are on a high carbohydrate diet and it is not giving you the health benefit you want, replacing sugar with starch may give you disappointing results compared to simply cutting down on carbohydrates.

How to Reduce Fructose Consumption.

If you want to encourage fructose reduction, encourage carbohydrate restriction (this is where we have the most information) with the additional proviso of recommending fructose reduction as the first carbohydrate to remove (may also help but we have less data).

Flawed Studies.

In combination with the previous post, a summary of things to look for in a study to make sure that the authors are not misleading you and/or themselves:

1. Understatement is good. “Healthy” is a value judgement. “Fructose-sweetened” is not the same thing as “fructose-sweetened in a high carbohydrate diet.”

2. Where are the pictures? The author has an obligation to make things clear. A graphic representation is usually an indication of a desire to explain.

3. Has Occam’s Razor been applied? Are secondary effects taken as primary?

Crabtree’s Bludgeon

Finally, we should not forget Crabtree’s Bludgeon which is described by Wikipedia as “a foil to Occam’s Razor” and “attributed to the fictitious poet, Joseph Crabtree, after whom the Crabtree Foundation is named.” It may be expressed as:

‘No set of mutually inconsistent observations can exist for which some human intellect cannot conceive a coherent explanation, however complicated.’

I Told George Bray How to do it Right.

Posted: January 19, 2012 in energy metabolism, low-carbohydrate diet, Protein, The Nutrition Story
Tags: , , ,
54

“Despite the claims of various diet gurus, excess calorie consumption alone and not the amount of protein in an individual’s diet contributes to the accumulation of unwanted fat….” That’s the tendentious and pretty much inaccurate first line of the press release from JAMA on George Bray’s over-feeding study “Effect of Dietary Protein Content on Weight Gain, Energy Expenditure, and Body Composition During Overeating.”  “Amount of protein?”  What’s going here?  It hasn’t really been about protein.  Most of us “diet gurus” have claimed that carbohydrate, not protein, in the diet was the key macronutrient in regulating metabolism, consistent with the basic biochemistry of the glucose-insulin axis, or as Dr. Bray described Gary Taubes’s position in a review of Good Calories, Bad Calories:

“The problem is the carbohydrates in the diet, their effect on insulin secretion, and thus the hormonal regulation of homeostasis – the entire harmonic ensemble of the human body.”

Reduction in dietary carbohydrate puts increased demands on protein for gluconeogenesis and other processes but the controlling variable is the carbohydrate. The controversy in nutrition has been largely about fat vs carbohydrate.  Should we be on a low-carbohydrate diet or a low-fat diet?

The quotation in the press release says accurately that “Earlier studies in human beings suggested that diets containing either high or low [levels of] protein are less ‘metabolically efficient’ than diets with normal protein levels.”  Accurate, but written as if metabolic efficiency had always been recognized for its importance in weight loss, as if there had not been a dispute over whether the costs of processing protein were important in energy balance, indeed, written as if Bray and coworkers had not maintained that only calories count in weight gain or loss.  The idea of metabolic advantage, that one diet could be more efficient — more weight gained/calorie — has been largely resisted by the nutritional establishment.  Is this slouching toward Metabolic Advantage? (“Who knows not [the Duke] is dead?  Who knows he is?”)

The debate is also about calories.  Should you cut calories or just cut out carbs?  Is it really “excess calorie consumption” and not the effect of excess carbohydrates ? “A calorie is a calorie” or not. Many of the gurus have gone beyond “claiming” to demonstrating that when carbohydrates are low, weight loss is greater than when carbohydrate is high and that the weight loss on a low-carbohydrate diet is primarily in fat stores rather than lean mass.  In head-to-head comparisons, for however long they are compared, low-carbohydrate diets generally out-perform low-fat diets on other parameters as well, glycemic control, the features of atherogenic dyslipidemia. This has been the major challenge to traditional nutrition and the general approach has been to simply ignore this data and dismiss the researchers with innuendo as above.

In some sense, Bray, et al. answered a question that we weren’t asking, but protein is important if more complicated than carbohydrate and fat. So what did the study find? Bray and coworkers compared three diets of 5 %, 15 % and 25 % protein at an excess of calories, that was nominally the same in each group. The study was a random controlled study and was carried out in a metabolic ward so the results are more accurate than the usual diet study that relies on dietary records.  There is something odd about this study, though, in that if you want to say that only calories are the independent variable, you can’t keep calories constant.  What was actually done was to determine the energy requirements for weight maintenance over a run-in period of 2-3 weeks on a maintenance and then an additional 40 % of calories was added.  So although the calories are constant relative to initial energy expenditure, they are not absolutely the same and this is a study of the effect of varying calories while keeping calories constant. The figure below, re-drawn from Figure 6 of the paper comparing intake of absolute energy to protein intake makes you stop and think.

When you have a small number of subjects, a single outlier can bias the results.  If you remove the single highest point (circled in red), the correlation is likely to get much weaker and the normals and low begin to separate.  In other words, the individual variation (the relative efficiency) is sufficient to make it hard to see the effect of variable energy or, perhaps, as the authors themselves set it up, it is energy normalized for baseline that is the key variable.  Then the authors are right (at least by inspection) that the protein intake does not effect the change in body fat but you have only a single value for the energy. In this case, you cannot say “calories alone account for the increase in fat” (Conclusion in Abstract) because you have only a single point.  If you keep constant the variable (carbohydrate) that is most likely to bring out differences, you shouldn’t be surprised in there are no big differences.

Even taking the conclusions at face value, the authors found, as other diet comparison studies have, that weight loss or, in this over-feeding study, weight gain, was not dependent on calories alone: “a calorie is a calorie” not.  It is likely that this was what the study was originally trying to disprove and the results must have been a disappointment.  The way out was that, in this particular case, the differential weight loss showed up in difference in lean mass, rather than in fat mass as has been found in other studies showing variable efficiency.  Since 5 % is very low protein it is probably not surprising that the diet could not provide enough protein for an increase in lean mass this group.

So what are the other diet studies that have found variable efficiency. The reduction in weight found in studies comparing low-carbohydrate diets and low-fat diets not only shows a difference favoring carbohydrate restriction but the improved weight loss is preferentially fat over lean mass. For example, Volek, et al. compared a low fat with a VLCK and the results are as shown below.  In their study, subjects were randomized to one of two hypocaloric diets, a very low-carbohydrate ketogenic (VLCK) diet (carbohydrate <10% of energy) or a low fat (LF) diet and after 8 weeks switched to the other diet. Reported energy was slightly higher during the VLCK but the VLCK group lost more weight and as shown below predominantly in fat, total fat loss, and trunk fat loss for men (despite significantly greater energy intake). The majority of women also responded more favorably to the VLCK diet, especially in terms of trunk fat loss the ratio of trunk fat/total fat was also significantly reduced during the VLCK diet in men and women.  These studies depend on diet recall so are less accurate than the JAMA study but because of the better experimental design, the changes are bigger and with appropriate correction make a less ambiguous case than the JAMA study. The more accurate measurements in the metabolic chamber suggest that individual variation is real and not just due to random error.

So what do we know from Bray, et al.? As described above, there is some ambiguity in what constant energy means. Still, nobody questions that under many conditions, a “calorie is a calorie,” but they actually found that weight gain was different so when metabolic advantage is “claimed” it cannot be dismissed out of hand.  This is different than widely cited studies in the literature that claim macronutrients do not effect weight loss, since if weight gain depends on macronutrient, it is reasonable that weight loss does too.  Similarly, if tissue distribution affects lean mass in this case, then studies where the tissue distribution shows preferential loss of fat can’t be dismissed — again, it is certainly not surprising that a low protein diet will lead to less storage of protein; generally, while it is just as bad a generalization as “a calorie is a calorie,” there is some truth in “you are what you eat.” Also, in the JAMA study, protein was exchanged for fat so a reduction in fat did not have an effect on fat which may or may not be a surprise to many people. Tom Naughton raised a few other questions about Bray, et al. but in the end, the paper reminds me of the joke about the Polish Mafia: they make you an offer you can’t understand.

How to do it.

But  I told George how to do it. A couple of years ago, he and I had a brief correspondence. I made the following proposal. I suggested we could apply for a joint grant and publication to get the answer.  The following is from my email to him in 2008  (I have added some highlights):

 “A modest proposal

 Proponents of carbohydrate-restricted diets (CRD) and critics of such diets cooperate to design a long-term comparison of CRD and low-fat diets.  The groups agree on methods of procedure, make-up of the diets, how compliance will be effected, and what parameters will be measured.

We write the paper first, leaving room for the data, that is, we agree in advance on what the possible outcomes are and what conclusions could be drawn from them.  The final MS can only be edited for language usage. There are no disclaimers, no Monday-morning-quarterbacking, no excuses.

The paper could be submitted while the grant application is being written and would have to be accepted because any objections could be incorporated in the plan.  The grant itself would surely be funded since it incorporates everybody’s specific aims.”

 George hasn’t answered and he obviously has a different approach to the problem but my offer still stands.

In the end, that is what it will take to solve the problem.  Unless we agree on what the question is, how it can be tested and work together to do the experiment, the lipophobes will ignore the low-carbohydrate studies and we will criticize their studies. The real losers, of course, will be the people suffering from obesity and diabetes.  The question everybody always asks me, is why can’t there be a meeting of the minds?  In the current case, why was the JAMA study done?

Why was this study done? 

 “Dr Bray discussed the results with news@JAMA via e-mail.

news@JAMA: What are the practical implications of these findings for patients trying to lose weight or for the physicians trying to counsel them?

 Dr Bray: The first practical implication is an old one: calories count. We showed very clearly that the increase in body fat was due to the increased intake of calories and that the amount of protein in the diet did not change it.

 To avoid that slow weight gain that many adults experience in their middle years, people need to watch their weight and increase activity, decrease food intake, or both; changing the diet alone will not do it.”

This sounds like the the same recommendations we’ve had for years.  Writing this, I suddenly realized that, as they say in German: that’s where the dog is buried.  It is about recommendations.  This research is following the recommendations.  It used to be (should be? assume it must be?) that recommendations follow from the research. Now, it’s the other way around.  Committees make recommendations and then research (sometimes by members of the committee) tries to support the recommendations. Something about this bothers me.

Evidence Based Medicine and Admissibility. IV. The Oslo Diet-Heart Study.

Posted: June 1, 2011 in Evidence Based Medicine, Lipophobes, The Nutrition Story
Tags: , , , , ,
15

“In the Viking era, they were already using skis…and over the centuries, the Norwegians have proved themselves good at little else.”

–John Cleese, Norway, Home of Giants.

With the 3-foot bookshelf of popular attacks on the low-fat-diet-heart idea it is pretty remarkable that there is only one defense.  Daniel Steinberg’s Cholesterol Wars. The Skeptics vs. The Preponderance of Evidence is probably more accurately called a witness for the prosecution since low-fat, in some way or other is still the law of the land.

The Skeptics vs. the Preponderance of Evidence

The Skeptics vs. the Preponderance of Evidence

The book is very informative, if biased, and it provides an historical perspective describing the difficulty of establishing the cholesterol hypothesis. Oddly, though,  it still appears to be very defensive for a witness for the prosecution.  In any case, Steinberg introduces into evidence the Oslo Diet-Heart Study [2] with a serious complaint:

“Here was a carefully conducted study reported in 1966 with a statistically significant reduction in reinfarction [recurrence of heart attack] rate.  Why did it not receive the attention it deserved?”

“The key element,” he says, “was a sharp reduction in saturated fat and cholesterol intake and an increase in polyunsaturated fat intake. In fact. each experimental subject had to consume a pint of soybean oil every week, adding it to salad dressing or using it in cooking or, if necessary, just gulping it down!”

Whatever it deserved, the Oslo Diet-Heart Study did receive a good deal of attention.  The Women’s Health Initiative (WHI), liked it.  The WHI was the most expensive failure to date. It found that “over a mean of 8.1 years, a dietary intervention that reduced total fat intake and increased intakes of vegetables, fruits, and grains did not significantly reduce the risk of CHD, stroke, or CVD in postmenopausal women.” [3]

The WHI, adopted a “win a few, lose a few” attitude, comparing its results to the literature, where some studies showed an effect of reducing dietary fat and some did not — this made me wonder: if the case is so clear, whey are there any failures.  Anyway, it cited the Oslo Diet-Heart Study as one of the winners and attributed the outcome to the substantial lowering of plasma cholesterol.

So, “cross-examination” would tell us why, if  “a statistically significant reduction in reinfarction  rate”  it did “not receive the attention it deserved?”

First, the effect of diet on cholesterol over five years:

The results look good although, since all the numbers are considered fairly high, and since the range of values is not shown, it is hard to tell just how impressive the results really are. But we will stipulate that you can lower cholesterol on a low-fat diet. But what about the payoff? What about the outcomes?

The results are shown in Table 5 of the original paper:   Steinberg described how in the first 5 years: “58 patients of the 206 in the control group (28%) had a second heart attack” (first 3 lines under first line of blue-highlighting) but only

“…  32 of the 206 in the diet (16%)…”  which does sound pretty good.

In the end, though, it’s really the total deaths from cardiac disease.  The second blue-highlighted line in Table 5 shows the two final outcome.  How should we compare these.

1. The odds ratio or relative risk is just the ratio of the two outcomes (since there are the same number of subjects) = CHD mortality (diet)/ CHD mortality control) = 94/79 =  1.19.  This seems strikingly close to 1.0, that is, flip of a coin.  These days the media, or the report itself, would report that there was a 19 % reduction in total CHD mortality.

2, If you look at the absolute values, however, the  mortality in the controls is 94/206 = 45.6 % but the diet group had reduced this  to 79/206 = 38.3 % so the change in absolute risk is  45.6 % – 38.3 % or only 7.3 % which is less impressive but still not too bad.

3. So for every 206 people, we save 94-79 = 15 lives, or dividing 206/15 = 14 people needed to treat to save one life. (Usually abbreviated NNT). That doesn’t sound too bad.  Not penicillin but could be beneficial. I think…

Smoke and mirrors.

It’s what comes next that is so distressing.  Table 10 pools the two groups, the diet and the control group and now compares  the effect of smoking: on the whole population,  the ratio of CHD deaths in smokers vs non-smokers is 119/54 = 2.2 (magenta highlight) which is somewhat more impressive than the 1.19 effect we just saw.  Now,

1. The absolute difference in risk is (119-54)/206 = 31.6 % which sounds like a meaningful number.

2. The number needed to treat is 206/64 = 3.17  or only about 3 people need to quit smoking to see one less death

In fact, in some sense, the Oslo Diet-Heart Study provides smoking-CHD risk as an example of a meaningful association that one can take seriously. If only such a significant change had actually been found for the diet effect.

So what do the authors make of this? Their conclusion is that “When combining data from both groups, a three-fold greater CHD mortality rate is demonstrable among the hypercholesterolemic, hypertensive smokers than among those in whom these factors were low or absent.”  Clever but sneaky. The “hypercholesterolemic, hypertensive” part is irrelevant since you combined the groups. In other words, what started out as a diet study has become a “lifestyle study.”  They might has well have said “When combining data from fish and birds a significant number of wings were evident.” Members of the jury are shaking their heads.

Logistic regression. What is it? Can it help?

So they have mixed up smoking and diet. Isn’t there a way to tell which was more important?  Well, of course, there are several ways.  By coincidence, while I was writing this post, April Smith posted on facebook, the following challenge “The first person to explain logistic regression to me wins admission to SUNY Downstate Medical School!” I won although I am already at Downstate.  Logistic regression is, in fact, a statistical method that asks what the relative contribution of different inputs would have to be to fit the outcome and this could have been done but in this case, I would use my favorite statistical method, the Eyeball Test.  Looking at the data in Tables 5 and 10 for CHD deaths, you can see immediately what’s going on. Smoking is a bigger risk than diet.

If you really want a number, we calculated relative risk above. Again, we found for mortality, CHD (diet)/ CHD (control) = 94/79 =  1.19. But what happens if you took up smoking: Figure 10 shows that your chance of dying of heart disease would be increased by 119/54 = 2.2  or more than twice the risk.  Bottom line: you decided to add saturated fat to your diet, your risk would be 1.19 what it was before which might be a chance you could take faced with authentic Foie Gras.

Daniel Steinberg’s question:

“Here was a carefully conducted study reported in 1966 with a statistically significant reduction in reinfarction  rate.  Why did it not receive the attention it deserved?”

Well, it did. This is not the first critique.  Uffe Ravnskov described how the confusion of smoking and diet led to a new Oslo Trial which reductions in both were specifically recommended and, again, outcomes made diet look bad [4].  Ravnskov gave it the attention it deserved. But what about researchers writing in the scientific literature. Why do they not give the study the attention it deserves. Why do they not point out its status as a classic case of a saturated fat risk study with no null hypothesis.  It certainly deserves attention for its devious style. Of course, putting that in print would guarantee that your grant is never funded and your papers will be hard to publish.  So, why do researchers not give the Oslo-Diet-Heart study the attention it deserves?  Good question, Dan.

Bibliography

1. Steinberg D: The cholesterol wars : the skeptics vs. the preponderance of evidence, 1st edn. San Diego, Calif.: Academic Press; 2007.

2. Leren P: The Oslo diet-heart study. Eleven-year report. Circulation 1970, 42(5):935-942.

3. Howard BV, Van Horn L, Hsia J, Manson JE, Stefanick ML, Wassertheil-Smoller S, Kuller LH, LaCroix AZ, Langer RD, Lasser NL et al: Low-fat dietary pattern and risk of cardiovascular disease: the Women’s Health Initiative Randomized Controlled Dietary Modification Trial. JAMA 2006, 295(6):655-666.

4. Ravnskov U: The Cholesterol Myths: Exposing the Fallacy that Cholesterol and Saturated Fat Cause Heart Disease. Washington, DC: NewTrends Publishing, Inc.; 2000.

The Seventh Egg.

Posted: October 22, 2010 in Uncategorized
Tags: , , , , , ,
9

Stepping back and looking at the recent scientific literature, I am struck with how life is a miracle.  How could humans have evolved in the face of threats from red meat, from eggs, even from the dangers of shaving?  (If you write about nutrition you have to create a macro that types out “I’m not making this up:” the Caerphilly Study [1] shows you the dangers of shaving… or is it the dangers of not shaving?).  With 28% greater risk of diabetes here, 57 % greater risk of heart disease there how could our ancestors have ever come of child-bearing age?  With daily revelations from the Harvard School of Public Health showing the Scylla of saturated fat and the Carybdis of sugar between which our forefathers sailed, it is amazing that we are here.

These studies that the media writes about, are they real?  They are certainly based on scientific papers.  If the media is not always able to decipher them, reporters do generally talk to the researchers. And the papers must have gone through peer review and yet many actually defy common sense.   Can the medical literature have such a high degree of error?  Could there be a significant number of medical researchers who are not doing credible science?  How can the consumer decide?  I am going to try to answer these questions.  When people ask questions like “could the literature be wrong?,” the answer is usually “yes” and I will try to explain what’s wrong and how to read the nutritional literature in a practical way. I am going to try to make it simple.  It is science, but it is pretty simple science.  I am going to illustrate the problem with the example of a paper by Djoussé [2].  But first, a joke.

It was a dumb joke. In my childhood, there was the idea, probably politically incorrect, that Indians, that is, Native Americans, always said “how” as a greeting.  The joke was about an Indian with a great memory who is asked what he had for breakfast on New Years day the previous year.  He says “eggs.”  They are then interrupted by an earthquake or some natural disaster and the interviewer and the Indian don’t meet again for ten years.  When they meet, the interviewer says “how.”  The Indian answers “scrambled.”

If the interviewer had been an epidemiologist he might have asked if he had developed diabetes.   Djoussé, et al. [2] asked participants about how many eggs they ate and then ten years later, if they developed diabetes it was assumed to be because of the eggs.  Is this for real?  Do eggs cause changes in your body that accumulate until you develop a disease, a disease that is, after all, primarily one of carbohydrate intolerance?  The condition is due either to the inability of the pancreas to produce insulin in response to carbohydrate (type 1) or to impaired response of the body to the insulin produced and a deterioration of the insulin-producing cells of the pancreas (type 2).  Common sense says that there is something suspicious about the idea that eggs would play a major role.  It is worth trying to understand the methodology and see if there is a something beyond common sense, and whether this is a problem in other studies besides   Djoussé’s.

What did the experimenters actually do.  First, people were specifically asked “to report how often, on average, they had eaten one egg during the past year,” and “classified each subject into one the following categories of egg consumption: 0, < 1 per week, 1 per week, 2-4 per week, 5-6 per week, and 7+ eggs per week.”  They collected this data every two years for ten years.  With this baseline data in hand they then followed subjects “from baseline until the first occurrence of a) type 2 diabetes, b) death, or c) censoring date, the date of receipt of the last follow-up questionnaire” which for men was up to 20 years.  Thinking back over a year: is there any likelihood that you might not be able to remember whether you had 1 vs. 2 eggs on average during the year?  Is there any possibility that some of the men who were diagnosed with diabetes ten years after their report on eggs changed their eating pattern in the course of ten years?  Are you eating the same food you ate ten years ago?  Quick, how many eggs/week did you eat last year?

Reading a scientific paper: the Golden Rule.

Right off, there is a problem in people reporting what they ate but this is a limitation of many nutritional studies and, while a source of error, it is depends on how you interpret the data.  All scientific measurements have error.  It is not a matter of ignoring the data but rather not interpreting results beyond measurement.  So, here’s how I read a scientific paper.   First, I look for the pictures.  What? A professor of biochemistry looks for the pictures first?  In a scientific paper, of course, they are called figures but it’s not just saving a thousand words.  (I get a thousand emails every couple of weeks). It’s about presentation of the data.

The principle is that a scientific paper is supposed to explain. The principle is laid out in what I call the golden rule of scientific papers.  It comes from the Book PDQ Statistics by Geoffrery Norman and David Streiner.  PDQ stands for Pretty Darned Quick and some of the humor is pretty sophomoric (e.g. it has Convoluted Reasoning or Anti-intellectual Pomposity detectors) but it is an excellent introductory statistics book.  Here’s the Golden Rule:

“The important point…is that the onus is on the author to convey to the reader an accurate impression of what the data look like, using graphs or standard measures, before beginning the statistical shenanigans.  Any paper that doesn’t do this should be viewed from the outset with considerable suspicion.”

— Norman & Streiner, PDQ Statistics [3]

In other words: teach.  Make it clear.  Eye-balling Djoussé, et al., we see that there are no figures.     A graph of number of eggs consumed vs number of cases of diabetes is what would be expected of the golden rule.  The results, instead are stated in the Abstract of the paper as the following mind-numbing statistics. (You don’t really have to read this)::

“Compared with no egg consumption, multivariable adjusted hazard ratios (95% CI) for type 2 diabetes were 1.09 (0.87-1.37), 1.09 (0.88-1.34), 1.18 (0.95-1.45), 1.46 (1.14-1.86), and 1.58 (1.25-2.01) for consumption of <1, 1, 2-4, 5-6, and 7+ eggs/week, respectively, in men (p for trend <0.0001). Corresponding multivariable hazard ratios (95% CI) for women were 1.06 (0.92-1.22), 0.97 (0.83-1.12), 1.19 (1.03-1.38), 1.18 (0.88-1.58), and 1.77 (1.28-2.43), respectively (p for trend <0.0001).”

What does all this mean? I will just state what the statistics mean because it is worth considering the conclusion as stated by the authors.

The meaning of the statistics is that there was no risk of consuming 1 egg/week compared to eating none.  Similarly, there was no risk in eating 2-4 eggs/week or 5-6 eggs/week.  But when you up your intake to 7 eggs or more per week, that’s it.  Now, you are at risk for diabetes.  The relative risk is small but there it is. You are now at greater risk.

Since I like pictures, I will try to illustrate this with a modified still from the movie, The Seventh Seal directed by Ingemar Bergman.  Very popular in the fifties and sixties, these movies had a captivating if pretentious style: they sometimes seemed to be designed for Woody Allen’s parodies.  One of the famous scenes in The Seventh Seal is the protagonist’s chess game with Death.  A little PhotoShop and we have a good feel for what happens if you go beyond 5-6 eggs/week.

 

1. Ebrahim S, Smith GD, May M, Yarnell J: Shaving, coronary heart disease, and stroke: the Caerphilly Study. Am J Epidemiol 2003, 157(3):234-238.

2. Djoussé L, Gaziano JM, Buring JE, Lee IM: Egg consumption and risk of type 2 diabetes in men and women. Diabetes Care 2009, 32(2):295-300.

3. Norman GR, Streiner DL: PDQ statistics. 3rd edition. Hamilton, Ont.: B.C. Decker; 2003.