Reading the Scientific Literature. A Guide to Flawed Studies.

Posted: July 11, 2012 in Association and Causality, The Nutrition Story, triglycerides
Tags: , , ,

Doctor:    Therein the patient

  Must minister to himself.

Macbeth: Throw physic [medicine] to the dogs; I’ll none of it.

— William Shakespeare, Macbeth

The quality of nutrition papers even in the major scientific and medical journals is so variable and the lack of restraint in the popular media is so great that it is hard to see how the general public or even scientists can find out anything at all. Editors and reviewers are the traditional gate-keepers in science but in an area where rigid dogma has reached Galilean proportions, it is questionable that any meaningful judgement was made: it is easy to publish papers that conform to the party line (“Because of the deleterious effects of dietary fructose, we hypothesized that…”) and hard to publish others: when JAMA published George Bray’s “calorie-is-a-calorie” paper and I pointed out that the study more accurately supported the importance of carbohydrate as a controlling variable, the editor declined to publish my letter.  In this, the blogs have performed a valuable service in providing an alternative POV but if the unreliability is a problem in the scientific literature, that problem is multiplied in internet sources. In the end, the consumer may feel that they are pretty much out there on their own. I will try to help.  The following was posted on FatHead’s Facebook page:

 How does one know if a study is ‘flawed’? I see a lot of posts on here that say a lot of major studies are flawed. How? Why? What’s the difference if I am gullible and believe all the flawed studies, or if I (am hopefully not a sucker) believe what the Fat Heads are saying and not to believe the flawed studies — eat bacon.

Where are the true studies that are NOT flawed…. and how do I differentiate? : /

 My comment was that it was a great question and that it would be in the next post so I will try to give some of the principles that reviewers should adhere to.  Here’s a couple of guides to get started. More in future posts:

 1“Healthy” (or “healthful”) is not a scientific term. If a study describes a diet as “healthy,” it is almost guaranteed to be a flawed study.  If we knew which diets were “healthy,” we wouldn’t have an obesity epidemic. A good example is the paper by Appel, et al. (2005). “Effects of protein, monounsaturated fat, and carbohydrate intake on blood pressure and serum lipids: results of the OmniHeart randomized trial,” whose conclusion is:

“In the setting of a healthful diet, partial substitution of carbohydrate with either protein or monounsaturated fat can further lower blood pressure, improve lipid levels, and reduce estimated cardiovascular risk.”

 It’s hard to know how healthful the original diet, a “carbohydrate-rich diet used in the DASH trials … currently advocated in several scientific reports” really is if removing carbohydrate improved everything.

Generally, understatement  is good.  One of the more famous is from Watson & Cricks’s 1953 paper in which they proposed the DNA double helix structure. They said “It has not escaped our notice that the specific pairing we have postulated immediately suggests a possible copying mechanism for the genetic material.”  A study with the word “healthy” is an infomercial.

2. Look for the pictures (figures).  Presentation in graphic form usually means the author wants to explain things to you, rather than snow you.  This is part of the Golden Rule of Statistics that I mentioned in my blogpost “The Seventh Egg”  which discusses a very flawed study from Harvard on egg consumption. The rule comes from the book PDQ Statistics:

“The important point…is that the onus is on the author to convey to the reader an accurate impression of what the data look like, using graphs or standard measures, before beginning the statistical shenanigans.  Any paper that doesn’t do this should be viewed from the outset with considerable suspicion.”

The Watson-Crick  paper cited above had the diagram of the double-helix  which essentially became the symbol of modern biology.  It was drawn by Odile, Francis’s wife, who is described as being famous for her nudes, only one of which I could find on the internet.

Krauss, et. al. Separate effects of reduced carbohydrate intake and weight loss on atherogenic dyslipidemia.

The absence of a figure may indicate that the authors are not giving you a chance to actually see the results, that is, the experiment may not be flawed but the interpretation may be misleading, intentionally or otherwise.  An important illustration of the principle is a paper published by Krauss. It is worth looking at this paper in detail because the experimental work is very good and the paper directly — or almost directly — confronts a big question in diet studies: when you reduce calories by cutting out carbohydrate, is the effect due simply  to lowering calories or is there a specific effect of carbohydrate restriction.  The problem is important since many studies compare low-carbohydrate and low-fat diets where calories are reduced on both. Because the low-carbohydrate diet generally has the better weight loss and better improvement in HDL and triglycerides, it is said that it was the weight loss that caused the lipid improvements.

So Krauss compared the effects of carbohydrate restriction and weight loss on the collection of lipid markers known collectively as atherogenic dyslipidemia.  The markers of atherogenic dyslipidemia, which are assumed to predispose to cardiovascular disease, include high triglycerides (triacylglycerol), low HDL and high concentrations of the small dense LDL.

Here is how the experiment was set up: subjects first consumed a baseline diet of  54% of energy as carbohydrate, for 1 week. They were then assigned to one of four groups.  Either they continued the baseline diet, or they kept calories constant but reduced carbohydrate by putting fat in its place.  The three lower carbohydrate diets had 39 % or 26 % carbohydrate or 26 % carbohydrate with higher saturated fat.  After 3 weeks on constant calories but reduced carbohydrate, calories were decreased by 1000 kcal/d for 5 week and, finally, energy was stabilized for 4 weeks and the features of atherogenic dyslidemia were measured at week 13.  The protocol is shown in the figure from Krauss’s paper:

The Abstract of the paper describes the outcomes and the authors’ conclusions.

Results: The 26%-carbohydrate, low-saturated-fat diet reduced [atherogenic dylipidemia]. These changes were significantly different from those with the 54%-carbohydrate diet. After subsequent weight loss, the changes in all these variables were significantly greater…(my italics)

 Conclusions: Moderate carbohydrate restriction and weight loss provide equivalent but non-additive approaches to improving atherogenic dyslipidemia. Moreover, beneficial lipid changes resulting from a reduced carbohydrate intake were not significant after weight loss.

Now there is something odd about this.  It is the last line of the conclusion that is really weird. If you lose weight, the effect of carbohydrate is not significant?  As described below, Jeff Volek and I re-analyzed this paper so I have read that line a dozen times and I have no idea what it means.  In fact, the whole abstract is strange.  It will turn out that the lower (26 %) is certainly “significantly different from.. the 54%-carbohydrate diet” but it is not just different but much better. Why would you not say that?  The Abstract is generally written so that it sounds negative about low carbohydrate effects but it is already known from Krauss’s previous work and others that carbohydrate restriction has a beneficial effect on lipids and the improvements in lipid markers occur on low-carbohydrate diets whether weight is lost or not.  The last sentence doesn’t seem to make any sense at all.    For one thing, the experiment wasn’t done that way.  As set up, weight loss came after carbohydrate restriction.  So, let’s look at the data in the paper.  There are few figures in the paper and Table 2 in the paper presents the results in a totally mind-numbing layout.  Confronted with data like this, I sometimes stop reading.  After all, if the author doesn’t want to conform to the Golden Rule of Statistics, if they don’t want to really explain what they accomplished, how much impact is the paper going to have.  In this case, however, it is clear that the experiment was designed correctly and it just seems impossible from previous work that this wouldn’t support the benefits of carbohydrate restriction and the negative tone of the Abstract needs to be explained.  So we all had to slog through those tables.  Let’s just look at the triglycerides since this is one of the more telling attributes of atherogenic dyslpidemia.  Here’s the section from the Table:

Well this looks odd in that the biggest change is in the lowest carb group with high SF but it’s hard to tell what the data look like.  First it is reported as logarithms. You sometime take logs of your data in order to do a statistical determination but that doesn’t change the data and it is better to report the actual value.  In any case, it’s easy enough to take antilogs and we can plot the data.  This is what it looks like:

It’s not hard to see what the data really show: Reducing carbohydrate has an overwhelming effect on triglycerides even without weight loss.  When weight loss is introduced, the high carbohydrate diets still can’t equal the performance of the carbohydrate reduction phase.  (The dotted line in the figure are data from Volek’s earlier work which Krauss forgot to cite).

The statements in the Conclusion from the Abstract are false and totally misrepresent the data.  It is not true as it says “carbohydrate restriction and weight loss provide equivalent…” effects. The carbohydrate-reduction phase is dramatically better than the calorie restriction phase and it is not true that they are “non-additive”  Is this an oversight?  Poor writing?  Well, nobody knows what Krauss’s motivations were but Volek and I plotted all of the data from Krauss’s paper and we published a paper in Nutrition & Metabolism providing an interpretation of Krauss’s work (with pictures).  Our conclusion:

Summary Although some effort is required to disentangle the data and interpretation, the recent publication from Krauss et al. should be recognized as a breakthrough. Their findings… make it clear that the salutary effects of CR on dyslipidemia do not require weight loss, a benefit that is not a feature of strategies based on fat reduction. As such, Krauss et al.  provides one of the strongest arguments to date for CR as a fundamental approach to diet, especially for treating atherogenic dyslipidemia.

An important question in this experiment, however, is whether even in the calorie reduction phase, calories are actually the important variable.  This is a general problem in calorie restriction studies if for no other reason than that there is no identified calorie receptor.  When we published this data, Mike Eades pointed out that in the phase in which Krauss reduced calories, it was done by reducing macronutrients across the board so carbohydrate was also reduced and that might be the actual controlling variable so we plotted the TAG against carbohydrate in each phase (low, medium and high carb (LC, MC, HC) without or with weight loss (+WL) and the results are shown below

This is remarkably good agreement for a nutrition study. When you consider carbohydrates as the independent variable, you can see what’s going on.  Or can you?  After all, by changing the variables you have only made an association between carbohydrate and outcome  of an experiment. So does this imply a causal relation between carbohydrate and triglycerides or not?  It is widely said that observational studies do not imply causality, that observational studies can only provide hypothesis for future testing. It certainly seems like causality is implied here.  It will turn out that a more accurate description is that observational studies do not necessarily imply causality and I will discuss that in the next posts.  The bottom line will be that there is flaw in grand principles like “Random controlled trials are the gold standard.” “Observational studies are only good for generating hypotheses,”  “Metabolic Ward Studies are the gold standard.” Science doesn’t run on such arbitrary rules.

  1. Good article Richard. Nice to know some people are trying to make science better. I would be interested in more info on how to best plan and control studies (especially diet studies) to ensure the results are statistically significant.

    • rdfeinman says:

      I think we need to recognize statistics as our servants not our masters. That will be the first step. A major flaw in published papers is mindless use of statistics without the intervention of common sense.

    • Richard,

      You have a wonderful talent for reading the literature. I would have given up. The causality vs correlation issue in the literature is rampant, let alone what the public receives. These people must believe that roosters make the sun come up since it happens with 100% reliability. Scientists need to ask what experiment will disprove their hypothesis and then do it. People seem to have lost sight of the fact that you can’t prove a hypothesis. If you test comes out as predicted you learn nothing, although it does change your confidence. But you only learn when the hypothesis fails.

      • rdfeinman says:

        No experiment can disprove the diet-heart hypothesis. When the experiment fails, it is because there were insufficient numbers of subject to provide the appropriate power (translation: we are applying for an even more expensive grant).

  2. FrankG says:

    Excellent! I can hardly wait for the next instalment(s). I particularity like the understated quote from Watson & Crick — brilliant! For me an healthy dose of humility is often a good sign of a credible scientist.

    • rdfeinman says:

      People thought that they were being coy but Watson said someplace that it was that they weren’t sure that they had it exactly right. The most important sign of a credible scientist is that you want to get it right.

      • Dr Feinman thank you for posting such an informative article on the matter. I am sure you are aware of the classic paper by Ioannidis that most research findings are false. This comes to confirm once again that sometimes conclusions are drawn well before the findings.

      • rdfeinman says:

        Yes. In nutrition we have recommendation-driven research rather than the other way around. Recommendations are made and then people do the research to support them. Peer review is provided by the people who made the recommendations in the first place.

  3. “…it is hard to see how the general public or even scientists can find out anything at all..”

    I never realized how hard it was to reach the public with the truth until recently. I’ve been involved with a company that is trying to make a commercial for a new sweetener. The script had to be rewritten multiple times to be sure it would satisfy the Federal Trade Commission. It seems you cannot say anything about sugar except that it has calories and carbs and can cause weight gain. And you cannot say that you can lose weight without exercize, because that is impossible.

  4. Alfonzo Luz says:

    Dear Dr. Feinman,

    Alright, let’s stipulate that every word you wrote is true without qualification. Agreed? Agreed.

    Can you show us a study that meets your requirements and proves that Homo sapiens scorbuticus needs dietary sugars? I am not referring to the essential sugars, just the rest of them.

    I am very impressed with your post; it gives me something to aspire to. If anyone can answer the above question, surely you can.

    I have just been reviewing an interesting site put up by Elizabeth Whelan of New York for the American Council on Science and Health
    She makes a similar case for careful scrutiny of innovations. I assume that includes HFCS.

    Best wishes and keep ’em coming!

    P.S. Have you seen the rat study that seems to show that rats fed fructose in lieu of sucrose lose track of where they are in the maze? I don’t recall whether they got fat. It’s somewhere here in my in-box. Rats might not be the best human surrogate but both McCarrison and Mayer used them. AL

    • rdfeinman says:

      I stipulate but I am not sure what you are asking. I don’t know that there are essential sugars and as for non-essential sugars what is the relation to scurvy? As for the rats in the maze, in nutrition it is easy to lose track of where you are.

    • Alfonzo Luz says:

      Dear Dr. Feinman,

      Thank you for taking the time to post your response and so promptly. I appreciate it greatly.

      I don’t know what your posting policy is but if I overstep it, I have no objection to vigorous editing; it’s better than being deep-six’d.

      Here is a list of the essential sugars:

      I’m not sure about glucose because, although it is listed here, it is made within the body by normal mammalian biochemical processes and is therefore not dietary.

      As you know, there is a small number of animals unable to synthesize ascorbic acid in vivo. Homo sapiens is one of them. Stone suggested that since this was due to a mutation, the scorbuticus descriptor should be added to Homo sapiens because an earlier version had this ability and the modern version does not. When Szent-Györgyi isolated vitamin C he thought it was a sugar, not a vitamin, thus, it too might qualify as an essential sugar, I’m not sure, but it is essential and cannot be made within the body from glucose as it was in the earlier version. Without a sufficient amount of this essential “sugar” in the diet, latent scurvy is chronic and frank scurvy is inevitable. The common lab rat doesn’t get scurvy and this limits its value as a human surrogate. A closer match is the guinea pig.

      If you can cite a study that shows a requirement for the non-essential sugars in human metabolic processes then there might be a case for deciding which one is the best and which one the worst. Perhaps you know of such a study.

      As to the maze problem, didn’t Jean Mayer also discover that rats fed on a high carbohydrate diet didn’t fare as well in the maze as their brethren fed on the standard rat chow of the day? I seem to recall a book on the subject. HFCS is a recent invention of the Japanese, I believe, and is chemically distinct from Karo corn syrup. It would not have been available for Mayer’s HSPH Dep’t of Nutrition experiments. Dr. Lustig’s source is correct.

      As quickly as I read your books, I send them on to my colleagues. I am looking for your latest. Are you familiar with the work of John Ralston Saul, your countryman? He, like you, has to use all his names because of the similarity between his and another fellow’s in the same line of work. He covers his territory as thoroughly as you do; I think it must be something in the education system or in the water, as they say. I recommend his work as well as yours to inquiring minds.

      Warm regards,


      • rdfeinman says:

        By definition, you cannot show a requirement for something non-essential. The reference to missing-class-of-nutrients is far outside the mainstream of current biochemistry.

      • There is enormous number of papers showing that almost all common forms of common sugars diminish vitamin C bioavailability, metabolic effects and so on. Glucose for instance competes with all types of C transporters both intra and extra cellulary. Fructose for instance seems to downregulate GULO gene in animals and and will also bioavailability of copper (and thus in some people lead to deficiency) which on its own can present as cardiovascular disease which will require more vitamin C to fix related mess. Furthermore it negatively regulate the innate immunity against viral and bacterial pathogens by interfering with mannose binding lectin.

        For other, less common forms of sugar or sugar-friends, chemicals that almost always come along with sugar (like policosanol), the situation is less certain and this is for me very interesting topic for which documentation is almost non existent to my knowledge. While those substances might not be essential in strict sense, they still could have great impact on health.

    • FrankG says:

      Regarding essential dietary carbohydrates, Eric Westman wrote this in a letter to the editor of The American Journal of Clinical Nutrition

      “The currently established human essential nutrients are water, energy, amino acids (histidine, isoleucine, leucine, lysine, methionine, phenylalanine, threonine, tryptophan, and valine), essential fatty acids (linoleic and α-linolenic acids), vitamins (ascorbic acid, vitamin A, vitamin D, vitamin E, vitamin K, thiamine, riboflavin, niacin, vitamin B-6, pantothenic acid, folic acid, biotin, and vitamin B-12), minerals (calcium, phosphorus, magnesium, and iron), trace minerals (zinc, copper, manganese, iodine, selenium, molybdenum, and chromium), electrolytes (sodium, potassium, and chloride), and ultratrace minerals (4). (Note the absence of specific carbohydrates from this list.) ”

  5. George Henderson says:

    It was said of Charles Darwin, “much died with him, owing to his excessive fear of ever making a mistake.”
    This is some of your very best writing, clear as a bell; it ought to be in schoolbooks.

    All studies are imperfect perhaps; mainly to do with money and time; they could be larger, longer, more strictly supervised, use a different species/age/genotype; that is why studies are repeated with slight variations by the same researchers. But a study is really “flawed” when we can’t rely on the results, even if we agree with them.
    The abstract of a good study can be flawed, as with Krauss et al., without the study design itself containing the flaw. And so many studies, like Ludwig et al.’s recent one, are invested with flaws they never possessed by the media.

    • rdfeinman says:

      Sure. Nothings perfect and Krauss’s study is a very good one. It is the political edge that made him underplay the results and prevent a real impact. But good to go with the data.

  6. Richard says:

    Prof. Feinman

    Thank you for posting this.

    In your presentation it seems to me that the concept “It certainly seems like causality is implied here.” should be listed as part of a third aspect in how to look at these studies.

    As you point out, science does not run under such arbitrary rules as are often posted, such rules generally being stated in the name of not claiming too much, a dangerous and misleading form of something like false modesty. When you get a straight line with the data neatly clustered, as in the final graph, it means a lot.

    But really, underlying that, we have to get a better understanding of what we mean by “causality.”

    First: Do the graphs and observational studies show that increasing the saturated fat in the diet reduces the “marker” for the disease (just another correlation, really, and no more) or actual reduction in the bad outcome itself?

    Second: “Causality” sounds a lot like “proof” in the legal world, of which I am a small part. In this legal context there is sometimes a lot of evidence, or forms of evidence, such as statements from witnesses, documents, timelines, and actions by the parties. Proof, or having something proven, is a judgment, and not something done by the evidence on its own. That is, “proof” (and I would say causality, too, in this context) is not a function of evidence on its own, but rather requires some external third party (a third party who is agreed by the parties to be the arbiter) to reach a conclusion. That is the function of the judge and jury. And sometimes they get it wrong.

    Thus, causality, like truth, and like proof, is a slippery concept. The reason we have a hard time with causality from observational studies is not because they do not prove anything, but because even double blind studies fall short of proving or showing causality. Obviously the studies measure what they measure, but even the double blind studies measure only what the study is meant to show or prove, etc., leading to mistakes and later re-interpretation. If there is no way the evidence can take us all the way to causality, we are left with various ways of getting nearer to the truth, with various levels of success and utility.

    And the other problem here is that we cannot trust our judges. Who are the judges? The media? The Department of Agriculture? In the end we have to think for ourselves.

    • rdfeinman says:

      We are in agreement and point 3. and 4. will be about association and causality. I had raised the question before in Crimson Slime. Comment there:
      “It is commonly said that observational studies only generate hypotheses and that association does not imply causation. Whatever the philosophical idea behind these statements, it is not exactly what is done in science. There are an infinite number of observations you can make. When you compare two phenomena, you usually have an idea in mind (however much it is unstated). As Einstein put it “your theory determines the measurement you make. Pan, et al. were testing the hypothesis that red met increases mortality.”

      I think that there is much to learn in comparison with the legal system. I had several posts on “Evidence Based Medicine: Who Decides Admissibility?” which raise the questions you are asking and, yes, it is important, beyond admissibility of evidence, to have vigorous cross-examination and other practices.

      What is your relation to law?

  7. Graham Lutz says:

    When I get this question, I tell my friends to look at:

    who funded the research?
    how was the study designed? (controlled, randomized, double blind, in humans?)
    could any other conclusions be drawn from the data?
    what do your trusted sources have to say about the study?

    I’ve had some “very non-scientist” friends begin to read the media’s shenanigans critically!

    • rdfeinman says:

      Agree with all but the first which is generally not appropriate unless the study is clearly funded by a stated interested party. If Kellogg Cereals (just an example) funds the research then you have to understand their bias but if the Kellogg Foundation funds the research and the investigator says that they did not interfere, you have to take them at their word. Otherwise nothing is possible. To do otherwise is to suggest dishonesty in reporting the data which is a very serious charge and should not be done without proof. Of course, all kinds of biased interpretations are possible and you can take an author to task for biased writing but that is different than asking who funded the study. For my own work, the NIH, AHRQ and other federal agencies are extremely biased against low carb research and you will never get funding from them. The Atkins Foundation has done a fantastic job in supporting research that would not otherwise see the light of day. They are, of course, very sensitive about interfering in any way with research once it is funded. I actually fault them for their funding well-known researchers who don’t need their money in a mis-guided attempt to gain respectability. When I read to people the statement from the Women’s Health Initiative that the NIH had participated in designing their experiment and writing the paper, they are astounded. (The WHI is a special case, I think). When I told that to my collaborator, though, he said “Hell, we can’t even get the Atkins Foundation on the phone.”

      • Graham Lutz says:

        Good point – I’m not trying to suggest that people are being intentionally misleading, but when you enter a project with a preconceived notion and an idea of what you “should” show, it’s probably easy to be less critical or objective than you would have been otherwise.

      • rdfeinman says:

        Of course but you can’t impugn the motives of the researcher because of the funding source. All granting agencies have priorities and interests and your grant application will undoubtedly try to cater to those interests but once funded, you have to report what you find and the agency has to expect that it may turn out differently than they hoped. Of course this is an ideal and there are many cases where it is not done but you can’t accuse the researcher without hard evidence.

      • “but when you enter a project with a preconceived notion and an idea of what you “should” show, it’s probably easy to be less critical or objective than you would have been otherwise.”

        Unless the data you get contradicts your bias and you’re brave enough to admit it.
        Some of the best studies I’ve read were written up in a state of stunned surprise.

  8. Love the references to displaying info graphically. While getting my masters in applied statistics, I had a professor for regression analysis who often stated “regression without graphing the data is truly regression”. Seeing the plots can reveal so much.

  9. […] Reading the Scientific Literature – Helpful piece by Dr. Feinman discussing how you can spot bad science. […]

  10. Alfonzo Luz says:

    Dear Dr. Feinman,

    Thank you again for a prompt reply and thanks also to Frank G for his list of essential nutrients which does not show any evidence of carbohydrates as essential. The lists with which I am familiar show no essential carbohydrates either. If the 8 “essential” sugars are someday discovered by biochemists and cease to be a mystery, perhaps they will be added to the list.

    Wherein the harm? Oxidative stress is one candidate. If we pile on the sugar even where none is essential, the body must still metabolize it. The combustion process produces reactive oxygen species which must be quenched or else they can cause damage. When the spare electrons to neutralize free radicals without generating new ones are exhausted, the surplus is at liberty to go on its merry way. If glucose can’t be converted to vitamin C for the dual purpose of reducing the level of blood sugar and elevating the electron donor supply, oxidative stress, related to functional decline in brain structures, becomes a chronic condition. Is this not an argument for limiting dietary sugars to the vanishing point, just as Dr. Walford suggests? What!?

    Another possible source of harm was suggested by Dr.Rath when he pointed out that a surfeit of sugars in the blood competes with other similar molecules for building and repairing the structural components of the body. The substitutions produce imperfect versions and lead to premature failure. Not an appealing outcome in my book.

    I would like to point out that the best argument against dietary fructose is the preference of the bears. If Disney and National Geographic are correct, Ol’ Smokey gorges on a variety of woodland fruits in the Autumn and adds enough adipose tissue to last him through the long hibernation period. It is clearly Nature’s plan that we employ fructose and the other sugars found in fruits to ‘lard up’ for the winter. If winter never comes and we don’t do the long fast as the bear does, then the result is chronic obesity. Once obesity is established, reversing the condition is like trying to get the value of your stock portfolio back to the baseline after a crash. It’s a slow process. Mayer was partly right when he espoused exercise but the main advantage of a workout is to arrest the trend by “burning off” surplus blood sugar with vigorous training. While it’s true that exercise makes one hungry, why do we patronize the candy machines in the hall outside the gym instead of snacking on pemmican and pine nuts the way out leaner ancestors did?

    As long as i have your attention and the subject of bacon was brought up, have you read the ingredients on a package of bacon lately? Take a look next time you get a chance. There’s more there than just the fat and the protein. I am as weak on the subject of bacon as the next guy and my favorite vegetable is the polish sausage wrapped in bacon that is sold on a big bun by the charming lady from a cart down by the bus stop. But, I have studied the matter and there is more there to alarm a vegan than just fat and protein! A friend of mine has created his own version of the blue plate special at Costco with their $1.50 hot dog-and-a-cola combo with all you can eat condiments which is a lot. He has a few extra pounds on but if he limited himself to the meat portion of the bargain meal, he could easily lose 10 lbs. in a month. Dr. Budwig pointed to the difference between visceral fat and subcutaneous fat and how they metabolize differently in a weight-loss regimen. She also said that her experiments showed that “bad fats” block the metabolism of good fats. Her advice: stick to the good fats and the bad fats will fade away. I don’t think a doughnut contains any good fat.

    The problem remains, however, why does Homo sapiens scorbicus (var.) crave sugar?

    Thanks for hangin’ in there,

    Yours truly,


    • Lu says:

      Al, the notion of “bad fats” blocking the metabolism of “good fats” is way too general of a point to be accurate. When any feeding studies have been done where most of energy calories came from fat (as opposed to carbohydrate), nothing was “blocked.”

      Also, why homo sapiens crave sugar isn’t that big of a mystery. Many anthropologists and epidemiologists who study the trends in societies notice things. Carbohydrates in their many forms can and do fatten us up barring specific genetics (a person whose body disposes of glucose very efficiently, for instance). From there, just ask yourself where and when it would he helpful for a human to have fat and you have your answer. Here’s a hint, it’s similar to the bear situation you named. But as you stated as we do not do the long fasts (nor do we do appropriate nightly fasts for the most part as most people do not sleep enough or according to biorhythms), hence, chronic obesity. It’s just one of the discrepancies of modern living, we still crave what we do not necessarily need, unless we eat as if we do not need it (most people on long-term low-carbohydrate diets lose their cravings for sugar/starch).

  11. […] Colpo would enjoy reading and hopefully understand this blog by Dr. Richard Feinman, a man that Colpo lumps into what he calls the MAD men (metabolic advantage […]

  12. This would make Colpo’s head spin in low-carb, scientific agony.

  13. My quick take on this is that they weren’t all that clear which results came from which analysis method. It would be nice if they just write out the model that they are testing. I haven’t messed around much with the ANCOVA, but for any general linear model, you might have something like: outcome(s) = f(carbs weightloss carbs*weightloss). In this model, if the c*wl term is significant, the carb and wl terms wouldn’t be merely additive. This shows a relationship between carbs and weight loss, but of course, some of us already know about that 😀
    If the interaction term is significant enough to affect the way that conclusion is written, it is significant enough to be plotted so we can see exactly what effect it does have. If it is not significant enough to be discussed in the paper, then the term should be removed from the model and the researcher has to live with the higher error term that s/he gets. At least that is the way I would like to see it done.
    The problem with the above model, and every one for that matter, is that a casual reader will assume that the outcome has been caused by the factors in the model. Outcomes are CAUSED by the carb amount, and outcomes are CAUSED by the weight loss. This thinking is what encourages “evidence-based” adherents to declare that the research shows that the positive outcomes are caused by the weightloss because that is the answer that they want it to be.
    As in stepwise regression, they would pick weightloss as their first and only term in the model, and assume because adding a second term, carb, wouldn’t add much to the model because it is highly correlated to the first factor, that it is not a “cause”.

    • rdfeinman says:

      1. The figure of TAG vs. CHO that I indicates that you don’t need an interaction term; carbohydrate is what controls TAG. All of us “already know about that” and “some of us” say it in print.
      2. There is every reason to believe that the CHO does CAUSE the change in TAG because we know about the underlying physiology, the association is strong and has been repeated numerous times. Weight loss is NOT A CAUSE per se, in fact:
      3. An important flaw that I did not mention is using “weight loss” altogether. Weight loss does not happen spontaneously, it is not primarily a cause, it is primarily a response. Referring to weight loss is another informercial, more subtle that ‘healthy’ but nonetheless selling something. A sensible scientific analysis would have compared carbohydrate restriction with calorie reduction because those are the independent variables, that is what the experimenter changed. Those are the stimuli in the experiment. The effect after the calorie reduction period is not even reported as it should have been.
      4. The model is that calories reduce weight and the reduced adipose tissue releases hormones which affect TAG. This is reinforced by giving you the privilege of not looking at reduced CHO or insulin although the actual impact of Krauss’s study is to show that this is a bad idea. The ADA does this so that it can say that weight loss is most important rather than blood glucose which everybody knows is primarily controlled by dietary carbohydrate.
      5. I think we have reached a critical mass scientifically. Dietary carbohydrate directly, or indirectly through insulin, is the primary control of weight loss and atherogenic dyslipidemia.

      • Not to mention weight loss – if it’s muscle or bone mass – is unrelated to the desired effect of fat loss, which most believe is the desired outcome.

  14. Ah, the heart of the matter. If you are trying to discover the driver of all the other things, it would be nice to test for it. Putting weight loss in the driver’s seat (after presumably just removing it from the analysis as a noise variable) puts all the “weight” on the fat person with the bad behaviors. Just curious, who was the editor at the time the original study was published?

    • rdfeinman says:

      It’s not the editor’s jobs to find the misleading presentation, at least without input from the reviewers. That’s what the reviewers are supposed to do.

  15. […] Re: My introduction….. Now that Drama Class is over, here's another interesting read: Reading the Scientific Literature. A Guide to Flawed Studies. […]

  16. Hello Dr. Feinman:

    Can you suggest a doctoral program for Nutrition or related discipline where there is cutting edge research being conducted by faculty on Ketogenic diets?

  17. Alfonzo Luz says:

    Dear Dr. Feinman,

    Have you had any experience with Bastyr?

    Perhaps talking to a graduate would be helpful to Suzanne.

    About By Maureen Williams, ND
    Maureen Williams, ND, received her bachelor’s degree from the University of Pennsylvania and her Doctorate of Naturopathic Medicine from Bastyr University in Seattle, WA. She has a private practice in Quechee, VT, and does extensive work with traditional herbal medicine in Guatemala and Honduras.

    Vitamin C Can Prevent Pregnancy Complication
    Women who supplement their diet with a small amount of vitamin C during the second half of pregnancy reduce their risk of one contributor to premature birth, according to the American Journal of Clinical Nutrition (2005;81:859–63).The end of…
    Date Posted: August 16, 2005

    Our HQ is not far from BU and I sometimes chat with them at the supermarket.


  18. Alfonzo Luz says:

    Dear Dr. Feinman,

    Lu makes some good observations.

    My information on good fats v. bad fats comes from Otto Warburg’s student, Johanna Budwig. She issued some reports and wrote some papers on the subject and was immediately sued by the bad fat processors for disparaging food and giving their product a bad name. She successfully defended her science several times. If you are interested, some of her work has been translated into English from the German.

    On the question of the bears, I have found references to human hibernation and fasts extending beyond a month. I am sufficiently moved by my natural curiosity to place the subject of human hibernation on my list of things to investigate. It would seem that in an ice age or at high altitude, Homo sapiens dormer might have an adaptive capacity to “hole up” during bad weather. A factor to look into is whether Inuit People and Polar Bears “lard up” for the winter and if so, on what sort of a diet. In the case of the Inuit, larding up the larder might involve blocks of blubber in place of adding adipose.

    I agree with Lu on the necessity of incorporating “biorhythms” into our calculations. The circadian clock that governs human urges can easily be disrupted by our improvidence. As an example, when we retire with a cup’a cocoa, late insulin secretion into the blood is the result. Biochemists of my acquaintance inform me that high insulin interferes with the normal circadian mediated nocturnal flows of HGH, and creates the mayhem that ensues. The known fact that some people wake up “like bears in the springtime” should be added to a cliche dictionary if it isn’t there already.

    It is my experience that grazers are seldom anxious for a meal or excited by the wonderful aromas that issue from a well-conducted kitchen, but they are always snacking on something and some of them are in danger of becoming obese. Now, the 3 square meals a day types seem to have an appetite for the meals as the time nears, but, the vegans and vegetarians I know (and Mainland Chinese) seem to be eating something all the time and never have any cravings that I can perceive but do seem to favor things like bran muffins, supersweet corn, bananas and grapes.

    If Lu knows of a population that doesn’t crave sweets (chocolate, tobacco, gum), maybe they could be persuaded to answer a questionnaire for the general edification. I think the craving thing was pretty well explained by Saunders and Ross (and Wells) but the metabolic stimulus is elusive, very basic, and easy to conflate with addictions, don’t you agree? Lendon Smith called a low blood sugar attack the “all gone” feeling. That falls short of a description of “hunger” or “a healthy appetite”.

    Thanks for your dedication and enterprise,


    • Al, in my biochem text it is suggested that bears may be able to synthesize essential amino acids from urea during hibernation.
      My 1973 nutrition text states that these amino acids are essential because we synthesize them too slowly to meet replacement needs, not because we cannot synthesize them.
      Apparently PNG highlanders on low protein diets have gut flora that supplies missing amino acids . This suggests the mechanism for the other suggestions (plus, methionine can be supplied by recycling homocysteine).

      “From discussions on the above findings it is presumed that 15N urea may be utilized in PNG highlanders and Japanese controls of LPD to produce amino acids, especially lysine in the intestine where the bacterial species are changed by a long-continued protein-deficient diet from those of Japanese controls of SPD. The possibility of urea recycling was thus verified.”

  19. Look at this:

    The message is leaking out, slowly but in ever widening circles…

  20. […] Check out Dr. Richard Feinman’s Reading The Scientific Literature. A Guide To Flawed Research Share this:EmailPrintFacebookTwitterPinterestLike this:Like2 bloggers like […]

  21. Walter Wood says:

    Of course observational studies “imply” causality. They do not “prove” causality. If they did not “imply” causality, there would be no basis for hypothesis based on them.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s