The Seventh Egg.

Posted: October 22, 2010 in Uncategorized
Tags: , , , , , ,

Stepping back and looking at the recent literature, I am struck with how life is a miracle.  How could humans have evolved in the face of threats from red meat, from eggs, even from the dangers of shaving?  (If you write about nutrition you have to create a macro that types out “I’m not making this up:” the Caerphilly Study [1] shows you the dangers of shaving… or is it the dangers of not shaving?).  With 28% greater risk of diabetes here, 57 % greater risk of heart disease there how could our ancestors have ever come of child-bearing age?  With daily revelations from the Harvard School of Public Health showing the Scylla of saturated fat and the Carybdis of sugar between which our forefathers sailed, it is amazing that we are here.

These studies that the media writes about, are they real?  They are certainly based on scientific papers.  If the media is not always able to decipher them, reporters do generally talk to the researchers. And the papers must have gone through peer review and yet many actually defy common sense.   Can the medical literature have such a high degree of error?  Could there be a significant number of medical researchers who are not doing credible science?  How can the consumer decide?  I am going to try to answer these questions.  When people ask questions like “could the literature be wrong?,” the answer is usually “yes” and I will try to explain what’s wrong and how to read the nutritional literature in a practical way. I am going to try to make it simple.  It is science, but it is pretty simple science.  I am going to illustrate the problem with the example of a paper by Djoussé [2].  But first, a joke.

It was a dumb joke. In my childhood, there was the idea, probably politically incorrect, that Indians, that is, Native Americans, always said “how” as a greeting.  The joke was about an Indian with a great memory who is asked what he had for breakfast on New Years day the previous year.  He says “eggs.”  They are then interrupted by an earthquake or some natural disaster and the interviewer and the Indian don’t meet again for ten years.  When they meet, the interviewer says “how.”  The Indian answers “scrambled.”

If the interviewer had been an epidemiologist he might have asked if he had developed diabetes.   Djoussé, et al. [2] asked participants about how many eggs they ate and then ten years later, if they developed diabetes it was assumed to be because of the eggs.  Is this for real?  Do eggs cause changes in your body that accumulate until you develop a disease, a disease that is, after all, primarily one of carbohydrate intolerance?  The condition is due either to the inability of the pancreas to produce insulin in response to carbohydrate (type 1) or to impaired response of the body to the insulin produced and a deterioration of the insulin-producing cells of the pancreas (type 2).  Common sense says that there is something suspicious about the idea that eggs would play a major role.  It is worth trying to understand the methodology and see if there is a something beyond common sense, and whether this is a problem in other studies besides   Djoussé’s.

What did the experimenters actually do.  First, people were specifically asked “to report how often, on average, they had eaten one egg during the past year,” and “classified each subject into one the following categories of egg consumption: 0, < 1 per week, 1 per week, 2-4 per week, 5-6 per week, and 7+ eggs per week.”  They collected this data every two years for ten years.  With this baseline data in hand they then followed subjects “from baseline until the first occurrence of a) type 2 diabetes, b) death, or c) censoring date, the date of receipt of the last follow-up questionnaire” which for men was up to 20 years.  Thinking back over a year: is there any likelihood that you might not be able to remember whether you had 1 vs. 2 eggs on average during the year?  Is there any possibility that some of the men who were diagnosed with diabetes ten years after their report on eggs changed their eating pattern in the course of ten years?  Are you eating the same food you ate ten years ago?  Quick, how many eggs/week did you eat last year?

Reading a scientific paper: the Golden Rule.

Right off, there is a problem in people reporting what they ate but this is a limitation of many nutritional studies and, while a source of error, it is depends on how you interpret the data.  All scientific measurements have error.  It is not a matter of ignoring the data but rather not interpreting results beyond measurement.  So, here’s how I read a scientific paper.   First, I look for the pictures.  What? A professor of biochemistry looks for the pictures first?  In a scientific paper, of course, they are called figures but it’s not just saving a thousand words.  (I get a thousand emails every couple of weeks). It’s about presentation of the data.

The principle is that a scientific paper is supposed to explain.  I tell graduate students that if you do an experiment and you don’t explain it well, it is as if you had never done it.  I ask students “Describe what you are supposed to do in a scientific seminar or other presentation.”  After a while, I say: “No. In one word.  What are you supposed to do?  One word.”  The answer is “teach.”  The same is true of a scientific paper.  The principle is laid out in what I call the golden rule of scientific papers.  It comes from the Book PDQ Statistics by Geoffrery Norman and David Streiner.  PDQ stands for Pretty Darned Quick and some of the humor is pretty sophomoric (e.g. it has Convoluted Reasoning or Anti-intellectual Pomposity detectors) but it is an excellent introductory statistics book.  Here’s the Golden Rule:

“The important point…is that the onus is on the author to convey to the reader an accurate impression of what the data look like, using graphs or standard measures, before beginning the statistical shenanigans.  Any paper that doesn’t do this should be viewed from the outset with considerable suspicion.”

– Norman & Streiner, PDQ Statistics [3]

In other words: teach.  Make it clear.  Eye-balling Djoussé, et al., we see that there are no figures.     A graph of number of eggs consumed vs number of cases of diabetes is what would be expected of the golden rule.  The results, instead are stated in the Abstract of the paper as the following mind-numbing statistics:

“Compared with no egg consumption, multivariable adjusted hazard ratios (95% CI) for type 2 diabetes were 1.09 (0.87-1.37), 1.09 (0.88-1.34), 1.18 (0.95-1.45), 1.46 (1.14-1.86), and 1.58 (1.25-2.01) for consumption of <1, 1, 2-4, 5-6, and 7+ eggs/week, respectively, in men (p for trend <0.0001). Corresponding multivariable hazard ratios (95% CI) for women were 1.06 (0.92-1.22), 0.97 (0.83-1.12), 1.19 (1.03-1.38), 1.18 (0.88-1.58), and 1.77 (1.28-2.43), respectively (p for trend <0.0001).”

What does all this mean.  In a future blog, I will interpret these “statistical shenanigans” and explain why what the study really shows is nothing.  Zero. Zilch. Zimbabwe.  I will explain the terms in these results and what to make of it and ask whether it was justified of the authors to conclude: “These data suggest that high levels of egg consumption (daily) are associated with an increased risk of type 2 diabetes in men and women.”

For the moment, I will just state what the statistics mean because it is worth considering the conclusion as stated by the authors.  The meaning of the statistics is that there was no risk of consuming 1 egg/week compared to eating none.  Similarly, there was no risk in eating 2-4 eggs/week or 5-6 eggs/week.  But when you up your intake to 7 eggs or more per week, that’s it.  Now, you are at risk for diabetes.  The relative risk is small but there it is. You are now at greater risk.

Since I like pictures, I will try to illustrate this with a modified still from the movie, The Seventh Seal directed by Ingemar Bergman.  Very popular in the fifties and sixties, these movies had a captivating if pretentious style: they sometimes seemed to be designed for Woody Allen’s parodies.  One of the famous scenes in The Seventh Seal is the protagonist’s chess game with Death.  A little PhotoShop and we have a good feel for what happens if you go beyond 5-6 eggs/week.

In a future blog I will explain hazard ratio and the related ideas of odds ratio and relative risk.  In the mean time, consider that these mean just what you think, that is, they are betting odds, that is, if you knew the odds at two different black jack tables, and the relative odds were less than 2:1 would you know which to play.

1. Ebrahim S, Smith GD, May M, Yarnell J: Shaving, coronary heart disease, and stroke: the Caerphilly Study. Am J Epidemiol 2003, 157(3):234-238.

2. Djoussé L, Gaziano JM, Buring JE, Lee IM: Egg consumption and risk of type 2 diabetes in men and women. Diabetes Care 2009, 32(2):295-300.

3. Norman GR, Streiner DL: PDQ statistics. 3rd edition. Hamilton, Ont.: B.C. Decker; 2003.

Comments
  1. Richard Koffler says:

    Could it be that egg consumption is accompanied by consumption of refined-carb staples like bread and jelly? Ergo, the more eggs consumed, the more bread and jelly consumed.

    • rdfeinman says:

      If there really is a correlation it could be for the reason you say but I don’t think the correlation has been shown. I didn’t go into everything that is wrong in the assumptions in this paper but the method is to collect data into quintiles (losing much information), to incorrectly assume that the trajectory of the population (going from low to high) reflects individual trajectories rather than, say, some people reducing risk while others increasing risk and, in the end, you have to go to the extremes of egg consumption to see their claimed increase in risk and so they could have said: there is no risk for type 2 diabetes associated with egg consumption except at the extreme difference between very low and very high. Most of all, there is no reason to look at eggs in the context of diabetes and the lack of mechanism suggests the data would only tell you something if the numbers were big which they are not. Somebody should be funded to re-evaluate this data with an open mind and see what dietary factors show strong associations with diabetes. This paper does not show anything at all, either way.

  2. Mie says:

    “there is no risk for type 2 diabetes associated with egg consumption except at the extreme difference between very low and very high.”

    Which is what comes across very clearly from the extract quoted from the paper. Academic language is not a synonym for statistical shenanigans.

    “Most of all, there is no reason to look at eggs in the context of diabetes”

    Well, how do you know that?

    “and the lack of mechanism suggests the data would only tell you something if the numbers were big which they are not.”

    But surely you understand that a study of this kind cannot show the exact mechanism? This can be inspected with a well-designed RCT. And how do you think the possible study design for a particular RCT could/should be formed, if not on the basis of issues discovered in cohort data?

    • rdfeinman says:

      First, this was a blog post, even included a joke. In fact, I think that is appropriate because, although I tried to present my arguments in a logical way, I think this is not an acceptable paper as it stands. If you think this paper was subjected to vigorous cross-examination, that is appropriate peer review, and that the conclusions are established without need to see more data, then you should quit eating all those eggs.

      If I had reviewed the paper, I would have asked for individual data to be shown graphically. Why do we use quintiles (or quartiles, or whatever)? It is an anachronism from when computers were not on everybody’s desk. Quintiles obscures data. What you want to know is if you look at the people who eat more than 7 eggs a week, how many get diabetes?

      “‘Most of all, there is no reason to look at eggs in the context of diabetes’
      Well, how do you know that?”

      If you think there is a reason, what is it? How do you know there is no reason to test pineapples? You do a correlation to test a hypothesis not, as is usually said, the other way around. I previously quoted Einstein as saying that your theory determines what you measure. Of course, sometimes you have a hunch and you see a previously unexpected strong association. But there is nothing strong here. In fact, I haven’t checked, but my recollection is that when d’Jousse checked a different group there was no association.

      I’m just a chemist. If you think you have nothing to learn from me about experimental design, ok, but you might consider that my criticisms of standards in the medical literature are worth turning around in your mind. There is really no such thing as training in science but many people feel that, whatever it might be, physicians are not well-trained and the medical literature is not where the high standards are. I though my Mozart analogy was a good one.

      • Mie says:

        “If you think this paper was subjected to vigorous cross-examination, that is appropriate peer review, and that the conclusions are established without need to see more data, then you should quit eating all those eggs.”

        You do realize that the paper doesn’t advice people not to eat eggs, nor does it claim to have established that eggs are dangerous etc., right?

        “If I had reviewed the paper, I would have asked for individual data to be shown graphically.”

        Yes, from what I’ve gathered you do like your graphs …

        “Why do we use quintiles (or quartiles, or whatever)?”

        Umm, since they serve a purpose?

        “If you think there is a reason, what is it?”

        One shouldn’t answer a question with a question. Anyway: I do not know. That’s THE point in science: nothing is certain. Some of the most important discoveries have been a result of a “leap in the dark”. Ok, that’s certainly more than far-fetched in this case, but the main point is that the absence of a credible mechanism at this point doesn’t guarantee that there isn’t one – or that the issue should not be subjected to testing.

        “You do a correlation to test a hypothesis not, as is usually said, the other way around.”

        And I thought that looking at data for correlations is quite normal and serves its purpose … How do you think that e.g. the relevance of lipoprotein concentrations in CVD risk was first discovered?

        “I’m just a chemist. If you think you have nothing to learn from me about experimental design, ok, but you might consider that my criticisms of standards in the medical literature are worth turning around in your mind.”

        You do have many good points. But your posts and ideas aren’t above criticism, either.

        “There is really no such thing as training in science but many people feel that, whatever it might be, physicians are not well-trained and the medical literature is not where the high standards are.”

        Undoubtedly yes. Yet, many of abovementioned “people” cite studies of a particular design as convincing evidence in point A and trash the very same study design in point B. Go figure …

  3. [...] are at high risk as a result of genetic or lifestyle risk factors. As SUNY professor and biochemist Richard David Feinman put it, “this is a limitation of many nutritional studies and, while a source of error, it is depends on [...]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s