Food Pyramids and Medical Literature: Flawed Studies

We’ve all seen multiple iterations of food pyramids highlighting key food groups; read hundreds of articles about the next big thing to avoid if we want to follow a healthy diet; tried to practice better eating habits and yet, we’re still feeling lost. All of the information is conflicting. It changes all the time. And nobody seems to agree.

It begs the question: what if everything we think we know is right is actually wrong?

The following excerpt is from Nutrition in Crisis by Richard David Feinman. It has been adapted for the web.

Nutrition is in crisis.

Almost every day a new study shows that you are at risk for diabetes, cardiovascular disease, or all-cause mortality brought on by a newly identified toxin that turns out to be something that you just had for lunch. It is not clear that any of these studies are subject to serious critical peer review, and for the curious, bloggers usually do a good job of dismembering them. The continuous cycle of weak studies and their deconstruction goes beyond mere time wasting. People are hurt because bad recommendations are left out there, even when research shows that they are inappropriate—and, in the process, science takes a big hit. The editors and reviewers of technical and medical journals who conduct peer review are supposed to act as the gatekeepers of scientific evidence, but the journals continue to publish papers showing very weak associations, and even some that are grossly misleading or contradictory.

The media, which might be expected to help our case, make it worse. It’s not really their fault. A science reporter cannot reasonably have the time to read the original study in detail, and must instead accept the conclusions in the abstract, and so that message is transmitted through mass media. When you do explain to the reporter how misleading these reports are, and how people will be hurt, they are truly concerned and sympathetic, but they don’t always have complete editorial control. In any case they would like to help, but tomorrow they have to cover a story that may be even worse. It is really hard for the consumer. This post from Facebook probably tells the story: “So epically confused about diet. Everything I read is contradictory on epic proportions. About the only consistencies are low-sugar raw veggies and water. How in the world is a girl to sort it out, other than try everything and see what works for me?”

She went on to ask why it isn’t “possible to come up with a system that takes inputs—body stats and genetic history—and outputs a general reasonable diet to follow?” Chances are that the population at large is no more comfortable than this woman on Facebook when it comes to matters of nutrition. I wrote to her through private messages and reiterated the first three rules: (1) If you are okay, you are okay; (2) if you want to lose weight, don’t eat, and if you have to eat, don’t eat carbs; and (3) if you have diabetes or metabolic syndrome, you have to try a low-carbohydrate diet first.

It is likely that many people wind up believing nothing at all, however, and simply assume that everything is exaggerated, save for the most ingrained popular notion that “maybe fat is bad and maybe I should not put so much salt on my food.” Then there is the progression of articles on raspberry-ketones, resveratrol, trans-fats, and methylglyoxal, each of which will either kill you or save you, depending on whom you ask.

Most discouraging are the health agencies.

The American Diabetes Association (ADA) wants people with diabetes to consume a lot of carbohydrates. They keep saying that they don’t have a diet, and that they’re not opposed to low-carbohydrate diets (for weight loss)—but instead they stress “individualization” without any indication as to which individuals benefit from which intervention. Despite the disclaimers and ever-shifting language, there is no doubt that the ADA is perceived as opposing low-carbohydrate—and it seems clear they are the only ones responsible for that perception.

The evidence that weight loss is not required for improvement in diabetes, from the work of Nuttall and Gannon,1 for example, is not mentioned by the ADA. They know about that evidence. I’m sure of it because I have told members of the committee personally, and they should already be aware because some of the work was funded by the ADA itself. The ADA guidelines do not cite important scientific work showing that weight loss is not required for improvement in diabetes. People who are not scientists ask me, understandably incredulous, “Can you do that? Are you allowed to make recommendations without citing other people’s work?”

There is a daily progression of sweeping statements that go way beyond the published data. At the same time there remains an inability or an unwillingness to zero in on real factors. The low-carbohydrate diet has attained the status of the name of God in Hebrew: It must never be said out loud.

Part of the problem is that the literature, especially major medical publications, is still predominantly subscription-based. Most people cannot access the information, so the results are then fed downstream to the media, who take anything they are fed at face value, and then pass it on further to the general public.

The rigid dogma of the literature has reached Galilean proportions. Fructose and sugar are bad (unless you try to lump them in with all carbohydrates). If you want your paper on fructose to be published, begin with: “Because of the deleterious effects of dietary fructose, we hypothesized that . . .” Never start with: “Whether dietary fructose has a deleterious effect . . .” Our paper on fructose was published with “whether . . .” as the opening sentence,2 but only after a hard-fought rebuttal of reviewers’ criticisms that turned out to be fifteen pages long. Worse, if you even mention low-carbohydrate, you are guaranteed real grief. When the Journal of the American Medical Association published George Bray’s “calorie-is-a- calorie”3 and I pointed out that the study more accurately supported the importance of carbohydrate as a controlling variable, the editor refused to publish my letter. Thankfully, blogs have performed a valuable service by providing an alternative point of view, but if unreliability is a problem in the scientific literature, that problem is multiplied exponentially in internet sources. In the end, consumers might feel that they are pretty much on their own.

It does take some confidence, especially for the layperson, to feel that their intuitive understanding is correct: that the difference between white rice and brown rice is so small that it really doesn’t matter what Harvard’s computer says. Most researchers are very much disinclined to get into a shooting match, or worse, whistle-blowing. The long blue line is a strong force in repressing investigation, not because the authorities think corruption is okay, but because scandal reflects badly on everybody. Whistle-blowing in this field is especially weird because sometimes the transgressions are right out in the open.

Statistics: Death of the Medical Literature

Many scientists believe that if you do a good experiment, you don’t need statistics. David Colquohon, a well-known neuroscientist and critic of poor scientific method, agrees. Colquohon is the author of an excellent, if technical, statistics book, which is now freely available online.5 The introduction to his book points out: “ The snag, of course, is that doing good experiments is difficult. Most people need all the help they can get to prevent them making fools of themselves by claiming that their favorite theory is substantiated by observations that do nothing of the sort.”

This kind of circumspection is, unfortunately, more common among people who write the statistics books than those who use them. A good statistics book will have an introduction that says something like “In statistics, we try to put a number on our intuition.” In other words, it is not really, by itself, a science. It is, or should be, a tool for the experimenter’s use, and like any tool, you have to know how to use it. Because statistics offers so many tools, there is not always agreement on which hammer goes with which nail. All of statistics is interpretation. The problem is that many authors of papers in the medical literature allow statistics to become their master rather than their servant: Numbers are plugged into a statistical program and the results are interpreted in a cut-and-dried fashion. Statistical significance (that two sets of data are not from the same population) is confused with clinical significance (that differences are sufficiently large to have a biological effect). Misuse of statistics is the subject of numerous papers and books, but this has had little effect.

Notes:

• M. C. Gannon and F. Q. Nuttall, “Control of Blood Glucose in Type 2 Diabetes without Weight Loss by Modification of Diet Composition,” Nutrition and Metabolism 3 (2006): 16.
• R. D. Feinman and E. J. Fines, “Perspective on Fructose.”
• G. A. Bray et al., “Effects of Dietary Protein Contention, Weight Gain, Energy Expenditure, and Body Composition During Overeating: A Randomized Controlled Trial,” Journal of the American Medical Association 307, no. 1 (2012): 47–55.
• G. R. Norman and D. L. Steiner, PDQ Statistics, 3rd ed. (Hamilton, Ontario: B. C. Decker, 2003).
• D. Colquhoun, Lectures on Biostatistics (London: Oxford University Press, 1971), http://www.dcscience.net/Lectures_on_biostatistics-ocr4.pdf.

Recommended Reads

Plant and Mushroom Medicines for Cancer

Mitochondrial Theory: Setting the Mitochondrial Thermostat