Half of the people I speak to seem to be very confused about what on earth they should and shouldn’t eat. And I can’t really blame them. There are an avalanche of news articles, blog posts, podcasts and social media posts, all reporting on “the latest study” and what it means for how you should eat. So we’ve heard about studies which show that champagne is good for you, butter is good, butter is bad, coffee is good, coffee is bad, and even that chocolate will boost your chance of winning a nobel prize (seriously, that was a real paper).

And some wellness gurus like to use a whole bunch of cherry-picked and semi-relevant study citations, tacking them onto the end of their posts and sales pitches to give scientific legitimacy to their dubious health claims and nutrition recommendations. But just because someone cites a study or two does not mean that what they are saying is correct. I’ve seen a lot of selective citing and cherry-picking of studies and references to support all manner of nutritional quackery.

Be especially wary of the health guru who cites in vitro (aka test tube) and mouse studies to give authority to their recommendations about what you, a human being, should eat. These studies have an important role in science. But there can be a huge leap between how a nutrient acts in the test tube or in mice and how they may impact us when consumed in the form of food as part of our daily diet.

We need to critically evaluate each new study and decide where it fits into the accumulated body of scientific inquiry

With a couple of decades of legal practice under my belt, this was a particularly difficult concept to grasp. In law, when we are engaged in a dispute, and it’s a tough issue we can’t agree on, we have these super helpful people called Judges who can tell us exactly what the law is and how it should be applied to people and situations. Courts can hand down a decision in a new case which can quite literally sweep away all that came before it. And decisions from superior courts are binding on lower courts. It’s all very well organised and provides a reasonable degree of certainty - if we can’t resolve our disagreements then the court will.

But that's not how science works. We don’t have “science court”.*

In nutrition science, each new study adds yet another piece to the overall puzzle. And there are a LOT of studies out there. Scientists all over the world are researching and publishing. And their studies may conflict. Some studies will provide us with more compelling answers than others. If a study is very small, or not designed to minimise bias or error, it may warrant dismissal. Some studies can never give us cause and effect answers. RCTs are designed to test cause and effect, but careful account has to be taken of factors that could "confound" the results, or introduce error or bias. Careful researchers do their very best to minimise error and bias, and account for all known confounders.

Here is a list of 5 things to take into account when reading about “the latest study”!

1.There is a hierarchy of scientific evidence

Here is is in colourful graph form. I’ll briefly explain the different types of studies below, but for now all you need to understand is that different types of studies occupy different places in the hierarchy.

The Hierarchy of Scientific Evidence. Image Source: https://www.vox.com/2015/1/5/7482871/types-of-study-design

2. There are many different types of studies, but they fall into two broad categories

Nutrition studies broadly fall into two categories:

1. Interventions (aka experiments): this is where scientists roll up their sleeves and introduce changes (whether to something in a test tube, a lab rat or a group of humans) to see if these changes have a health outcome. So scientists might add a chemical or micronutrients to a cancer cell and report on its effects. Or they might feed mice a particular supplement or diet and measure changes in blood markers. Or they might take a group of people and ask half of them to eat a specific diet while the other half eats their usual diet, and then see if those on the special diet had better health outcomes like blood pressure or blood sugar levels.

2. Observational (aka epidemiological/population) studies which observe and sometimes compare groups (sometimes very large groups) to try to identify links between food or nutrient intakes and certain health outcomes. The scientists don’t change peoples’ diets or lifestyles, they just carefully observe them. Public health scientists often need to rely on these kinds of studies when investigating the harms of food and lifestyle choices like excessive saturated fat, high salt diets and smoking. That’s because it would be extremely unethical to recruit people to smoke cigarettes for example, or add high amounts of salt to their diet, when we know that doing so can harm them.

Read this EXCELLENT Vox article - it’s one of the the clearest and most accessible explanations of the different nutrition science studies I have ever come across (and it’s where I got the diagrams above and below from too!)

This Diagram Shows the Different Types of Experimental and Observational studies. Image source: https://www.vox.com/2015/1/5/7482871/types-of-study-design

3. The quality of studies varies

Nutrition studies need to be read with a critical eye. Some studies are meticulously set up to minimise bias and confounding. And others less so. So we can’t always take the findings of every nutrition study as gospel. This Vox article (https://www.vox.com/science-and-health/2018/7/13/17569730/savvy-science-reader) contains a very clear explanation of the things that can undermine the strength and veracity of scientific studies.

If you want to go full armchair science nerd, the Understanding Health Research site (https://www.understandinghealthresearch.org) is an excellent tool to help you assess the quality of any studies you might be reading.

4. Money talks in nutrition science, just as it does everywhere else

Food and drink is big BIG business. So are supplements. There are a lot of industry funded studies out there. It’s important to remember that industry funding of scientific studies isn’t necessarily a bad thing and studies that are industry-funded aren’t automatically biased. But we need to recognise that when a company or industry body funds research that could benefit it, that a conflict of interest potentially exists. Scientists may feel pressured to deliver favourable conclusions, or industry may choose to fund only those research teams or areas of scientific inquiry that are going to support their bottom line. A recent study** by investigators at the Charles Perkins Centre at the University of Sydney concluded that bias has been found in studies funded by industry, and the full extent is yet to be understood.

If you would like to read more about the influence of industry on nutrition studies and much more, I highly recommend you check out Marion Nestle’s Food Politics blog (https://www.foodpolitics.com), especially her “Industry Funded Study of the Week” posts and everything in her “Conflict of Interest” collection, or check out her book “Unsavoury Truth: How Food Companies Skew the Science of What We Eat.”

I wish more nutritionists and dietitians would openly discuss the way that the food and beverage industry has infiltrated universities, research teams and even their own professional associations. It amazes me that people who purport to rigorously adhere to evdence-based practice seem to ignore the significant body of scholarship that exists around conflicts of interest and the impacts that industry finding can have. Ignoring conflict of interest doesn’t make it go away. #micdrop

Careful nutritionists know that money can sometimes buy influence, so they usually take a peek at whether the authors of a study have declared any interests. And cynical semi-retired lawyers like me might also do a bit of googling around to see if particular researchers have any industry affiliations that aren’t declared. And they may choose to take this into account when assessing the quality of a study and how it fits into the overall body of scientific evidence for each topic.

I personally take note of industry funding and/or connections, and then carve out a little extra time to see how the conclusions of the industry funded study compare with other, non-industry-funded studies in the same area of inquiry.

5. Nutrition is a relatively new and rapidly evolving area of scientific inquiry

The first vitamin was isolated and studied less than 100 years ago.

The following diagram from the British Medical Journal is a fascinating representation of how nutrition science has grown and evolved in response to the growing body of research and accumulated knowledge.

You can see that prior to around the year 2000, nutrition scientists took a “reductionist” approach; looking at isolated nutrients and how they act in vitro, in animals or in humans, with a particular focus on correcting nutritional deficiencies.

But as nutrition science has evolved, scientists have gradually uncovered the flaws in taking a reductionist approach to the impact of diet on human health. As scientific understanding and research methods advanced, the focus is shifting to our overall dietary patterns and complex interactions like the impact of our diet on our microbiome and the effects it can in turn have on our health.

A Timeline of Nutriton Research. Image source: https://www.bmj.com/content/361/bmj.k2392

Want to read some more on this topic?

Check out this great Washington Post article: https://www.washingtonpost.com/lifestyle/food/heres-what-the-governments-dietary-guidelines-should-really-say/2019/03/25/69f86e12-4beb-11e9-93d0-64dbcf38ba41_story.html?utm_term=.192e0aaeb088

***

* I REALLY wish we had a science court. So some awesome science judge could just tell me exactly where the science is (taking into account every study and every strength and flaw) as of today for myself or my clients. It would save me a LOT of research hours!

REFERENCES

Maurage P, Heeren A, Pesenti M. Does Chocolate Consumption Really Boost Nobel Award Chances? The Peril of Over-Interpreting Correlations in Health Studies, 2. The Journal of nutrition. 2013 Apr 24;143(6):931-3.

Mozaffarian D, Rosenberg I, Uauy R. History of modern nutrition science—implications for current research, dietary guidelines, and food policy. bmj. 2018 Jun 13;361:k2392.

** Chartres N, Fabbri A, Bero LA. Association of industry sponsorship with outcomes of nutrition studies: a systematic review and meta-analysis. JAMA internal medicine. 2016 Dec 1;176(12):1769-77.