How to Dissect a Nutrition Research Study

Nutrition research is ALWAYS evolving. There’s a new study every week that eggs are bad for you but they’re good for you but if you’re a diabetic they’re bad but if you eat more than 14 a month they’re bad. It’s EXHAUSTING and a roller coaster and perhaps makes you second guess how you’re eating. Nutritional research in general is incredibly hard to perform, and is perhaps why things seem to be constantly in motion.

Rather than being overwhelmed by what the research is, it may be more beneficial to dissect how it was performed. That way, you can be your own personal sleuth for finding out what this research actually means and how it translates to the general public / your eating habits.

Being realistic, these are the best ways we have to collect data so it’s not to say that all the results are trash. But it should motivate a little antenna to go up in your brain and make you say “hmm” okay maybe these results don’t 100% translate to real life.

1. How was the research collected? Was it something called a food frequency questionnaire or memory based assessment? These are notoriously unreliable, but unfortunately how most nutritional research is performed. It’s difficult for a few different reasons. First, human error. I’m sure if someone asked you what you ate for lunch two weeks ago you’d absolutely laugh in their face because there’s no chance you can remember. Also, sometimes participants seek approval from those they’re reporting to. Especially if it’s a study where consumption of a “bad” food is involved. So they either overestimate or underestimate their behaviors / actions and falsely report.
2. Are the researchers reporting relative risk or absolute risk? Often the relative risk (risk of something happening to you vs someone in another group) is a much higher number than the absolute risk (risk of something happening to you over a time period), therefore skewing the way in which you interpret the results.
   1. Relative risk: relative riskis when the study involves comparing the likelihood, or chance, of an event occurring between two groups. 
   2. Absolute risk: absolute riskof a disease is your risk of developing the disease over a time period
   3. Sometimes, scientists like to use relative risk to make their results sound more impressive. If a treatment reduces the risk of a disease from 2% to 1%, the absolute risk reduction is 1%. Treatment or no treatment, your absolute risk of getting the disease is pretty small. However, you could also truthfully say that the treatment reduces the risk by 50%, the relative risk reduction is 50%. This sounds more impressive, but it gives a skewed impression of how valuable the treatment actually is ( 1 ). According to Chris Kresser (acupuncturist and functional medicine expert dedicated to deciphering nutrition research), you have to see at least a doubling of risk, aka a 100% absolute risk increase in order to know you’re not just dealing with random chance. For example, when studying the links between smoking and cancer, the increased risk was 2,000-3,000%. Studies showing increases in risk 10-15% often times can be explained by other variables and are thereby “meaningless.” ( 2 )
3. What type of study is being performed? The gold standard of all research is randomized control, which is the only research that can absolutely determine causation rather than correlation. But this is extremely difficult with nutrition studies to look at real outcomes. Especially studies that require long-term data to track trends or patterns. It’d require locking people away for YEARS in order to get the precise kind of data to draw large conclusions, which is not only not possible but just straight up unethical.
   1. Randomized controlled: The gold standard is randomized control but this is extremely difficult with nutrition studies.
      1. benefits: can make causal inferences, so it’s strong evidence of a treatment’s efficacy; usually balance between the groups being studied, removes bias potential
      2. limitations: complex, expensive, time consuming
   2. Population based: A studyof a group of individuals taken from the general population who share a common characteristic, such as age, sex, or health condition. This group may be studied for different reasons, such as their response to a drug or risk of getting a disease.
      1. Benefits: “data already exist and valuable time has passed, complete study populations minimizing selection bias and independently collected data ( 3 )
      2. Limitations: “necessary information may be unavailable, data collection is not done by the researcher, confounder information is lacking, missing information on data quality, truncation at start of follow-up making it difficult to differentiate between prevalent and incident cases and the risk of data ( 3 )
   3. Observational: researchers observe the effect of a risk factor, diagnostic test, treatment or other intervention without trying to change who is or isn’t exposed to it. 
      1. Benefits: observing participants in their natural settings / environments
      2. Limitations: cannot conclude causation, only correlation ( 4 ). There can be a lot of factors that get crossed and there are a lot of conclusions that are pulled out of these on a regular basis that are actually correlations and not causations, and so we need to be careful about how we use the evidence that is shown in an observational paper. (1 )
   4. What context is the research taken in? For example, there are many research studies linking meat consumption to decreased life expectancy, heart disease, etc. But how are the researchers defining meat consumption? Are the participants eating Big Macs alongside a standard American / westernized diet? Or are they eating mostly plants with a few grass-fed / responsibly sourced servings of meat a few times a week? Obviously there would be extremely varied results between the two population groups.
   5. How long was the study run for? Long or short term? Good to know if major conclusions are being drawn based off of behaviors only tracked for a few years. Or if the study took place over a longer period of time, such as twenty years.
   6. Is the data correlation or causation? As mentioned above, the only way to really get down to a causal effect between two intervention is through a randomized controlled study, which is really difficult to do with nutrition studies. Otherwise, it’s merely correlation which doesn’t often carry much weight. For example, one researcher found that “Per capita consumption of margarine in the United States and the divorce rate in the state of Maine are correlated at 99.3 percent.” ( 1 ) Obviously these two things seem completely unrelated. For more funny examples of these, check out this website.
   7. Are there other things that could have caused the outcome? What are possible confounding variables in the study?
      1. Confounding variable: outside influence that changes the effect of a dependent and independent variable. This extraneous influence is used to influence the outcome of an experimental … Confounding variablescan ruin an experiment and produce useless results.
      2. For example, when looking at alcohol consumption (independent variable) and mortality (dependent variable), you may conclude that those who consume more alcohol are more likely to die. But this doesn’t take into effect that those who consume less alcohol are more likely to eat healthier or less likely to smoke, both of which could be a confounding variable ( 5 ).
   8. Who was the study performed on? Animals vs humans. While animal studies can provide extremely useful data, they don’t always translate 1:1 to humans.
   9. WHO IS FUNDING IT? This is extremely useful information to know. If there’s a study touting the benefits of dairy that’s funded by Hood, it maybe make you second guess the results. Often times companies want their research to support their industry. And if there were results that countered this, they may have simply left them out or skewed the research to match a desired result.

Hopefully using the above tools allows you to look at research in a more meaningful way, rather than blindly accepting whatever claims are in the conclusion portion of the paper. At the end of the day, YOUR own results say it all. If you’re eating a certain way, you feel great and your labs are great, that should say it all! Rather than getting so hyper-focused on every little new nuance in nutrition literature.

My favorite sources for finding new research: google scholar and pubmed

If you’re still skeptical or just kind of at a loss, Chris Kresser has amazing resources on all of this. Specifically, he has a lot of material about:

• how to decipher research studies
• saturated fat
• if eating meat is good for you
  • And more about that here
• You can also listen to him on this podcast which is another re-cap of some of the information listed above.