Photo by Rustic Vegan on Unsplash
Game Changers
The last few weeks I have been bombarded with questions about the documentary Game Changers. In this documentary the benefits of eating plant-based foods are highlighted as well as the dangers of eating meat. Viewers, many of them athletes, are alarmed by some of the messages, scientists are telling us how dangerous eating meat is and how great eating vegetables is for your health and performance. Have they missed something? Should they now switch to a plant only diet? The documentary that pretends to search for the truth… but does it?
Before we look into the evidence a bit more, let me first start with saying that I have no problem with vegetarian eating, with vegetarians or vegans, and I think everyone should make their own decisions when it comes to food intake. If you want to eat paleo that is your choice, if you want to eat ketogenic, that is your choice, if you want to be a vegetarian or vegan this is absolutely fine. My daughter is a vegetarian, even though she doesn’t like vegetables… It is her choice. But although I don’t have a problem with people’s individual food choices, I do have a serious problem with people who want everyone else to eat their way, or people that claim that their way is the only way. I have a problem with information that is confusing, misinforming or even misleading. I also have a problem with bad science and when claims are made that scientific methods are used when they clearly are not. I also have a problem with scare mongering: if you eat meat you will die!
Well, Game Changers seems to do all of these things that I have a problem with. Game Changers ticks almost all the boxes of pseudoscience, and none of the boxes of science (Below we will go through some examples and discuss what pseudoscience is and what science is).
Game Changers ticks almost all the boxes of pseudoscience, and none of the boxes of science
Are you confused by all these diets we have been bombarded with in the media? We had paleo a little while ago. We had to eat like our ancestors otherwise we would develop diseases and die. Then we had keto, we need to go keto otherwise you will get diabetes and die. Avoid carbohydrate! Now we can’t be keto and definitely not paleo, we need to avoid meat! We have seen three massive trends in the space of a few years. Believe me, scientific insights haven’t changed in this short period of time! Science doesn’t move that fast…. However, different interpretations and especially opinions do change fast… Unfortunately, these are not always based on science or evidence, even thought they may pretend that they do.
There are some conclusions and statements in the documentary I personally agree with and the scientific community in general agrees with as well. The main point of agreement being that we should eat more fruit and vegetables. I think there is a lot of evidence that this can help our health. But this is also a general recommendation we are given, anyway! Haven’t we been bombarded with this message? Haven’t we bombarded with the 5 a day message? This doesn’t mean you have to become vegetarian or vegan, you just need to eat more fruit and vegetables. If this is the takeaway from the documentary or the effect of it than that is great.
A closer look
I watched the documentary 3 times. The second and third time I specifically paid attention to how the arguments were built, what evidence was used and also what evidence was NOT used. It is actually the latter that is the bigger problem and the first sign that we are dealing with pseudoscience… nowhere in the entire documentary is there a discussion or even a mention of different views. In fact, the body of evidence against some of the arguments, is far greater than the few very small studies that are cherry picked to support some of the points in the documentary. Most of the studies that are picked are also published in lower quality journals where there is generally less scrutiny. Some of the large scale studies are not mentioned because the results do not align with the messages of the documentary. This is what pseudoscience does. If you are engaged in pseudoscience you have an opinion and search for evidence that supports your view. You present all of this information and say: see this proves my theory is right. This is completely different from what science is, and does. Science does not prove anything. Science will bring us closer to the truth. Science will formulate a hypothesis and design a study to test this hypothesis. The results are critically discussed: arguments and counterarguments are carefully analysed. One study will not give the answer. Science is evaluated in the totality of evidence, not just the cherry picked studies that support your hypothesis. Science is scrutinised by experts whom may have different views, but objectively look at the data collected.
Science is evaluated in the totality of evidence, not just the cherry picked studies that support your hypothesis.
The documentary is the personal journey of James Wilks, a former MMA fighter who describes how he got injured, started to read and stumbled across what he calls “a study on the Roman Gladiators” that “blew his mind” - A scientific looking article gets flashed in front of us on the screen… - I checked this reference and was disappointed that it was not a study but a narrative, a magazine article. The narrative describes how roman Gladiators needed to be fat to be well protected and to do this they ate very large amounts of barley, and legumes, like beans. I looked for the underlying study by Dr Fabian Kanz and this study shows evidence from isotopic data that suggest that gladiators had a mixed diet with a high share of vegetables. So, they were NOT vegetarians or vegans. They just ate a lot of vegetables. Kanz explains how they found that gladiators had very strong bones (high bone mineral density) which can be explained by training in combination with a good diet. In the documentary it is not mentioned that the reason gladiators did this was to get fat (not to get strong bones).
The red flame makes great TV, but is not reality.
A study in PLOSone 1 uses different isotopic methods to deduce what the diet of gladiators might have been. Different amounts of foods in the diet will results in different amounts of strontium in bones. In general, low strontium levels are found in carnivores and high levels in vegetarians. Strontium levels can be visualised as a flame. If the flame turns red there is a lot of strontium.. and a huge red flame flashes in front of us: Conclusion the gladiator was vegetarian…. Well no. Although our mind likes black and white, or in this case red and blue, the reality is that most real life situations are shades of grey. So what we see, a red flame, would only occur in an extreme case. Usually the distinction between diets is not that clear. There is considerable overlap in the light spectra between carnivorans and vegetarians. The red flame makes great TV, but is not reality.
There are some similarities to an example I often use when discussing a really obvious example of pseudoscience: oxygenated water. In one of the adverts of this magic “water” that is promoted as the cure for virtually everything, two thermal images of a person are presented. One image with a subtitle BEFORE is red, the other image with the subtitle AFTER is mostly green.
Before drinking oxygenated water, the person is red. The text on the web site explains that this means inflammation. After drinking the oxygenated water, the person turns green. The inflammation is gone. Thus, oxygenated water removes inflammation! There is the evidence! Well no, this is not evidence: the amount of oxygen you can get into water is incredibly small and we do not absorb oxygen in our intestine: we have lungs for that. So, you might as well breath in a couple more times, and that is free! Drinking this expensive water does not affect inflammation. These sorts of before after comparisons are meaningless, unless we are told exactly how the results were obtained but the answer to this can never be found in pseudoscience. But the flames, the lab coats the flashing of research papers on the screen quickly gives the impression that we are dealing with real science here!
It is also not surprising that the diet of a gladiator was predominantly plant-based. Roman legionnaires had daily expenditure of energy that can be estimated at around 5000 kcal for the legionnaire performing engineer work and at 6000 kcal in training or war action. This is comparable to Tour de France cyclists, who have a high protein intake, but the vast majority of their intake is carbohydrate (pasta, rice, bread, cereals, potatoes)! For legionnaires this came mainly from wheat or barley. There is no practical way to get this many calories from meat and it couldn’t provide the carbohydrates you need to perform the training.
Graphs pop up on the screen.. looking highly scientific but these graphs are misleading and really make no sense at all. Early on there is a pie chart that shows that protein becomes about 60% and carbohydrate only 40% when meat is consumed. This does not make any sense at all…. 60% of what??? 40% of what? Protein intake is typically 10-15% of someone’s energy expenditure. A high protein intake can result in 20% of energy intake coming from protein. I don’t think 60% would even be possible to achieve with normal foods. The graphs, look scientific, but do no represent reality and are misleading.
Carbohydrate is plant and plant is carbohydrate?
It also seems that throughout the documentary the words carbohydrate and plant foods are used interchangeably. A plant food is not a carbohydrate and a carbohydrate is not a plant food. Plant foods can of course be high in carbohydrate but many of them are not. So, it is nonsense and misleading to use the terms as synonyms. All of this happens already in the first few minutes of the documentary, setting the tone for what is to come… but the lay viewer is terribly impressed by the huge shifts (whatever they are), the big numbers and the take home message is clearly that eating meat affects your performance.
They say...
An approach that is often and successfully used by pseudoscience is making a statement as if this is consensus (everyone believes….) and then this statement is attacked. The statement here is “Athletes believe protein is for energy”…. I have worked with many athletes in the last 25 years and I don’t think I have met one athlete who thinks protein is for energy. I think that almost all athletes would say that it is for recovery, for muscle building and maybe some smart ones would say it is for adaptation. But the documentary starts to build on this “athlete’s belief that protein is for energy…”. It is traced back to a paper by von Liebig in the late 1800s and indeed this paper claims that protein is an energy source. But this theory got refuted and in 1912 it was shown that carbohydrate and fat were the main fuels. More studies followed in the 1920s. The statement that athletes 100 years later think protein is for energy is absurd. But it helps the makers of the documentary to then say, can you believe it, we have been wrong all this time? But now we have found the solution! Now we know better!.... the reality is that nothing has changed. We didn’t believe protein was a fuel before, and we still don’t.
Enough protein from plants?
Wilks acts surprised that you can get enough protein from plants. Even he as an athlete with increased needs, can still meet his needs using plant based foods. But is this really so surprising? Athletes have slightly higher protein needs than the average person but they also need more of most other nutrients. They also spend more energy and they will also eat more and thus meeting these needs is not often a problem. Meeting these needs from plant based foods is not impossible, and I am not sure anyone has ever claimed that it is not possible, but a vegetarian or vegan athlete requires a little more knowledge of what to eat and perhaps a little more effort.
Plant based inferior
In the documentary James Wilks says: “I have always heard that plant based proteins are inferior”. Well they are, if you compare them like for like! Studies consistently show this… the documentary doesn’t provide the references so I will list some of them for you here: 2, 3, 4, 5, 6.
Soy is probably the most studied and most effective plant based protein, but studies assessing the protein synthetic response to soy protein ingestion have shown that the ingestion of amounts ranging from 17.5 to 40 g soy protein do not increase muscle protein synthesis to the same extent as the ingestion of comparable (isonitrogenous) amounts of whey protein 2 4, skimmed milk 5, or beef 6, both at rest and post-exercise.
Incomplete
Next another statement is attacked that no scientists ever said… “Plant based proteins are not complete so you are not going to get all the amino acids”… I don’t think we are concerned about not getting all of the amino acids… we are concerned about the optimal amounts or rations of amino acids. And it is not just about the mounts of certain amino acids in the protein we eat, it is also about the digestibility and absorption of that protein. We know, for example, that in order to optimise protein synthesis we need 2-3 grams of leucine as part of your protein intake in a meal. This is simply easier to obtain from animal protein than plant based protein. For example drinking a glass of milk is probably easier than eating a cup of lentils, . No one said it is not possible to get 3 grams of leucine from plants.
Of course, the more you eat, the easier it becomes to reach this target. The documentary concludes: “research comparing plant and animal protein has shown that AS LONG AS THE PROPER amount of protein is consumed the source is irrelevant”.
This is true. But the practicality of this is that you will need to eat quite a lot of vegetables to meet the targets for optimal protein synthesis whereas the same could be achieved much easier with smaller meals containing animal based proteins. This is conveniently left out of the discussion.
Many many papers..
Instead what is shown to the viewer is this: Research has consistently shown…. Papers flash up on the screen to indicate the large number of studies and the references are shown to back it up… the viewer thinks “Wow all these studies”… However, if you look at these references and the papers flashing up on the screen in more detail… they are not actually studies apart from one! Misleading! The rest are review papers where the literature is summarised and where perhaps a statement is made to say that “if you eat the proper amount of amino acids, the source does not matter”. The only paper that somewhat addressed the issue is a study where whey and rice protein are compared and the results show a similar response between whey and rice protein over an 8 week period. Time to have a conversation with myself about this:
Me: Hang on…. Did subjects eat only whey or rice protein for 8 weeks??
Me: No, of course not, they ingested some on training days.
Me: What about the rest of the day?
Me: They ate their normal protein sources including animal and plant based proteins
Me: So, it is possible that of no supplement was ingested the same results would be obtained.
Me: Yes, this is entirely possible.
Me: So, then the study doesn’t really show much?
Me: Correct
Anecdotes
Patrick Baboumian, the worlds strongest man, claims to have gained 25kg after switching to a plant based diet. Whilst this may be true, is it really the plant based diet that is responsible for this? Would it have been different with a different diet? Did the training have anything to do with it? Should other factors perhaps be considered? Strongman competitions do not have an official drug testing policy in place. There is also no mention of protein powders and various other supplements that would have been used. There are so many factors besides the diet that could have contributed to the weight gain. With such anecdotes you can conclude that it is possible to gain 25kg on a plant based diet but does not mean this could not have been achieved with a different diet. It does not even exclude the possibility that another diet would have resulted in greater effects. All it is, is an anecdote. It would be possible to find an anecdote of a meat eater with similar results, but that would also be meaningless.
Inflammation
Then the discussion inn the documentary turns to inflammation.. Inflammation is bad! We need to prevent it, right? Well, no, inflammation is what you need to get better. Inflammation is a signal to turn on repair processes in the body and these will ultimately make you better. No inflammation, no improvement. What we do want to prevent is excessive inflammation.
A plant based diet will reduce inflammation by 29% and bench press performance will improve 19% by drinking a glass of beetroot juice… I think anyone who has been in a gym would (and definitely should) raise their eyebrows with this kind of reporting.
We should not take results from one studies and present them out of context. If we want to know what the effect of beetroot juice is, we should look at the collective evidence: all the relevant studies on beetroot juice. Some of these studies will have results one way, other studies have results a different way and the scientists will have to come up with a conclusion looking at all these studies. That conclusion is NOT: drink a glass of beetroot juice and increase your benchpress by 19%!
Study on Hamburgers
But now the documentary turns to a study on hamburgers. “The same study that showed that a single hamburger impairs blood flow (27%) also reported an increase in inflammation by 70%. In the arteries inflammation reduces blood flow, in muscles and joint it can increase soreness and delay recovery”. Viewer: So, meat causes reduced blood flow and inflammation and inflammation is really bad…. Well let’s have a closer look at this study.
It is a very small study in which 11 subjects in randomised order ate a burger or a burger with avocado. The study shows a significant reduction in blood flow and this reduction does not occur when avocado wad added. A number of markers of inflammation are measured as well. There is no difference in serum TNF-alpha. There is also no difference in serum IL-6 one hour after the meal, or 2 h after the meal or 3 hours after the meal, 5h after the meal or 6h after the meal, but at 4 h after the meal there is a higher IL-6 concentration in with the burger without the avocado. This hardly means that inflammation is reduced by 70%.... Inflammation is a complex process with many, many mediators. When you measure a change in one of the hundreds of mediators that could have been measured and this change is only observed at one time point, in a small number of subjects, the conclusion that plant based diets reduce inflammation by 70% is perhaps a little premature. Oh, and by the way, the study was funded by the Avocado industry. When the documentary refers to a study that was funded by the meat industry, it is immediately noted that in the fine print states that the study was funded by the meat industry…. However, the same level of scrutiny was not used in the documentary when this avocado study was discussed..
Strong claims
The documentary continues. Strong claims are made: In plant based protein you are getting protein that is packed with antioxidants, phytochemicals, minerals and vitamins that are gonna reduce inflammation, optimise the microbiome, optimise blood supply and optimize your body’s performance! Wow! It would be great to have the evidence to back up this statement! But there is nothing to support these claims in the literature. Take the last claim. Optimise your body’s performance (whatever that means). There is not one study that I can think of that shows that plant based proteins optimise your body’s performance. I am not saying it not true or not possible, I am just saying that there is no evidence that this is the case.
The problem with relative risk
Next, a clear consensus is presented when you cook food or even digest proteins highly inflammatory compounds are formed: heterocyclic amines, nitrosamines, trimethylamine N-oxide and these compounds "corrode the cardiovascular system". Only there is not that much consensus and clarity. In fact there is quite a bit of debate about the role of these compounds and the effects they have in real life situations where they are often consumed together with fruits and vegetables, because that seems to reduce any potential negative effect.
The next claim and big number comes flying in! “People who get their protein from plant sources reduce their risk of cardiovascular disease by 55% !” Now this is a scary number! A 55% greater chance of cardiovascular disease that is huge… or not?
I am now turning to an example from Ben Goldacre’s book Bad science, a great read about mistakes that are often made interpreting the results of scientific studies. The article “You are 80 times less likely to die from a meteor landing on your head if you wear a bicycle helmet all day”, describes the problem with interpreting relative risks {Goldacre, 2016 #10926}.
It described what happened in the media when they reported on a new drug: rosuvastatin “Most papers called it a “wonder drug”. The Express, bless them, thought it was an entirely new drug. “Heart attacks were cut by 54 per cent, strokes by 48 per cent and the need for angioplasty or bypass by 46 per cent among the group on Crestor (the brand name for rosuvastatin) compared to those taking a placebo or dummy pill”, said the Daily Mail. Dramatic stuff. The Guardian reported: “Researchers found that in the group taking the drug, heart attack risk was down by 54% and stroke by 48%”.
Is this true? Yes. Those are the figures on risk, expressed as something called the “Relative Risk Reduction“. It is the biggest possible number for expressing the change in risk. But 54% lower than what? This was a trial looking at whether it is worth taking a statin if you are at low risk of a heart attack (or a stroke), as a preventive measure: it is a huge market – normal people – but these are also people whose baseline risk is already very low.
If you express the exact same risks from the same trial as an “Absolute Risk Reduction“, suddenly they look a bit less exciting. On placebo, your risk of a heart attack in the trial was 0.37 events per 100 person years, and if you were taking rosuvastatin, it fell to 0.17 events per 100 person years. 0.37 to 0.17. Woohoo. And you have to take a pill every day. And it might have side effects”.
The massive 55% reduction in cardiovascular disease is a similar story and becomes a lot less spectacular when expressed as an absolute risk. Elsewhere in the documentary there is another mention of a relative risk. Eating processed meats every day increases risk of colorectal cancer by 17%. The absolute risk is much smaller and although any increase in risk is of course undesired, it shows how the presentation of information can be misleading.
After the statement “People who get their protein from plant sources reduce their risk of cardiovascular disease by 55% !” the documentary continues: “It would also help explain why the only diet that has ever been shown to reverse heart disease is a plant based one”. Another big claim that cannot be supported because this study we are referring to here, used a lifestyle intervention. This lifestyle intervention did include a low fat, vegetarian diet, but it also included stopping smoking, stress management training and moderate exercise). All of these lifestyle interventions together caused the observed effects, but it does not mean that any of the individual interventions would have any of this effect.
Bias
Bias comes in many shapes and forms. In fact, in research methods classes we usually learn about at least 8-10 different types of bias . Our brain is susceptible to bias in our attempt to make the world around us easier to understand. But it is also dangerous and it can hide the truth. We need to understand how to recognise bias. The documentary warns us about bias! Studies funded by industry can be biased!
Indeed if a study is funded by industry it is important that this is mentioned in the paper, not because it means the study is wrong, but because it makes the reader aware that there is a greater chance of bias. And bias is everywhere, even in science where enormous efforts go into reducing all forms of bias.
Our brain is susceptible to bias in our attempt to make the world around us easier to understand.
But is there any potential bias in the documentary? For sure there is and lots of it! If we follow the money it becomes evident very quickly (James Cameron, Executive Producer – award winning film maker (Titanic, Avatar), story teller, and founder and CEO of Verdiant Foods, an organic pea protein company with the goal of becoming “the largest pea protein fractionation facility in North America.” Almost all of the medical professionals interviewed in the documentary also sell vegan products. Famous persons are interviewed who are passionate vegans ( a passionate belief will also cause bias). Not one source but many sources of (potential) bias. The documentary is not objective at all, even though it pretends to be objective by pretending to be very scientific.
Another difference between science and pseudoscience is that science is a very dry objective discussion of facts, whereas pseudoscience is all about emotion: fear, distrust, conspiracies, disbelief, anger…. None of this has anything to do with science.
Unfortunately, a TV programme without emotion would be boring and no one would watch so we need to bring emotion to bring our points across! The meat industry is making you ill! Are you angry yet at the meat industry? Or we move to Zimbabwe where we hear about hunters that want to kill rhino’s for their horn… not their (meat).. I am angry, how can they kill these beautiful animals? We see a group of hero soldiers who will stop the killers… but I wonder what does this have to do with a plant based diet?
Cruel
It becomes clear when their commander explains that he wants to protect animals (rhinos as well as cows), because killing is cruel. This is the reason why my daughter is a vegetarian. Her choice. But this is not a scientific argument, it is an emotional one. It also has nothing to do with health or performance. The documentary then moves into sustainability discussion. For me this is perhaps the best reason, to eat more plant based. But I will be the first to admit that I am less familiar with this literature and therefore can’t judge how strong the evidence is for the arguments made here.
Cholesterol of firefighters
The documentary shifts to a group of firefighters, many with high cholesterol. They are told how bad meat is and how good vegetables. They go away, come back a week or so later and guess what? Their cholesterol is better, and their LDL improved. Conclusion: a plant based diet fixes your health! But could it not simply be the fact that they have increased their intake of fruits and vegetables? Is this proof that meat is bad? The talking is done by a person with the word KALE on his T-shirt… perhaps giving away a slight bias already?
Then the documentary finishes up with showing an impressive number of high profile athletes and celebrities who are vegetarians and have obviously done well. One of them is Arnold Schwarzenegger famous for his “you hit like a vegetarian” quote. I actually met Arnold Schwarzenegger earlier in the year in Brazil when I was speaking a nutrition conference. He had a few nutrition tips for the audience that were pretty basic (you need to put the right stuff in your body) but his message in this documentary was quite sensible: Once a week, just chill it with the meat… He did not say “drop the meat”!
So what should we eat?
The bottomline of my analysis of this documentary is not that we should eat meat or we should eat vegetarian. My main point is that we should stop thinking black and white, good or bad, meat of veg! high fat or high carb. Antioxidants are good, inflammation is bad.
Iceberg lettuce is good, salmon and eggs are bad. This dichotomous thinking is extremely unhelpful and not healthy. Healthy eating is about balance and eating for performance is making sure that the diet provides the nutrients for your specific goals. You can achieve your goals by eating a wide variety of foods.
We should also stop the pseudoscience and pretending to be science based when clearly, there is cherry picking, non scientific language, non scientific methods and really obvious sources of bias without an attempt to reduce this. The documentary, which was undoubtedly made with the best intentions, warms for this sort of confusion causing information, but then falls in the trap of doing exactly that!
Read also:
References
Losch S, Moghaddam N, Grossschmidt K, et al. Stable isotope and trace element studies on gladiators and contemporary Romans from Ephesus (Turkey, 2nd and 3rd Ct. AD)--mplications for differences in diet. PloS one 2014;9(10):e110489. doi: 10.1371/journal.pone.0110489 [published Online First: 2014/10/22]
Tang JE, Moore DR, Kujbida GW, et al. Ingestion of whey hydrolysate, casein, or soy protein isolate: effects on mixed muscle protein synthesis at rest and following resistance exercise in young men. J Appl Physiol (1985) 2009;107(3):987-92. doi: 10.1152/japplphysiol.00076.2009 [published Online First: 2009/07/11]
van Vliet S, Burd NA, van Loon LJ. The Skeletal Muscle Anabolic Response to Plant- versus Animal-Based Protein Consumption. J Nutr 2015;145(9):1981-91. doi: 10.3945/jn.114.204305 [published Online First: 2015/08/01]
Yang Y, Churchward-Venne TA, Burd NA, et al. Myofibrillar protein synthesis following ingestion of soy protein isolate at rest and after resistance exercise in elderly men. Nutr Metab (Lond) 2012;9(1):57. doi: 10.1186/1743-7075-9-57 [published Online First: 2012/06/16]
Wilkinson SB, Tarnopolsky MA, Macdonald MJ, et al. Consumption of fluid skim milk promotes greater muscle protein accretion after resistance exercise than does consumption of an isonitrogenous and isoenergetic soy-protein beverage. Am J Clin Nutr 2007;85(4):1031-40. doi: 85/4/1031 [pii] [published Online First: 2007/04/07]
Phillips SM. Nutrient-rich meat proteins in offsetting age-related muscle loss. Meat Sci 2012;92(3):174-8. doi: 10.1016/j.meatsci.2012.04.027 [published Online First: 2012/05/29]
