Should you trust medical research?
I keep reminding myself on a daily basis that I should ignore news headlines about any kind of medical advice. Every day we are bombarded with all kinds of trivial advice on how to “improve” our health and live longer. I am a scientist and I am confronted on a regular basis with fairly complex systems that need to be analyzed. I have learned over the years how difficult it is to draw any reliable conclusions about even the simplest systems, especially in the absence of diverse and extensive datasets.
Compared to the systems I am working with, the human body is a vastly more complicated machinery. My intuition tells me that no study can draw any reliable conclusions, no matter how many individuals it follows and for how long. There are so many parameters that are impossible to control that any reasoning will be flawed (the variability of diet and lifestyle, and the inherent unreliability of subjects are two examples).
On a yearly basis I remember reading certain advice that was refuted only months afterwards. An anecdotal example was evidence that eating more white chocolate has positive effects for your cholesterol levels; at the same time drinking whole milk and eating a lot of sugar is bad for your cholesterol… Hence, I learned not to heed any advice, even if it comes from respected medical professionals. I’d rather heed my own common sense. I eat white chocolate every once in a while.
Thankfully, the other day I came upon a very interesting article in The Atlantic, titled “Lies, Damned Lies, and Medical Science” that scientifically reinforces my belief. The article talks about Dr. John Ioannidis who is on a mission to quantify how many medical studies have been already refuted. Quoting from the article:
He zoomed in on 49 of the most highly regarded research findings in medicine over the previous 13 years, as judged by the science community’s two standard measures: the papers had appeared in the journals most widely cited in research articles, and the 49 articles themselves were the most widely cited articles in these journals. … Of the 49 articles, 45 claimed to have uncovered effective interventions. Thirty-four of these claims had been retested, and 14 of these, or 41 percent, had been convincingly shown to be wrong or significantly exaggerated. If between a third and a half of the most acclaimed research in medicine was proving untrustworthy, the scope and impact of the problem were undeniable.
It turns out that scientist cannot agree even on claims based on common sense. For example, longitudinal studies about the effect of obesity on longevity, as it turns out, are not conclusive. Apparently, slightly overweight people live longer. This reinforces my point that no conclusions are possible. The experimental setup itself is chaotic and impossible to control. Now, take into account confirmation bias and the financial conflicts of interest and it is no wonder that Dr. Ioannidis has a very simple advice on how to choose which of these findings to take seriously: “Ignore them all”. I certainly do…