Qualifier- I taught a science (psychology) for over a decade and trained to be an experimental psychologist. And I believe we’re failing our youth.
We suck at teaching science.
If we succeeded, our society would be adept at understanding the nature of the scientific method. We would understand that any one line of research is inherently limited by the size of the subject pool, methodological limitations, and experimental biases. We would understand that a study with a sample size of 10 college sophomores cannot be generalized to the entire world population.
Yet that’s exactly the opposite of what we do.
As runners (or consumers), we like to buy into the idea that the latest line of research is THE answer that will help us attain greatness. We rarely if ever consider the possibility that the latest research may not apply to us as an individual. Worse, we don’t consider the possibility that the research could be wrong. We treat scientific discovery with the same fervor and blind allegiance usually reserved for religion.
Here’s a typical scenario: Some researchers conduct a study to test if eating kitty litter enhances sexual performance and get mildly positive results. They publish it in a peer-reviewed journal. The popular media picks up on the research and reports the findings in newspapers, on TV, and across the Interwebs. The reporters conveniently leave out the part of the conclusion where the researchers note the limitations of the study, warn against overgeneralization, suggest their measurement tools may not be reliable or valid, and suggest future research that will improve the tests on their hypotheses.
Joe public reads the reporter’s account. Immediately kitty litter is flying off the shelves of the local A&P. Pundits begin debating the best method to ingest kitty litter. Bloggers begin reviewing various brands of kitty litter. Grassroots groups pop up to host kitty litter parties. Kitty litter is touted as a miracle “marital aid”. Fresh Step stock prices soar. Congress holds hearings to set up a regulatory commission to tax kitty litter.
One thing is noticeably absent: critical skepticism. Nobody seemingly considers the possibility that the kitty litter research could be flawed, even if the original researchers warned about the limitations of the conclusion.
This is the way we treat science- as a series of undeniable truths that stand until another new study comes along to replace it. We seem to place unwavering trust in single studies that are clearly limited in scope. This makes us susceptible to repeatedly jumping from one bandwagon to the next without ever questioning the logic behind our decisions.
Unfortunately, that’s not how science works. Science is based on the idea that we’re working toward understanding the world around us using direct observation in a systematic manner. We observe, come us with a guess about how something works (what we call a hypothesis), then test our guess using an experiment that tries to isolate the variable we’re testing. Based on the outcome of the experiment, we’re either right or wrong. If we’re right, we test it again with a different subject pool or different methods. If we’re wrong, we go back to the drawing board.
There’s one critical element to this- we never find what would be considered the truth (or scientific fact, if you will). Science doesn’t prove anything. Simply put, all hypotheses must be falsifiable. The experiment has to be designed in a way that will prove the hypothesis wrong. If a hypothesis isn’t falsifiable, it’s not a hypothesis that can studied with science. If you hear someone say “This is scientific fact” or “Science proves this is true”, they’re an idiot that doesn’t really understand how science works.
So what happens if we do a lot of experiments and they all turn out the same way? Doesn’t that mean it’s a scientific fact?
NO! Science is limited to that which can be measured. At some point in the future, we may develop a tool that is better at measuring a hypothesis and will net different results. Ask Copernicus about this idea. Or the researchers that recently measured matter traveling faster than the speed of light. Anything we discover under the guise of science can be wrong. If it can’t be wrong, it’s not science.
Ben Heinrich, in his excellent book “Why We Run“, perfectly explains how this idea relates to runners. Heinrich is a biologist by trade and used his knowledge of the animal kingdom to maximize his training in an attempt to win the 1981 100k National Championship.
“We live in a biological world of conflicting truths that together create the ever changing new out of the ageless. Our world is not a linear-logical construct that yields truths through ad infinitum extrapolations by the use of scientific tools such as mathematics. That world, the one of unbending physical truths, exists in theory, but theory tends to be just an academic exercise when it has to compete with the reality of existence, or the real world of biology that we inhabit- the one that is both incredibly finely structured and chaotic. There is no precise formula that specifies how to prepare. There are only approximations, and best approximations, until something better comes along. Like infinity, the ultimate truth can be approached but never attained.”
Our problem- we believe what we know right now is the ultimate truth. And it lasts a short period of time until the next ultimate truth comes along.
“The people who told us about sun block were the same people who told us, when I was a kid, that eggs were good. So I ate a lot of eggs. Ten years later they said they were bad. I went, “Well, I just ate the eggs!” So I stopped eating eggs, and ten years later they said they were good again! Well, then I ate twice as many, and then they said they were bad. Well, now I’m really fucked! Then they said they’re good, they’re bad, they’re good, the whites are good, th-the yellows – make up your mind! It’s breakfast I’ve gotta eat!”
Two things are noticeably absent:
1. Skepticism. We don’t actually consider the limitations or context of the research we consume. We also don’t consider if the results can reasonably be generalized to us as an individual. Sadly, almost all researchers clearly state both of these limitations in their conclusions. Though rare, some researchers purposely alter their data. Be careful when consuming scientific research. Even barefoot running should be viewed with a degree of skepticism.
2. Self-experimentation. We rarely do empirical self-testing to determine if something is actually helpful… we just do it. Instead, we should test any new idea against what we already do.
So how should you use scientific research?
First, try to read the actual text of the published research. If you’re not into that, at least read the abstract (summary usually available via Google Scholar) and, if possible, the conclusion. Use the findings of the research as a resource for ideas to fuel your won self-experimentation. Let’s say some research indicates holding your water bottles upside down increases running efficiency by 3%. If you want to see if the idea works for you, try it on your next run. Write down how you felt during the run. Next, run the way you would normally and record the results. Try each condition a few times. If it seems to work, keep doing it. If it doesn’t, go back to the way you did it before. Go with what works better for you. This can be used to test running form, diet, pacing strategies… whatever.
For those science teachers that actually teach about the importance of skepticism- GREAT JOB! I used to work with a few, and have some friends that attempt to do exactly what I described here. For those science teachers that teach science as if it were a set of undeniable facts- shame on you. You’re doing us a disservice and bastardizing the very idea of empirical inquiry. And you’re making people eat kitty litter.