Our Science Teachers Suck
Qualifier- I taught a science (psychology) for over a decade and trained to be an experimental psychologist. And I believe we’re failing our youth.
Why?
We suck at teaching science.
If we succeeded, our society would be adept at understanding the nature of the scientific method. We would understand that any one line of research is inherently limited by the size of the subject pool, methodological limitations, and experimental biases. We would understand that a study with a sample size of 10 college sophomores cannot be generalized to the entire world population.
Yet that’s exactly the opposite of what we do.
As runners (or consumers), we like to buy into the idea that the latest line of research is THE answer that will help us attain greatness. We rarely if ever consider the possibility that the latest research may not apply to us as an individual. Worse, we don’t consider the possibility that the research could be wrong. We treat scientific discovery with the same fervor and blind allegiance usually reserved for religion.
Here’s a typical scenario: Some researchers conduct a study to test if eating kitty litter enhances sexual performance and get mildly positive results. They publish it in a peer-reviewed journal. The popular media picks up on the research and reports the findings in newspapers, on TV, and across the Interwebs. The reporters conveniently leave out the part of the conclusion where the researchers note the limitations of the study, warn against overgeneralization, suggest their measurement tools may not be reliable or valid, and suggest future research that will improve the tests on their hypotheses.
Joe public reads the reporter’s account. Immediately kitty litter is flying off the shelves of the local A&P. Pundits begin debating the best method to ingest kitty litter. Bloggers begin reviewing various brands of kitty litter. Grassroots groups pop up to host kitty litter parties. Kitty litter is touted as a miracle “marital aid”. Fresh Step stock prices soar. Congress holds hearings to set up a regulatory commission to tax kitty litter.
One thing is noticeably absent: critical skepticism. Nobody seemingly considers the possibility that the kitty litter research could be flawed, even if the original researchers warned about the limitations of the conclusion.
This is the way we treat science- as a series of undeniable truths that stand until another new study comes along to replace it. We seem to place unwavering trust in single studies that are clearly limited in scope. This makes us susceptible to repeatedly jumping from one bandwagon to the next without ever questioning the logic behind our decisions.
Unfortunately, that’s not how science works. Science is based on the idea that we’re working toward understanding the world around us using direct observation in a systematic manner. We observe, come us with a guess about how something works (what we call a hypothesis), then test our guess using an experiment that tries to isolate the variable we’re testing. Based on the outcome of the experiment, we’re either right or wrong. If we’re right, we test it again with a different subject pool or different methods. If we’re wrong, we go back to the drawing board.
There’s one critical element to this- we never find what would be considered the truth (or scientific fact, if you will). Science doesn’t prove anything. Simply put, all hypotheses must be falsifiable. The experiment has to be designed in a way that will prove the hypothesis wrong. If a hypothesis isn’t falsifiable, it’s not a hypothesis that can studied with science. If you hear someone say “This is scientific fact” or “Science proves this is true”, they’re an idiot that doesn’t really understand how science works.
So what happens if we do a lot of experiments and they all turn out the same way? Doesn’t that mean it’s a scientific fact?
NO! Science is limited to that which can be measured. At some point in the future, we may develop a tool that is better at measuring a hypothesis and will net different results. Ask Copernicus about this idea. Or the researchers that recently measured matter traveling faster than the speed of light. Anything we discover under the guise of science can be wrong. If it can’t be wrong, it’s not science.
Ben Heinrich, in his excellent book “Why We Run“, perfectly explains how this idea relates to runners. Heinrich is a biologist by trade and used his knowledge of the animal kingdom to maximize his training in an attempt to win the 1981 100k National Championship.
“We live in a biological world of conflicting truths that together create the ever changing new out of the ageless. Our world is not a linear-logical construct that yields truths through ad infinitum extrapolations by the use of scientific tools such as mathematics. That world, the one of unbending physical truths, exists in theory, but theory tends to be just an academic exercise when it has to compete with the reality of existence, or the real world of biology that we inhabit- the one that is both incredibly finely structured and chaotic. There is no precise formula that specifies how to prepare. There are only approximations, and best approximations, until something better comes along. Like infinity, the ultimate truth can be approached but never attained.”
Our problem- we believe what we know right now is the ultimate truth. And it lasts a short period of time until the next ultimate truth comes along.
“The people who told us about sun block were the same people who told us, when I was a kid, that eggs were good. So I ate a lot of eggs. Ten years later they said they were bad. I went, “Well, I just ate the eggs!” So I stopped eating eggs, and ten years later they said they were good again! Well, then I ate twice as many, and then they said they were bad. Well, now I’m really fucked! Then they said they’re good, they’re bad, they’re good, the whites are good, th-the yellows – make up your mind! It’s breakfast I’ve gotta eat!”
-Lewis Black
Two things are noticeably absent:
1. Skepticism. We don’t actually consider the limitations or context of the research we consume. We also don’t consider if the results can reasonably be generalized to us as an individual. Sadly, almost all researchers clearly state both of these limitations in their conclusions. Though rare, some researchers purposely alter their data. Be careful when consuming scientific research. Even barefoot running should be viewed with a degree of skepticism.
2. Self-experimentation. We rarely do empirical self-testing to determine if something is actually helpful… we just do it. Instead, we should test any new idea against what we already do.
So how should you use scientific research?
First, try to read the actual text of the published research. If you’re not into that, at least read the abstract (summary usually available via Google Scholar) and, if possible, the conclusion. Use the findings of the research as a resource for ideas to fuel your won self-experimentation. Let’s say some research indicates holding your water bottles upside down increases running efficiency by 3%. If you want to see if the idea works for you, try it on your next run. Write down how you felt during the run. Next, run the way you would normally and record the results. Try each condition a few times. If it seems to work, keep doing it. If it doesn’t, go back to the way you did it before. Go with what works better for you. This can be used to test running form, diet, pacing strategies… whatever.
For those science teachers that actually teach about the importance of skepticism- GREAT JOB! I used to work with a few, and have some friends that attempt to do exactly what I described here. For those science teachers that teach science as if it were a set of undeniable facts- shame on you. You’re doing us a disservice and bastardizing the very idea of empirical inquiry. And you’re making people eat kitty litter.
###
33 Comments
athensguy
December 9, 2011As long as people are taught from day one by their religious parents that science is evil and wrong, it doesn’t matter much how good their science teachers are. I think, though cannot prove, that ingrained ideas, regardless of their correctness, are much harder to relinquish if acquired during the first 7 or 8 years of life.
corey
December 2, 2011Journalists are to blame not science teachers. There are very few real journalists left, especially know that the newspapers are in trouble financially. The problem is people still trust the news, even if they probably shouldn’t. But in the end it is the failure of our media to step up and do a better job of reporting on things that matter.
Also, you mention that people should read the abstracts from the actual journal articles. Yeah, right Jason; who the hell would want to wade through that quagmire of sterile english writing. My god, it takes a few weeks to get a handle on ones own research speciality, and you expect the average citizen to do that every time the results of study are reported, sheesh
Erik
November 30, 2011Most scientific results published in popular media are based on observational studies, not double-blind testing. Moreover, Newtonian physics are wrong yet still work fine if you want to build a bridge. Finally, Noam Chomsky is completely wrong about language, yet he is by far the most cited living scientist. I conclude that barefoot running is good for me.
John
November 29, 2011Science exists to be tested, its rules made to be broken. It is not staid, it should never be construed as such. However, publish it in a…”Journal” or “Paper” or “Controlled Study” and suddenly all of the lemmings are cliff bound, and all the Pavlovian canines start to drool. I went to college to become a teacher until I realized that “Teaching” and “Science” were not as were originally published in my collegiate syllabi. Teaching is an artform, an organic and creative process (or lifestyle) that really reaches out to a select few that truly get it. Sciences (Earth, Biological, Chemical, et al) are all of life’s collective parts scattered about hither and yon…and the right teacher(s) can actually make sense out of them and enlighten even the most critical young mind(s). I do not see the artform flourishing, I see it dying. With the interwebs science will be nothing more than an endless collection of digitalized “Controlled Studies” that will sound the charge for a new collective of lemmings. I think I hear the thundering of their furry paws now…
Linkscoach
November 29, 2011I’ve been a teacher (HS and MS Technology Education) for 14 years. My science teaching buddy has been teaching for over 30 years. We couldn’t agree more with your post.
Most of the problems with teaching in general have nothing to do with the teachers. It’s not the kids or the parents either. Ask a teacher in any field to tell you who decides what they teach and why they teach it. The answers may surprise you.
The public education system isn’t broken. That doesn’t mean that it works as well as it should either.
I guess the point of your post Jason was to encourage people to question the “experts”. The only test anyone needs is to listen to their own bodies and determine what works best.
Joe Wrigley
November 29, 2011I’m mostly amused that you consider psychology to be a science.
Harry
November 29, 2011Did you say Kitty litter will help me when I get jiggy?? Well what flavour works best? Oh sorry did I just prove your theory?
I just do/buy/inbibe what ever the mighty Jason recommends ( I am too lazy to do my own research )
I also believe that the world is flat
Chaser
November 29, 2011What exactly was the sample size of your study here? Eh? lol.
kelly
November 29, 2011I thought this was a post about how the media (one author in particular) has sensationalized running without shoes, and that there are no proven facts that shoes prevent or cause injuries. So do what works for you, barefoot or hoka ones.
Jason
November 29, 2011Talking about that Ken Bob guy?
David Sutherland
November 29, 2011Climate theory? Evolution vs. Creation? We’re getting into some touchy subjects here. Your next post should be about the fact that Obama is a great president despite him clearly being both an evangelical Christian radical and a Muslim extremist at the same time. Chew on that, Schrodinger!
You’re right on point about the need for skepticism and falsifiableness (is that a word?) in scientific theory. In explaining Einstein’s theory of gravity to my kids, I proposed the alternate theory that what we feel as gravity is really just space monkeys pushing down on us. It’s a good theory – it makes predictions, it’s testable.
Enough rambling. I gotta go to the store and pick up some kitty litter.
Jason
November 29, 2011Note- I purposely avoided those two topics.
And yes, falsifiableness is indeed a word. In science, you can add as many suffixes as needed.
Space monkeys generating gravity. Hmmmm… I think I have the topic for my next book. Are the monkeys invisible and personally pushing us down, or do they employ some sort of reverse tractor-beam technology?
Wiglaf
November 29, 2011Bingo, David! They’re touchy subjects because “science” has become a tool of politicians to force public policy down people’s throats and money out of their pocket books. Science is used, abused, and tossed aside. It’s no surprise that kids these days don’t have respect for science when the supposed adults leading our country use science like a cheap whore. Just wait ’til we start talking about childhood obesity, government nutrition standards, whether or not pizza is a vegetable, and all the supposed “scientific” studies done to prove this or that on nutrition.
Conventional wisdom currently says eggs are bad, right?
Steve
November 29, 2011This is pretty cool. One of my favorite science bloggers is working on a project to help separate the good science from the chaff. Unfortunately he doesn’t get into naming the project or anything in the post.
http://scienceblogs.com/startswithabang/2011/11/a_new_challenge_a_new_job_and.php
Jen
November 29, 2011I 100% agree with JS’s comment above. I think the key here is not to only encourage skepticism, but scientific literacy in general. As Jason noted above, the general population has a limited understanding of how science “works” and how to interpret results. For example, the “debate” around climate change. Scientists (good ones at least) will not say that climate change is a definite fact, but they will say that all signs support the fact that climate change is occurring. However, people take this as less than 100% certainty and say, “well, they say it’s not a fact and that it can’t be proven.” Then they deny that climate change is occurring, which then affects public policy.
I think we should teach logic and analytical thinking in science classes. I wasn’t taught such things until I went to get my Ph.D. in biology! We need to have a population that is able to think critically.
Jason
November 29, 2011I agree, Jen. What it really comes down to is a lack of scientific literacy and a basic understanding of how science works. I think the popular media is more to blame than science teachers as they often sensationalize results by overgeneralizing.
Wiglaf
November 29, 2011Yet the likes of Phil Jones and Michael Mann would be happy to see the media sensationalize it. The problem here, is that the bad scientists are the ones with the primary influence and support in the IPCC. Then there’s this (who’s doing the science and who’s making it up on the fly??):
http://wattsupwiththat.com/2011/10/18/replicating-al-gores-climate-101-video-experiment-shows-that-his-high-school-physics-could-never-work-as-advertised/
Wiglaf
November 29, 2011Jen, when you say “all signs support the fact”, what do you mean? Are you saying that climate change is occurring and is fact? What does that mean? Climate is always changing. When you say “people take this”, to which people are you referring? You surely are not referring to skeptical scientists who have repeatedly pointed out the errors in the hockey stick graph, the errors in the statistical adjustment methods, the general bias of the warmists (see climategate emails), the ridiculous assertions of the bad warmists like Al Gore, the unproven assertion that man contributes to global warming, etc and etc. Have you read the skeptical scientists position like those of Anthony Watts, Pielke, Judith Curry, Steve Mosher, Bishop Hill, Tim Ball, and many others? Don’t fall for the global warming religion and don’t call skeptics deniers. Thank you.
Wiglaf
November 29, 2011One more thing, how is science supposed to work when the scientists won’t release the data and argue that, as Dr. Phil Jones stated, “Why should I make the data available to you, when your aim is to try and find something wrong with it?” If that doesn’t make you mad, as a scientist, because of the bias and an attempt to circumvent the scientific method, then I don’t know what would. More here: http://wattsupwiththat.com/2011/11/27/an-open-letter-to-dr-phil-jones-of-the-uea-cru/
Matt
November 29, 2011Good article, but you are focusing too specifically on science. It is not our methods of teaching science that are failing, it is our methods of teaching anything. It is the rare teacher, especially among those teaching our children (I.e. elementary through high school) who teaches his or her students to think critically. It’s this absence of thinking critically, about anything, that causes the kinds of reactions you describe.
briderdt
November 29, 2011Of course, this all ignores the difference between “fact” and “truth”.
But we are all experiments of one. The only sample size that really matters.
Jason
November 29, 2011In science, there is no “truth”. That enters the realm of belief systems, which by definition, cannot be falsified. That which cannot be falsified cannot be empirically measured.
But I do agree, “n= ourselves” is what really matters.
Wiglaf
November 29, 2011I just wish I could figure out how to use myself as a part of my own double blind study.
OreMan
November 29, 2011You are perhaps mostly true about that nothing can be perceived as an ultimate truth in some fields of science. I guess, in psychology, social sciences, even physics – `facts` of those sciences are changing with time.
However, then there is the Mathematics – the science where it is possible to define precise theorems that are to be true FOREVER. And there are indeed scientific facts. For example, in any right angled triangle the square of the hypotenuse is equal to the sum of the squares of both legs of the triangle, as it is said in the Pythagorean theorem. It has always been so, it is so now and it will always be so, no matter what. This can be called the general truth. Noone can design an experiment that falsifies this `hypothesis`.
So perhaps only our science-that-is-not-mathematics teachers suck..
Jason
November 29, 2011I knew someone would bring up mathematics.
A person smarter than me once explained math as an entirely theoretical field created by man. It’s precise because we designed it that way. Math can even be used to explain many naturally-occurring phenomena, but not all. For example, I’m pretty sure math cannot predict the movement of liquid through a pipe (is that chaos theory?)
Anyway, this discussion is getting too sciency for my small brain.
JS
November 29, 2011I mostly agree with you except when you say: ” If you hear someone say “This is scientific fact” or “Science proves this is true”, they’re an idiot that doesn’t really understand how science works.”
Scientific facts do exist, they just don’t mean what some people think. I’ll let Stephen Jay Gould explain it: “Evolution is a theory. It is also a fact. And facts and theories are different things, not rungs in a hierarchy of increasing certainty. Facts are the world’s data. Theories are structures of ideas that explain and interpret facts. Facts do not go away when scientists debate rival theories to explain them. Einstein’s theory of gravitation replaced Newton’s, but apples did not suspend themselves in mid-air, pending the outcome.”
Jason
November 29, 2011Ah, but even scientific facts must be falsifiable, which a good scientist understands. A fact is an empirical observation. How we make an observation is subject to our own limitations of measurement. Evolution, while widely supported and accepted as a “scientific fact” may indeed be wrong. As dumb as the idea is, the creation science folks may be right. Granted, the probability may only be 1 in 10 trillion chance of that, but there’s a chance. Based on the definition of falsibility, we have to acknowledge that.
The problem lies in the popular definition of “fact”. The public recognizes “fact” as an undeniable truth, whereas a scientist would define a “fact” as something that is accepted to be true. It’s a small but important difference that affects the degree of faith the public places in any single study. Ergo our lack of skepticism, which ironically is the very foundation of scientific inquiry.
Wiglaf
November 29, 2011In addition, some aspects of the theory of evolution are observable; commonly referred to as micro-evolution. Other aspects are not observable. To call “evolution” a fact or even a theory would be lumping a whole lot together; some of which, cannot even be tested as of yet.
JS
November 29, 2011My main point is that if someone says “this is a scientific fact”, I don’t think they are an necessarily idiot…
If I say that evolution does occur and that apples fall off trees and that those are facts. Am I an idiot?
Keeping in mind that when I say evolution, I’m referring to the change in allele frequency in a population over time. Not how it happens or where humans came from.
Jason
November 29, 2011Agreed, and that was a bit of hyperbole on my part. Evolution in regards to change in allele frequency and apples falling from trees does occur- at least to the best of our ability to measure such phenomenon. However, we have to concede that there’s a possibility that neither may actually occur based on our limits of observation today. Facts are facts only in that they are based on our current level of understanding.
Having said that, I have encountered scientists that dismiss the possibility that evolution (in the broad sense) may be wrong. To not even consider the possibility that our most-supported theories may be wrong is simply bad science.
Wiglaf
November 29, 2011Well, that just makes you a denier, Jason.
Daniel Howell
November 29, 2011As a science teacher myself, I could not agree with you more. Excellent.
Erik
November 30, 2011Is this some kind of meta-statement?