The Skeptics Society & Skeptic magazine

Science Education is
No Guarantee of Skepticism

by Richard Walker, Steven J. Hoekstra,
and Rodnet J. Vogl

Many skeptics take a measured amount of pleasure in the kinds of tasks often set before them: evaluating blurry photographs, conducting laboratory experiments that reduce or eliminate trickery, critiquing flawed science and pseudoscience, and countering the claims of obvious charlatans. Of course, skeptics hope that their efforts aid in advancing science education.¹ In spite of these efforts, survey data from several sources suggests that paranormal belief and pseudoscientific thinking continue to be commonplace.²

Skeptics often use these findings to reinforce arguments for more science education. Their argument is based upon the largely untested assumption that increased science knowledge reduces the number of paranormal beliefs an individual holds. However, this assumption may not be valid. Andrew Ede recently argued that science education may do little to raise the level of rational thinking and may, in fact, actually deter it!³ Recent debates about including creation science and/or eliminating evolution from high school biology curricula⁴ are a case in point indicating that many policy makers, members of the public, and a few educators are confused about how to critique and compare theories in order to separate facts from beliefs. Ede identified three reasons why this may be true:⁵

1. Science classes, broadly defined, primarily teach technical skills rather than emphasizing critical thinking. Labs are conducted in which there is a “right answer” that the instructor knows, and it is up to the student to manipulate the project until the “right answer” is realized.
2. Science classes typically review research findings without placing the research in the proper context. This can lead to incorrect assumptions or overgeneralizations.
3. Science implicitly emphasizes its elite status over other points of view. Therefore, data and graphs are accepted uncritically because they are based on “scientific,” “clinical,” or “laboratory” studies. A lab coat guarantees an aura of expertise.

The overall result is that teaching scientific “facts” is emphasized, while individuals are not given the skills with which to critically evaluate the claims that are presented to them. People are placed in the position of accepting or rejecting claims based on what they are told to believe, rather than being able to critically evaluate the evidence.⁶

A quick inspection of introductory college textbooks supports Ede’s basic arguments. As an example, most introductory psychology texts are now in excess of 500 pages, yet fewer than 15 pages are typically spent on research issues. Little or no discussion is given to the importance of evidence or how scientific methods can be used to weigh evidence. Instead, the primary emphasis of many texts is to enumerate as many scientific findings as possible. Since it is reasonable to suspect that many instructors follow the basic format of the text that has been selected for class, it is likely that class lectures spend more time on specific research findings than on the more abstract topics of empiricism and skepticism. Hence, it is possible for a student to accumulate a fairly sizable science knowledge base without learning how to properly distinguish between reputable science and pseudoscience. Fortunately, there is recently a stronger push in introductory psychology texts to correct this oversight, most strikingly by Carole Wade and Carol Tavris,⁷ but it still remains the exception to the rule.

Assessing Science Knowledge and Pseudoscientific Beliefs

The primary goal of this paper is to examine the relationship between science knowledge and paranormal beliefs. We reasoned that if Ede’s argument is true, then a person’s scientific knowledge base should be unrelated to pseudoscientific beliefs. If, on the other hand, science leads to skepticism about pseudoscience, science knowledge and paranormal beliefs should be inversely related.

We tested this relationship using survey methodology at three small undergraduate universities (Christian Brothers University, Kansas Wesleyan University, and Winston-Salem State University). Across our samples, a total of 207 students were surveyed (66 at CBU, 70 at KWU, and 71 at WSSU). While the precise wording of the questions varied somewhat across institutions, each survey contained two essential units, administered in counterbalanced order.

In one unit, students were given one or more measures of science knowledge. Efforts were made to select scales from nationally recognized tests and to include questions from many areas of science, including biology, chemistry, geology, and astronomy. At CBU and WSSU, we used randomly sampled science questions from practice test banks for the Praxis Series National Teacher’s Exam. At KWU, we used items selected from the General Social Survey and a self-constructed measure of science values.⁸ At least two test versions were used at each university (WSSU used four separate test versions). Sample questions from the Praxis Series National Teacher’s Exam⁹ included:

1. Which of the following is the dominant source of all or nearly all of the Earth’s energy? (A) Plants, (B) Animals, (C) Coal, (D) Oil, (E) The Sun
2. Which of the following is true? (A) Energy may be converted from one form to another, (B) Energy may not be converted from one form to another, (C) The energy that a moving object possesses because of its motion is correctly known as potential energy, (D) Objects which possess energy because of their position are said to have kinetic energy, (E) Most scientists readily agree that energy from nuclear fission will be the chief source of energy by the year 2005.
3. Which of the following situations might cause harm to an embryo? (A) The father is RH-positive; the mother, RH-negative, (B) The mother had German measles during the first trimester of pregnancy, (C) The father is RH-negative; the mother, RH-positive, (D) A and B only, (E) B and C only.
4. Heavy infections of Trichinella in people may cause a disease called trichinosis; such a situation may best be described as which of the following? (A) Parasitism, (B) Mutualism, (C) Commercialism, (D) Benevolent, (E) Benign.
5. Which of the following is the main difference between an organic and an inorganic compound? (A) The former is a living compound, while the latter is a nonliving compound, (B) There are many more of the latter than of the former, (C) The latter can be synthesized only by living organisms, (D) The latter can be synthesized only by nonliving organisms, (E) The former are those that contain carbon.
6. On the periodic table the symbol Pb represents which of the following? (A) Iron, (B) Phosphorus, (C) Lead, (D) Plutonium, (E) Potassium.
7. Which of the metric terms is closest to the measurement of a new piece of chalk? (A) Meter, (B) Liter, (C) Gram, (D) Decimeter, (E) Kilometer.
8. Which of the following is a genetic disorder? (A) Down’s Syndrome, (B) Syphilis, (C) Malaria, (D) Leukemia, (E) Emphysema.
9. A litmus test conducted on HCl would have which of the following results? (A) There is no effect on the color of the litmus paper, (B) The litmus paper disintegrates, (C) The litmus paper turns blue, (D) The litmus paper turns red, (E) The carbonation causes oxygen.
10. When is the Earth closest to the Sun? (A) During the summer, (B) During the fall, (C) During the winter, (D) During the spring, (E) During the spring and summer.

In a second unit, students were asked to rate how much they believed in various paranormal and pseudoscientific claims. Again, efforts were made to select a cross-section of pseudoscientific claims. As skeptics, we found writing unbiased items to be a fairly difficult task, but the scale presented below, used in various forms at KWU, CBU, and WSSU, had acceptable reliability.¹⁰ Questions included:

Please rate how much you believe the following statements. Use the 7-point scale provided.

1=I do not believe in this at all; 2=I doubt very much that this is real; 3=I doubt that this is real; 4=I am unsure if this is real or not; 5=I believe this may be real; 6=I believe this is real; 7=I strongly believe this is real

• A person’s personality can be easily predicted by their handwriting.
• A person can use their mind to see the future or read other people’s thoughts.
• A person’s astrological sign can predict a person’s personality and their future.
• An ape-like mammal, sometimes called Bigfoot, roams the forests of America.
• The body can be healed by placing magnets on to the skin near injured areas.
• Healing can be promoted by placing a wax candle in your ear and lighting it.
• A dinosaur, sometimes called the Loch Ness Monster, lives in a Scottish lake.
• Sending chain letters can bring you good luck.
• The government is hiding evidence of alien visitors at places such as Area 51.
• Voodoo curses are real and have been known to kill people.
• A broken mirror can bring you bad luck for many years.
• Houses can be haunted by the spirits of people who have died in tragic ways.
• Water can be accurately detected by people using “Y”-shaped tree branches.
• Animals, such as cats and dogs, are sensitive to the presence of ghosts.

Science Test Scores

After scoring this portion of the survey, we used the total number of correct responses as data for each participant. While the scores did show a fair degree of variability, the averages were generally near the midpoint (CBU M=7.42, SD=2.27, out of a possible 15; KWU M=3.52, SD=.33, on a 1 to 5 scale; WSSU M=5.8, SD = 1.96, out of a possible 10).¹¹

Paranormal / Pseudoscientific Beliefs

Unlike previous studies that have examined specific beliefs (e.g., UFOs, ESP), we were interested in these beliefs as a whole. We wanted a single representative score for each participant. We calculated the average belief score for each participant, with higher ratings indicating greater overall belief. When looking at these scores, remember that the rating ranged from “No belief” to “Total belief.” Looking across the three samples, the average belief rating was at or below the midpoint of the scale (CBU M=1.96, SD=.76, on a 7- point scale; KWU M=2.52, SD=.58, on a 5-point scale; WSSU M=2.9, SD=.91, on a 7-point scale), although there were individuals ranging from complete skeptics to complete believers.

Science Test Scores and Paranormal / Pseudoscientific Beliefs

We were interested in whether science test scores were correlated with paranormal beliefs. For each sample, we correlated the participant’s test score with their average belief score. Across all three samples, the correlation between test scores and beliefs was non-significant (CBU r(65)=-.136, p>.05; KWU r(69)=.107, p>.05; WSSU r(70)=.031, p>.05). In other words, there was no relationship between the level of science knowledge and skepticism regarding paranormal claims.

Wanted: A Good Baloney Detector!

These results are consistent with the notion that having a strong scientific knowledge base is not enough to insulate a person against irrational beliefs. Students who scored well on these tests were no more or less skeptical of pseudoscientific claims than students who scored very poorly. Apparently, the students were not able to apply their scientific knowledge to evaluate these pseudoscientific claims. We suggest that this inability stems in part from the way that science is traditionally presented to students: Students are taught what to think but not how to think.

These results need to be replicated using different materials and participants, although the diversity of measures and samples presented here suggests that there is some validity to our conclusions. While some might contend that our tests did not fully measure science knowledge, we counter this concern by emphasizing that our test questions were drawn from national tests designed to assess scientific reasoning. Thus, if there is a bias in our procedure, this bias is entrenched in science education. In our view, addressing the following questions can serve to clarify the relation between science education and pseudoscientific thinking.

First, do pseudoscientific beliefs vary by academic major? If science does discourage such beliefs, then one might suspect that science majors should view these beliefs more skeptically than perhaps religion or art majors. While there is evidence that scientists are more skeptical of claims than other individuals, it is unclear whether this skepticism is readily transferred to undergraduate science majors. After all, strange beliefs seem to be found among students of many disciplines. Unfortunately, our access to samples was too small to adequately test differences between majors.

Second, do pseudoscientific beliefs vary by education level? As students advance through college and gain experience and critical thinking skills, one might expect pseudoscientific beliefs to decrease. Although our data did not address this issue, other studies on skepticism suggest that an individual’s education level may not ward off such beliefs.¹²

Finally, can academic courses or programs that systematically raise or lower belief in pseudoscience be identified? It is possible that particular courses may encourage or discourage pseudoscientific beliefs. These courses need to be identified, and in the case of the latter, the key elements of these courses need to be disseminated to other science instructors. This is especially important when classroom information can support or contradict information from other sources, such as mass media. Our results suggest that we should not be overly optimistic, but more systematic investigations are needed.

Baloney Detection Kit
by Michael Shermer and Pat Linse

This 16-page booklet is designed to hone your critical thinking skills. It includes suggestions on what questions to ask, what traps to avoid, specific examples of how the scientific method is used to test pseudoscience and paranormal claims, and a how-to guide for developing a class in critical thinking.
ORDER the 16-page booklet

We hope that our findings force fellow skeptics to rethink some of their assumptions. Science education, in its current form, seems to do little to offset pseudoscientific beliefs, and may in fact give students reason to accept science fiction as science fact. As skeptics and teachers, we need to do more than merely debunk extraordinary claims. While these demonstrations are informative and entertaining, they need to be coupled with thoughtful discussions of scientific reasoning. Carl Sagan suggested that good scientific reasoning demands the same type of skepticism that is needed to buy a good used car. In short, he said that students of science need a good baloney detector.¹³ We agree. The Skeptics Society’s recent publication of the Baloney Detection Kit that instructs teachers on how to teach just such a course is a step in the right direction.¹⁴ Provisional scientific truth must be separated clearly from myth. We urge skeptics to help students gain the necessary skills to make such distinctions both inside and outside the classroom.

ACKNOWLEDGMENTS: We would like to thank the undergraduate research assistants who helped collect and tabulate the data on this research project: Reggie Andrews, Paul Delph, Stefanie McGee, and Lakisha Pinson. Correspondence concerning this article should be directed to W. Richard Walker, Department of Social Sciences, Winston-Salem State University, Winston-Salem, NC 27110

Watch The Baloney Detection Kit Video on YouTube