The Skeptics Society & Skeptic magazine

The practice of discussing educational testing in the same sentence with the term “war” is not necessarily new or original.¹ What may be new to readers, however, is to characterize current debates involving educational testing as involving a war against: (1) accurate perceptions about the way things really are (reality), and (2) sound judgment in practical matters (common sense).

Education, Testing, and the Real World

Education is compulsory in American society, and no one escapes testing—whether standardized or unstandardized—in their schooling experience, even before entering school. As newborns, infants are given Apgar scores to assess their overall health.² When a child is ready to enter preschool, s/he may be assessed with a standardized test to determine school readiness in understanding basic concepts, cognitive and language development, and early academic achievement.

As children matriculate through the primary school years, they are required to pay attention to teacher lessons; resist natural impulses to fidget, talk out of turn, or bother one’s neighbor; complete worksheets quietly at one’s desk; complete and return homework assignments; and complete national or state-mandated standardized academic achievement tests that measure “what students know and can do.”³ In some cities, students must complete tests to determine eligibility for entrance into elite or specialty high schools,⁴ and students in some states must successfully complete tests in order to graduate high school.⁵ Well before students are scheduled to graduate, they have, until recently, been required to complete standardized college admissions tests in order for their applications to be competitive for colleges of their choice.⁶

Enter Basic Common Sense

When enough years are spent surrounded by age peers in schools, everyone—regardless of background, race, ethnicity, or socioeconomic status—intuitively understands that comparatively, some peers are intellectually smarter, other peers are roughly the same, and others are intellectually slower. These differences are most determinative of one’s overall level of academic achievement from kindergarten to high school graduation and beyond. Some pupils have a natural proclivity to be voracious readers and progress successfully through their academic programs much more quickly than others. They are able to grasp and understand difficult and abstract academic material more quickly, have a wide range of intellectual interests and hobbies, and are much more likely to be selected for admission to programs for the gifted and talented. These are generally the A and B students and tend to enroll in advanced foreign languages, trigonometry, pre-calculus, chemistry, and other advanced placement (AP) classes in high school.

Then there are students who struggle with school—particularly as the curriculum becomes more conceptual, complex, and abstract. These students have often been identified as “slow learners,” and school generally becomes a profoundly aversive experience. In higher grades, many tend to select vocational courses or may sometimes drop out of school before graduation, and these are generally known as the C and D students in their classes. The majority of pupils, however, fall somewhere in between those two extremes.⁷

When interacting with curricula, brighter students can generalize learning more easily to new classes of similar information never before encountered, while slower students have more difficulty in remembering what has been previously learned. Using a simple illustration from the early elementary school years (and barring specific reading disabilities), teachers can teach brighter students the phonetic rules for sounding out words such as “groan” and “moan.” Later, when these students encounter similar words that they have not seen before, such as “Joan” or “loan,” they can more easily apply what they have previously learned and correctly sound out these new words as well as understand their meaning. In contrast, when slower students encounter new words that have the same phonetic spelling and pronunciation as previously learned words, they find it more difficult to spontaneously apply what they have previously learned to sound out these new words, and consequently, word identification mastery takes them more time.⁸

Similarly, slower students will be easily confused over the rules that govern the correct pronunciation of words with the same “ei” letter combination but different pronunciations, e.g., neighbor, heist, and weird. In contrast, brighter students will internalize these nuances more quickly, readily identify these words correctly, and so move on to master more complex words. These differences in word identification skills also influence reading comprehension.

To be sure, slower students will eventually learn how to pronounce correctly similar words governed by different phonetic rules, but teaching such students requires instruction where broader learning objectives are broken down into smaller hierarchical steps, teaching is much more intentionally explicit, and greater amounts of time are devoted to learning and practice.⁹ If you learned academic subjects more quickly than other students, you probably have found other areas (e.g., art, music, athletics, home and auto repair, cooking, or just learning to get along with others) that took you longer than others, including those who took longer than you on the purely academic subjects.

Regardless of grade level brighter students can more quickly internalize and consolidate the required mental schemata for representing material that is learned, and then use this knowledge as a foundation upon which to build new schemata.¹⁰ Slower students have more difficulty consolidating information to be learned, or at least it requires more time to consolidate prerequisite information compared to brighter age peers. When slower peers attempt to mentally consolidate new information built on a shaky foundation, new information is poorly understood.

Brighter students can generally follow along at the pace of regular instruction, while slower students cannot, and eventually fall further and further behind as they get older. The older pupils are, the more they begin to self-select into secondary school tracks that are more suitable to their intellectual capabilities and interests, resulting in extremely wide individual differences in academic performance at higher grades. By the time students reach 11^th and 12^th grades, for example, brighter students are able to solve complex mathematical equations while slower students still struggle with mastering elementary fractions. As a result, the brightest students in high school tend to enroll in advanced placement courses such as foreign languages, pre-calculus, chemistry, and physics, while slower students gravitate to vocational courses.

Anti-testing hostility has found a powerful, organized voice whose prime directive is to diminish the influence—if not the outright banishing—of standardized testing.

Psychologists refer to this basic phenomenon as “individual differences in mental ability and learning potential,”¹¹ and no one knows this better than teachers. In the elementary grades, for example, teachers regularly come into contact with wide individual differences in performance on standardized achievement tests, despite all students being taught the same material under the same teacher. That’s why it is a bit unfair to hold teachers solely responsible for the achievement test performance of their students, since students can perform poorly on achievement tests despite exemplary teaching, and can also perform well on achievement tests despite mediocre teaching.

Enter Painful Realities

There are no racial, ethnic, language, or socioeconomic subpopulation groups, anywhere on any continent on the globe, that display equal means in their respective distributions of mental test scores.¹² These individual differences in mental test scores, when consolidated and averaged, will inevitably result in statistically significant average differences in academic achievement across subpopulation groups. Of course, there is also significant overlap among these groups. Although the full range of test scores and performance—from severe intellectual disability to mental genius—can be found within all racial and ethnic subpopulation groups,¹³ it is nevertheless true that these abilities are not equally distributed across such groups. Group differences have been observed since the beginning of standardized testing. In fact, they begin as early as three years of age, remain consistent over decades, and have proven stubbornly resistant to intervention.¹⁴ The largest gaps between subpopulation groups in both mental test scores and the achievement outcomes that result from such scores will be most noticeable at the extremes of their respective distributions.¹⁵ Because this is such a sensitive subject it should be noted that these are average differences between groups and tell us nothing about the ability of any single member of any group.

Differences in academic achievement are not equally distributed across socioeconomic groups or across communities and school districts, as these have more or less different concentrations of low to high performing students. Studies consistently show that even massive allocation of funds to school districts, without other interventions, has no significant effect on raising academic achievement.¹⁶ School systems are keenly aware of this, which is why comparisons of achievement test scores across school districts are careful to use race, ethnicity, and socioeconomic status as a covariate in comparing scores. That is, schools having similar concentrations of students from particular racial/ethnic groups and socioeconomic backgrounds are compared to other schools with similar backgrounds. This way, when schools having concentrations of students from similar backgrounds show significantly different levels of academic achievement, higher performing schools can be studied intensively to determine the key factors that are responsible for their relative success.¹⁷

For purposes of this analysis, the term education establishment refers to the constellation of education school professors, teacher education textbooks and journals, teacher certification training programs, and professional teaching associations (e.g., the American Educational Research Association, or AERA; the National Education Association, or NEA) that dominate thought and opinion within the education and teaching professions. Within that group, there are four arguments held by anti-testing critics that are given prominence that far outweighs their scientifically demonstrated validity.

Claims That Testing Harms Students

Eighteenth Century social philosopher Jean-Jacques Rousseau’s notion of children born in freedom and innocence, but eventually corrupted and enslaved by society,¹⁸ is the basic assumption that undergirds hostility toward standardized testing among many educators. According to critics, standardized testing places undue emotional stress on students due to test scores’ relation to important outcomes. They argue that testing fails to measure accurately the capabilities of students with different learning styles and penalizes pupils who are not good test takers.

Another common argument is that standardized testing fails to account for language deficiencies, empty stomachs, learning disabilities, difficult home lives, or cultural differences.¹⁹ The tests are said not to measure student progress or improve student performance, but rather penalize students’ critical thinking and creativity due to the multiple-choice testing format (or its opposite), namely, that tests confer an unfair advantage to students who perform well on multiple-choice tests by learning test-taking strategies without having deep knowledge of the subject matter.

Anti-testing hostility has found a powerful, organized voice in numerous movements whose prime directive is to diminish the influence—if not the outright banishing—of standardized testing in pre- and post-higher education. The opt-out movement, for example, began in New York in 2014 among mostly White, highly-educated, and politically liberal parents who were united in their refusal to have their children sit for standardized testing in schools.²⁰ They claimed that judging teacher performance by students’ test scores is unfair and that testing unduly narrows the school curricula by creating a “teaching-to-the-test” instructional ethos. Some stated they were in outright opposition to the implementation of Common Core State Standards.²¹

It would not be an overstatement to say that certain criticisms have their origin in various neo-Marxist ideologies. There, standardized tests are portrayed as instruments of oppression designed by capitalistic test-construction companies to crush students’ dreams of a better life and trap them in the social classes in which they were born. One such critic writes:

Rather than providing for an objective and fair means of social mobility, the tests were a tracking mechanism limiting the odds of improving on one’s family’s economic and social position in America…. The SAT aptitude test in particular was designed from the beginning to facilitate social Darwinism, selecting for White Anglo-Saxon males; Jim Crow segregation, eugenics, and protecting the Ivy League’s racial stock provided the legal and cultural context in which the SAT was born.²²

These criticisms are feeble, shallow, and above all, dishonest. Rebuttals to these fallacies, patiently documented and dissected by recognized testing scholars, are readily available to anyone with a fair and open mind.²³

Claims of Cultural Bias in Tests

The critically acclaimed 1991 film Boyz N the Hood told the tale of three Black youths growing up in a South Central Los Angeles ghetto, and the differences in their eventual life outcomes as a function of having (or not having) a strong father figure. One of the boys has a strict but caring father figure (named Jason “Furious” Styles), while the other two do not. In numerous spots throughout the movie, Mr. Styles imparts pithy pearls of wisdom to the boys, intended to guide them throughout life. In one such sequence, he opines on the SAT requirement for college:

Most of those tests are culturally biased to begin with. The only part that is universal is the Math.²⁴

Wrong. Although popularly believed, the claim that contemporary standardized mental testing is culturally biased is patently false, as revealed in hundreds of empirical studies.²⁵ When critics accuse standardized tests of cultural bias, they typically mean that a test includes words, concepts, or ideas that are perceived to be more familiar to White middle-class examinees compared to other groups, or that a test’s standardization samples fail to include sufficient representation of non-White, lower socioeconomic status (SES) persons.²⁶

Both of these conditions are alleged to foster an unfair disadvantage to lower SES non-White examinees, purporting to cause them to have lower average scores relative to more advantaged White test takers. While some critics may not be familiar with the content of tests or the racial/ethnic makeup of standardization samples, they nevertheless believe that standardized tests are biased simply because the average scores achieved by different subpopulation groups are not equal. Such a definition of test bias is widely rejected by contemporary testing experts.²⁷

The cold reality, however, is that test companies, like all other companies that must be profitable in order to stay in business, routinely and carefully examine their test items for any evidence of statistical bias in the production phase, before any updated test revisions are published. Items that show actual evidence of statistical bias (i.e., items that statistically perform differently for test takers of different racial/ethnic groups) are discarded, and the results of statistical tests for biased test items are typically published in test manuals for open review by the general public.²⁸

Crying Racism

Whenever attempts to tar and feather tests with charges of cultural bias fail, the next step is to simply smear them with the charge of racism. In today’s heated political climate few things are more effective in attracting panicked attention than labeling persons, organizations, or products as “racist.” In the 1990s, test critics began to point out that the term “aptitude” in the (then-called) Scholastic Aptitude Tests (SAT), could be perceived as measuring something innate that is impervious to effort or instruction.²⁹ This, coupled with the fact that these tests reflect the significant subpopulation group differences in mean scores discussed above, prompted the College Board to change the middle word of the SAT from “aptitude” to the more bland descriptor “assessment” in 1993.³⁰ That euphemism, however, did little to quell the ire of critics, who continued to accuse standardized college testing of being racist.³¹

In today’s heated political climate few things are more effective in attracting panicked attention than labeling persons, organizations, or products as “racist.”

To be fair, it is relatively easy to locate offensive quotes by 19^th and early 20^th-century testing supporters who freely ascribed the adjectives “inferior” and “superior” to racial groups on the basis of significant mean differences in IQ scores.³² It comes as little surprise, therefore, when Ibram X. Kendi, founder and director of the Center for Antiracist Research, declares that:

Standardized tests have become the most effective racist weapon ever devised to objectively degrade Black and Brown minds and legally exclude their bodies from prestigious schools.³³

Kendi and many others never doubt that contemporary testing must be racist, based on the false belief that such testing was birthed out of a history of racism.³⁴ There is no doubt that these types of claims are very effective in poisoning contemporary public discourse, but such invective does not hold up under critical examination or hard evidence.

First, many early researchers were extremely cautious about, and resistant to, interpreting group differences in text performance as ironclad indicators of any innate inferiority/ superiority of groups. While racist attitudes were certainly more prevalent a century ago compared to today, many early American IQ test researchers were keenly aware of racial discrimination and unequal social circumstances of racial groups during the times in which they wrote, and so urged their peers to avoid hasty and intemperate generalizations from performance on tests until environmental disadvantages could be properly ruled out.³⁵

Second, not a few early 20th-century researchers intentionally showcased the exceptional IQ test performance of high-scoring non-White (particularly African-American) students, who achieved scores several standard deviations above the general mean.³⁶ Their writings disprove the assertion that there is something intentionally nefarious deeply embedded within mental tests that unfairly suppresses the intellectual capabilities of examinees who are not White and/or middle class.

Third, one study using a large and representative dataset of school-aged students in California, analyzed the sources that account for IQ test score variance (using Analysis of Variance, a long-standing, well-established, and widely-used statistical method), and demonstrated that the largest sources of IQ test score variability are within and between families that in many cases share the same racial group and social class.³⁷ If two members of this same dataset are selected at random (regardless of race, ethnicity, social class, or family) and the difference in their IQ scores are calculated and averaged and the procedure repeated an infinite number of times, the average difference between randomly selected pairs of IQ scores is 17 points.

Given that the mean of modern IQ tests is 100 and its standard deviation is 15, this average 17-point difference between such randomly chosen pairs exceeds the average score differences between Black and White students in the dataset (i.e., 12 points). Simply stated, the average IQ point difference between siblings in the same family exceeds the average test score difference between African Americans and White Americans. Taken together, these findings demonstrate the oft-repeated claims that IQ and other mental tests are inherently flawed and discriminate unfairly along racial lines, are simply false. This won’t convince Ibram X. Kendi, however, since his definition of racism is any group difference of any kind anywhere, thereby rendering the concept unfalsifiable.

Lowering Standards

Whenever two or more subpopulation groups achieve unequal means in their test score distributions, any set cutoff score that a college or university uses to determine acceptance or rejection for admission will display unequal percentages across groups as to who is selected or rejected. That is a statistical reality. For admissions committees that champion Diversity, Equity, and Inclusion (DEI) mandates, standards must be lowered for members of lower-scoring groups in a manner that camouflages what is actually being done.

Researchers have long acknowledged that obtaining data on college admissions decisions is an uphill battle, as colleges strive to prevent access to the criteria on which acceptance decisions are made. When such information is obtained, the results confirm what many have always suspected.

That is to say, Black and Latino applicants are admitted with test qualifications that are as much as one standard deviation or more below the average test scores of White and Asian applicants,³⁸ and this practice has predictable consequences. To illustrate, many Black and Hispanic students find themselves on academic probation or switch majors (from the major into which they were initially admitted) to enter disciplines that are less demanding.³⁹ Many of those so admitted will simply drop out and fail to graduate, creating “artificial failures” that would have been successful if properly matched to institutions that enroll students with comparable qualifications.⁴⁰

This observation was solidly reinforced in Richard Sander and Stuart Taylor’s 2012 book Mismatch: How Affirmative Action Hurts Students It’s Intended to Help, and Why Universities Won’t Admit It. In it, the authors examined and compared enrollment, graduation rates, and doctorate/STEM graduate degrees of Black and Hispanic students in the state of California in the eras before and after Proposition 209 was passed in that state. Proposition 209 (Prop 209, also known as the California Civil Rights Initiative, or CCRI), was a ballot proposition approved in 1996, which prohibited state governmental institutions from considering race, sex, or ethnicity in public employment, contracting, and education.

When comparing the pre-Prop 209 to the post-Prop 209 eras, the number of Black students receiving bachelor degrees from University of California (UC) schools, the number of UC Black and Hispanic freshmen who went on to graduate in four years (as well as graduate with STEM degrees), and the number of graduates with GPAs of 3.5 or higher all significantly rose. This hard data was used to support the general thesis that when students are matched (through objective standardized test scores) to institutions where all students are admitted under the same standards (and standards are not artificially lowered to satisfy diversity goals), minority students benefit significantly.

These practices are so pervasive, that Black students who meet the same college admissions requirements as their peers often write of their frustration and resentment at being unfairly judged by other students as having been admitted solely because of their race and under lower standards.⁴¹ In one particularly heartbreaking account, a successful Black journalist wrote of his frustrations taking two years out of his professional life to teach journalism to Black students, admitted under lowered academic standards, at a small, historically Black college. He writes of his reluctant efforts to repeatedly lower basic academic expectations in order to accommodate a critical mass of students whose attitudes, values, achievement motivation, academic preparation and qualifications, and intellectual capabilities demonstrated that they had no business being at an institution of higher learning.⁴²

One strategy for justifying lowering standards is for college admissions committees to claim that their admission standards are “holistic.”⁴³ That is, criteria for admission presumably must take into account a wide range of factors that provide a more “three-dimensional picture of the whole person,” as opposed to the more “narrow” consideration of standardized test scores. Yet critics charge that the deep subjectivity of such practices represents little more than academic flimflam.⁴⁴

The oft-repeated claims that IQ and other mental tests are inherently flawed and discriminate unfairly along racial lines, are simply false.

Recently, testing companies have come to serve as enablers of lowered college admissions standards. For example, the College Board spent two years (2017–2019) creating an “adversity index,” a 100-point scale that provides a rough measure of the degree of adversity versus privilege in the life of a prospective applicant. In theory, adversity index scores could be used to balance lower standardized test scores in an effort to justify lower admissions standards. Ultimately, however, these efforts of testing companies to placate their critics once again proved futile.⁴⁵

Another strategy is to claim that empirical research supports the benefits of having diverse academic settings compared to those not as diverse. For example, a DEI advocate cited research support for claims that students who enroll in more diverse classrooms earn higher GPAs, more diverse college discussion groups generate “more novel and complex analyses,” and that greater exposure to diversity in college settings increases civic attitudes and engagement.⁴⁶

However, studies of such an important topic as the benefits of diversity in college admissions require at minimum systematic replication as well as hundreds of studies by independent researchers (conducted at a wide variety of institutions) if they are to yield results that can be subjected to appropriate meta-analyses.

One study, however, is notable for its elegance, clarity, and simplicity. In 2002, researchers specifically evaluated the claim that increased racial diversity in college enrollments enriches students’ educational experience and improves relations between students from different cultural groups.⁴⁷ They argued that prior self-report data claimed to demonstrate support for this notion were misleading, as they suffered from biased item wording, methodological flaws, and the tendency for responses to reflect social desirability effects.

To correct for these flaws, the researchers analyzed self-report data from a random sample of more than 4,000 American college students, faculty, and administrators who were asked to simply evaluate various aspects of their educational experience and campus environment, but without any direct references to racial/ethnic diversity. They then correlated their data with the percentage of Black student enrollment in predominantly White student bodies. They found that, contrary to what diversity advocates would predict, no consistent positive correlation was found between increased diversity and respondents’ assessments of educational satisfaction.

Delete Standards Altogether

Eventually, what was previously unthinkable, has now become unavoidable: objective standards in and of themselves are seen as an impediment to the goals of achieving diversity, equity, and inclusion. Hence, testing necessary for demonstrating mastery of taught subject matter must itself be abolished.

This article appeared in Skeptic magazine 28.3
Buy print edition
Buy digital edition
Subscribe to print edition
Subscribe to digital edition
Download our app

In one example, the Oregon state legislature eliminated (for two years, until the state can re-evaluate its graduation policies) the long-standing requirement that students successfully pass a high school exit exam in order to demonstrate proficiency in reading, mathematics, and writing. This was done in response to criticisms that the testing requirement was inequitable because higher percentages of Black and Hispanic students failed the test.⁴⁸

Various anti-testing writers and organizations applaud the news that more and more institutions of higher education no longer require standardized test scores as a condition for selection,⁴⁹ under the pretense that “the social and academic costs of continuing to rely on…tests outweigh any possible benefits.”⁵⁰

Where are we headed?

At the time of this writing, the U.S. Supreme Court has ruled that the admissions programs at Harvard University and the University of North Carolina (where race is used as one of many factors in student admissions) violate the equal protection clause of the 14^th Amendment of the United States Constitution, which guarantees equal protection for all U.S. citizens.⁵¹ In a videotaped reaction to the decision, President Biden stated that the decision “effectively ends affirmative action in college admissions,”⁵² a sentiment echoed by many who support the continued and fair race-neutral use of standardized tests. Nevertheless, many commentators have also suggested ways in which admissions committees can circumvent the decision by no longer requiring standardized testing, or by changing the manner in which applicants write their college essays to signal their racial group membership.⁵³

There is simply no way to produce a mental test that effectively measures the abilities and skills needed to predict success in educational programs but at the same time satisfies the political goals of racially proportional representation as demanded by DEI advocates.⁵⁴ Given this reality, the war involving standardized testing has by no means ended, but rather is just beginning.