There has been a recurring pattern in American education for more than 100 years: When a concern is raised about student achievement and learning, there’s typically one go-to solution—more testing.
This flawed approach not only masks core problems—it keeps us from finding the true solutions to help improve outcomes for our students. At the heart of this is the reliance on what I call the “deceptive precision” of standardized tests.
A major example of this pattern is playing out in math education, set off by a report issued last November by a working group at the University of California San Diego that is still making waves. The authors confidently stated that today’s incoming students just aren’t prepared for college math, which quickly led to headlines like “STEM Professors In California Tired Of Teaching Stupid Kids For ‘Equity.’”
The rush to paint such findings as evidence of a broad emergency by taking bits of data and combining them with preconceived beliefs in unproductive ways follows a trend I call “crisis construction.”
The report took limited, small-N data and presented it as established fact. That, in turn, got amplified by the media, which stripped away the remaining nuance. Headlines trumpeted that “UC San Diego Finds One In Eight Freshmen Lack High-School Math Skills,” and journalists quoted easily misinterpreted phrases from the report, with one saying that “70 percent of them, or one in 12 members of the incoming class couldn’t do middle school–level math.”
Some of the biggest claims in the report are taken from a sample set so small it should surprise close readers.
Consider the item from the assessment most commonly cited to support the most alarmist headlines: “7 + 2 = □ + 6.” It was answered correctly by 103 of 138 students, or 75 percent. The other 35 responses are reported only as “wrong.” No answer distribution is published. If any of the 35 answered “9,” those students solved the left side and stopped, an attention slip. If they entered a variety of wrong answers, that’s a different problem entirely. Binary scoring can’t tell these apart, and the report never asks. From a mathematical-pedagogy point of view, exploring the processes and reasons that lead to the answer is standard and expected practice.
The claim that many of the students couldn’t do middle school math was based largely on a 30-question, 45-minute, no-calculator test given to 138 students already placed into remedial math due to their performance on a previous test. That is not a representative sample of UC students. The test measured the lowest-performing subset of one cohort, at one campus, in one term.
Even worse, the scoring of the assessment was binary: right or wrong. That means there was no chance for partial credit that would have revealed whether, say, a student used the right technique but entered the wrong number. And there was no attempt to distinguish between a student who did not understand fractions and one who stopped too early or simply had not done arithmetic by hand since middle school. In fact, a student who got all the first to fourth grade–level questions wrong and all the fifth to eighth grade–level questions right would be ranked the same as the student who got all the first to fourth grade–level questions right and all the fifth to eighth grade–level questions wrong. Those should be profoundly different diagnoses. The report makes no effort to tell them apart.
Even when scores are high, it’s hard to be sure they prove mastery. High scores could indicate a weakness of the test rather than student proficiency. It’s important to know the difference, because a solution is only as good as the tools used to implement that solution.
Too often, education leaders fail to look into what test results actually mean, and they overlook policy changes that would emerge from a more nuanced view of student performance that could be drawn from test data if approached differently.
Without the why, the current data support alarm more than they support policy.
Amid that alarm, around 2,300 STEM professors across the system, or approximately 15 percent of UC faculty, signed an open letter this summer calling for more testing: specifically, for the university to bring back the requirement that applicants take the SAT or ACT.
At colleges nationwide, scores on these admissions tests are used to make decisions about students’ course placement that the tests are not designed or calibrated to make.
For instance, a recent University of Maryland report analyzed the SAT’s alignment to state math standards and found it weak or unacceptable in 10 of 12 instances. Yes, the SAT tests math. But given that SAT math was found to cover less than half of Algebra I standards, the total SAT math score should not be treated as a comprehensive measure of state Algebra I content, nor as a stand‑alone diagnostic of readiness for more advanced courses, such as calculus.
The UCSD report makes this logical leap regarding the ACT, which suffers from a similar misalignment with math curriculum. The report states that an ACT math score of 24 (out of 36 possible) predicts success in Math 4C, a precalculus course. But a statistical relationship between an admissions score and course performance doesn’t prove that the score means students demonstrate knowledge required for the course. There is a near-perfect correlation between the number of math and statistics bachelors degrees awarded and the average amount of milk produced per cow in the U.S., but that doesn’t mean that cow’s-milk production has an impact on student math intelligence.
On the 2024 ACT, a math score of 24 could be achieved without the test taker correctly answering a single question strongly relevant to calculus. A student who answered all 33 weakly relevant questions correctly and only three or four moderately relevant questions correctly, while missing every strongly relevant question, would earn a 24. In other words, they would achieve the placement score but might lack the knowledge for calculus success.
None of this is unique to UCSD. At Old Dominion University, ACT math scores alone can qualify a student for placement into advanced math classes. A 25 to 29 can place a student into Precalculus I, a 30 to 32 into Precalculus II, and a 33 to 36 into Calculus I. In creating these hard boundaries, Old Dominion makes a distinction between a 24 and a 25. But, according to ACT, the typical measurement error in math is almost 2 points, so a 1-point difference doesn’t indicate that a student with a 25 is meaningfully better prepared than one with a 24. Furthermore, the ACT’s technical manual states, “it is advisable to consider using additional measures … in addition to ACT scores when making placement decisions.” It also explicitly warns against cut scores.
These admissions tests have some utility as general measures of math knowledge, but they should not be treated as accurate predictors. They provide broad signals. And those signals are noisy, at best.
A few years ago, many in education expressed concern that admissions testing requirements limited access to higher education. After all, not everyone has the information, time, or resources to prepare for the test or take it multiple times. One study by the College Board linked score gains of 115 points on the SAT to just 20 hours of practice. Then the pandemic shut down test centers across the country.
The UC system stopped considering the SAT and ACT starting in the fall of 2021. In part, that was due to concerns about how much admission tests excluded some deserving students. But officials also cited the limited predictive validity the tests provided. Many education leaders praised those decisions for expanding opportunity.
Since the end of the pandemic, though, some colleges have felt pressure to reinstate their test requirements. The shifting political landscape has brought back the 1950s–1970s view that a test score is a reliable marker of merit, and some colleges are aligning their policies to this viewpoint.
Reports are surfacing that encourage a reliance on test scores as the central feature of merit, citing lower performance by students who were accepted without or with lower test scores. But these studies often use easily misconstrued data to make their case. One study defined getting one C in the first year as not successful. Another study relied on the probability that a student who didn’t submit a test score would have a low first-year GPA but defined a low GPA as any below a 3.5, which sets the bar very high. Many of these students are not performing poorly—they simply have not reached an arbitrary standard.
We can find other solutions to make sure students are prepared for college math—solutions that might take time and effort but could greatly increase both math readiness and equity.
There are many approaches to this problem, including better coordination between high schools and colleges, increasing dual enrollment, and redesigning courses to better engage students. In the coming weeks, my colleagues will address these and other solutions in greater depth as part of this series.
Akil Bello is an educational access expert with more than three decades of experience working with underserved students and the organizations that support them. He is an admissions testing policy consultant at the National Center for Fair & Open Testing, or FairTest.
For more insights on the role of math in ensuring educational equity, subscribe to Just Equations’ newsletter.
