鈥淭he nation鈥檚 report card鈥� was released today, reporting the 2022 results for reading and math in grades 4 and 8. While I could ramble on about the National Assessment of Educational Progress鈥� latest outcome, I was far more interested in hearing what Marty West made of them. Academic dean for Harvard鈥檚 Graduate School of Education, research fellow with the National Bureau of Economic Research, editor-in-chief of Education Next, member of the Massachusetts board of education, and member of the National Assessment Governing Board (which oversees NAEP), West has as good a sense of these results as anyone in the nation. Here鈥檚 what he had to say.
鈥搁颈肠办
Rick: What鈥檚 the bottom line here? Are these results better or worse than expected?
Marty: The bottom line is that 4th and 8th grade reading and math NAEP scores are down from 2019 levels nationwide. They are down substantially. And they are down nearly everywhere: Every state (and the District of Columbia) saw scores drop by a statistically significant amount on at least one of the four tests administered this spring. The same is true for each of the 26 large urban school districts that participate in NAEP. And in those states or districts whose NAEP scores in a particular grade and subject were officially unchanged from 2019, scores were typically lower than they had been that year鈥攋ust not by enough to achieve statistical significance. It is fair to say based on these results that there are very few school systems nationwide where students didn鈥檛 lose considerable ground over the course of the pandemic.
We certainly expected 4th grade scores to be down based on the long-term trend NAEP results for 9-year-olds we released in September. This release revealed that the declines were larger and more pervasive in 8th grade math, where scores were down in 49 of 50 states (all but Utah) and in DC. That was the bad news.
As for good news, there were certainly some systems that seem to have weathered the pandemic better than others. At the state level, Alabama, Alaska, Hawaii, Illinois, Louisiana, and Iowa stand out as places where scores fell less on average across grades and subject areas than others. Among very large districts, Dallas, Miami-Dade, New York City, and especially Los Angeles fared surprisingly well. Catholic schools nationwide saw student scores hold steady on three of the four tests, though it is possible that this reflects shifts in enrollment into those schools in response to public school closures. And the schools operated by the Department of Defense Education Activity actually registered gains since 2019 in 8th grade reading and matched their prior performance on the other three tests. I expect we will see many efforts in the coming weeks to explain those outliers.
Rick: We鈥檝e seen a number of troubling test results over the past couple months: September鈥檚 long-term trend NAEP results were dismal, ACT recently reported its worst results in 30 years, state assessments have been grim. What do these NAEP numbers tell us that we didn鈥檛 already know?
Marty: You鈥檙e right that we have multiple sources of evidence that achievement fell during the pandemic, but they all have limitations. Not all students take college-entrance exams like the ACT, which makes it tricky to interpret changes in scores over time鈥攅specially during a pandemic. State tests differ from one state to the next, which makes it hard to compare both achievement and changes in achievement over time. Only NAEP can speak with authority about what has happened to student achievement nationwide and how that varies across states.
Rick: Parents and educators sometimes worry that there鈥檚 too much testing and may wonder why we need NAEP in addition to all those other assessments. Why should readers pay attention to NAEP, in particular? For those who aren鈥檛 familiar with it, what鈥檚 distinctive about it?
Marty: The NAEP program provides data that parents, educators, and policymakers need to put results from other tests and other measures of student performance into context. It is the only test that is routinely taken by a nationally representative sample of students and by representative samples of students in each state. It is this feature that makes it possible to understand how student achievement differs across all 50 states and, through the creative work by researchers at , across school districts nationwide.
A second feature of NAEP is that it is a no-stakes test. Except perhaps for governors or chief state school officers, no one鈥檚 evaluation or performance rating hinges on the results. In fact, NAEP results aren鈥檛 even available for individual schools and students. This means that we don鈥檛 have to worry as much that score gains could reflect the use of crude test-prep strategies or an over-emphasis on the test content as opposed to other education goals鈥攁 common concern when it comes to state test results.
One final unique feature of the test that parents may appreciate is that no one student takes the entire exam. We instead use an approach called matrix sampling, where several students each take one part of it. This is designed to minimize the burden placed on schools and students selected to participate.
Rick: Can looking at the results tell us more than we already know about which kids have been hit hardest by the pandemic? Are there any surprises here?
Marty: One troubling pattern in the long-term trend NAEP results released for nine-year-olds in September was that lower-performing students seemed to have lost the most ground. That is, the achievement of students at the 10th and 25th percentiles of the national distribution fell considerably more than that of students at the 75th and 90th percentiles. The gaps between higher- and lower-performing students on various NAEP tests had already been growing throughout the decade leading up to the pandemic, and the pandemic seems to have accelerated that trend.
Looking at the results out today, we see this same pattern in both math and reading at grade 4鈥攚hich is obviously cause for concern. The Black-white and Hispanic-white achievement gaps also grew in 4th grade math, which is a reversal of a long-term trend toward greater racial equity in NAEP results.
What is more surprising is the fact that 8th grade math scores fell dramatically鈥攁nd by roughly the same amount鈥攆or both higher- and lower-performing students. This is the main reason why the change in average scores was largest on this test.
Rick: OK, but how much faith should we put in these results? A number of superintendents and a few state chiefs have told me that, during the pandemic, kids got out of the routines of test prep and test-taking鈥攁nd that鈥檚 what we鈥檙e seeing. Do you think that鈥檚 a significant issue?
Marty: The fact that NAEP is a no-stakes test means that it is not a main focus of test-prep efforts in schools, even in ordinary times. The test prep that I worry most about when interpreting scores is activity focused on the unique features of a specific testing program鈥攆or example, coaching students on topics that are always included or questions that tend to be asked the same way every year. This is generally not possible on the NAEP, and educators don鈥檛 have an incentive to try to prepare students for the test. If students are out of practice when it comes to answering the types of questions posed by NAEP, I鈥檇 think that would also affect their performance in their classes and ability to make progress over time. For that reason, I鈥檓 not sure that I could separate a decline in NAEP performance from learning loss generally.
That said, the theory you cite is testable in the sense that, if it is correct, we鈥檇 expect a rapid recovery from this year鈥檚 losses by the time the main NAEP, which measures reading and math performance every two years, is next given in 2024. If that happens鈥攁nd I hope it does鈥攔enewed familiarity and comfort with test-taking could be part of the explanation. But I鈥檓 far from ready to dismiss this year鈥檚 results based on that concern.
Rick: You co-direct the annual Education Next survey on education. One of the things the survey has historically looked at is public views toward testing. Where is public sentiment on testing right now, and do you think there鈥檚 anything in these results, and in other grim assessment outcomes, that鈥檚 likely to move the public鈥檚 views?
Marty: What鈥檚 most striking to me is the public鈥檚 continued strong support for the federal requirement that states test students annually in grades 3-8 over the course of the pandemic. Roughly three of four adults favor that policy, which suggests that there hasn鈥檛 been a groundswell of anti-testing sentiment among the mass public. That pattern doesn鈥檛 mean that the public favors all uses of tests, of course, but Americans do still seem to see tests as a valuable source of information. I don鈥檛 see anything in these latest results that would change that.
In fact, my hope is that these results can play a role in shaping parents鈥� views of where their children are academically, as we emerge from the pandemic. When we at Education Next parents in May 2022, the parents of only 9 percent of students said they were not confident their child will 鈥渃atch up鈥� from Covid-related learning loss within a year or two; the parents of the rest either are confident the child will catch up or perceive no learning loss in the first place. Today鈥檚 results suggest that parents are far too optimistic鈥攁nd I hope the results spur some parents to take a closer look at their children鈥檚 academic progress over the course of the pandemic and what may be needed for them to catch up.
This interview has been edited and condensed for clarity.
