In , I pointed out that, when the first PISA results were released in 2001, students in Germany and the United States both performed at about the average for all countries in the survey, but Germany reacted with what came to be known as “PISA Shock” and the U.S. shrugged. In the years that have gone by since then, Germany has risen sharply in the rankings, while the U.S. remains in the middle, having improved not at all. The Germans, I said, leaped into action while we continued to sleepwalk through history.
That blog elicited two very interesting responses, one from Checker Finn and the other from Mike Petrilli, that I want to discuss in this blog.
This from Checker: “Our ‘PISA shock’ was A Nation at Risk, and the U.S. has been struggling/stumbling/fumbling/striving to fix its schools ever since. PISA may be...one form of education shock...but it’s not the only kind there is.”
And this from Mike: "...I don’t think it’s fair to argue that we’ve been sleepwalking since 2000. No Child Left Behind, circa 2002, was a very big deal. Perhaps it was wrong-headed, but it was a major shift in national policy. And it had its desired effect—boosting the performance of our lowest performing kids. The question, vis-à-vis Germany, is why we were satisfied with making a play for “equity” while they wanted a push for excellence—at all achievement levels, all income groups, etc. And, of course, Common Core was an attempt to go big and broad as well. And there was Race to the Top. So, my sense is that policymakers from 2000-2015 were willing to take action. Perhaps the actions were the wrong ones. But sleepwalking they were not.”
So Checker argues, in effect, that A Nation at Risk was as much of a shock to our system as PISA was to the German system and Mike argues that No Child Left Behind, the Common Core and Race to the Top were as consequential as anything the Germans did. And Mike goes on to point out that, while average student performance in the United States has not improved since 1983, a substantial part of the gap in performance between our average students and our disadvantaged students was closed, which may not be as much progress as we would like but would not justify my claim that we were sleepwalking through history.
On Checker’s point, it is certainly true that “PISA Shock” is not the only way to galvanize a national education system into action. I did not say it was. What I said was that we have been sleepwalking through history since then. I will get to the reasons I think we’ve been sleepwalking in a moment. What I want to say to Checker’s point is that A Nation at Risk sowed the seeds of the ineffectual national reform program that followed, whereas the PISA report provided just the fuel the Germans needed to turn their system around.
There is an elegiac ring to . It describes a fatal slide in the rigor of the American school curriculum, grading, university admission requirements and course requirements. It dwells on a supposed dramatic slide in SAT scores. In a section on American teachers that could be written today, it complains that our schools are sourcing too many of their teachers from the bottom quarter of high school graduates, pay very poorly, offer unacceptable working conditions to teachers, allow many teachers to teach subjects they have no expertise in, fail to demand teachers know their subjects and suffer from unending teacher shortages, especially in mathematics and the sciences. One is left with the sense that this is all the fault not of public policy, but of the professional educators who run and staff our schools. It is a litany of decline.
We know now that the data says otherwise. A fine paper when he was at the Congressional Budget Office acknowledges declines in average standardized test scores in the 60s and 70s but points to an upturn in those scores after that and to substantial improvements made by disadvantaged students during the period when A Nation at Risk said the country’s schools were going to hell in a handbasket. Most of the gains for disadvantaged students were made in the lower grades, which is where the federal funds for disadvantaged youth were concentrated. The data from the high school level NAEP surveys tell us that the reading and mathematics performance of our high school students hardly varied at all for decades following the creation of NAEP in 1964. But, whether you look at the elementary schools or the secondary schools, there had been no massive decline in performance of the kind portrayed in A Nation at Risk. The entire premise of the report was false.
You might say, well, what’s the difference? Maybe the claims of decline had little if any basis in fact, but the nation got the kick in the pants it needed anyway. But one of the biggest mistakes one can make in public policy as in your own health care is to get the diagnosis wrong. We now know that there was indeed a giant problem with the performance of American education. But it was not a decline in average test scores. It was, in fact, that the dynamics of the global economy were changing dramatically, creating a world in which all students in the high-wage countries would have to be highly educated to keep their heads above water—and the United States was doing little or nothing to meet that challenge, while other nations were beginning to do exactly that.
If the educational challenge the country faced was a decline from some standard of provision formerly reached, then the answer is obvious: restore the policies and practices formerly in place. But if the challenge is to provide to everyone the kind of education formerly provided only to the elite, then the response has to be to create a whole new system of provision that can deliver on that promise. Germany set out to do the latter. We did not.
But the problem goes deeper than that. A Nation at Risk not only described a failure that did not exist, its ringing language about "..a rising tide of mediocrity...” and its statement that “If an unfriendly power had attempted to impose on America the mediocre educational performance that exists today, we might well have viewed it as an act of war” implied that the nation’s professional educators were responsible for a precipitous decline in the performance of our schools, thereby laying the ground for what would later become a wholesale attack on the country’s teachers and reform strategies focused on the premise that the way to raise student achievement was to hold teachers accountable for their students’ performance on standardized tests.
There was no such language in the PISA report, nor were there any prescriptions of the kind found in A Nation at Risk. The first PISA survey just presented data comparing the performance of high school students in the participating countries. But it also connected that performance data to a wealth of background data of a kind that NAEP does not provide, helping policy makers and practitioners make links between policy, practice and student outcomes. That is, it not only provided accurate data on the “symptoms” of the patient, but it also provided plenty of data that would help the doctor make an accurate diagnosis of the nature of the “disease.”
Yes, correlation is not cause. But the kind of correlations that the PISA data make possible closes off a lot of rabbit holes. The comparative method makes it possible to weigh the consequences of organizing education systems in different ways. The authors of A Nation at Risk could not do that. In fact, they were not even able to compare the consequences of the differences in policies among American states, because American states were resisting proposals to use data to compare student performance among themselves and had no interest in comparing their performance to other countries.
Yes, as Checker said, A Nation at Risk was, in its own way, as much of a shock to the United States as the first PISA report was to Germany. But that is where the comparison ends. The PISA report led the Germans to focus, as Mike points out, on raising student performance across the board. Because it provided a lot of clues as to what the countries with highest performance and the most equity were actually doing, the Germans could use the data to take a range of actions designed to greatly improve both equity and student achievement.
We did nothing of the kind. Why? Mike points out that, while the countries that now outperform us on the PISA surveys were determined to raise student performance across the board, the United States concentrated on equity, on closing the gap between the average performers and those on the bottom. That is true. The Gallup surveys and others like them have shown for years that most Americans think their schools are doing well, that it is only the schools serving poor and minority students—mostly the inner-city schools—that really need reform. OECD research shows that American students are generally happier with their school performance than students in countries where student performance is much higher. Our expectations—both adults and students—appear to be much lower in this country.
So, one of the most important reasons that we did not decide to go for across the board improvement in student performance is because, all the evidence notwithstanding, we thought, and still think, that we are in general doing fine. It is not our schools that need reform, it is their schools that need reform.
There is an irony here. A Nation at Risk did not tell Americans that their schools were, in general, doing fine. It said that their schools were in catastrophic decline and needed major reforms that would, in effect, return them to their former glory. And it offered a set of reforms that the authors thought would accomplish that goal. The irony is that the United States then set off to fix a problem it did not have—a broad decline in school performance—by focusing on equity—the one area in which it had made substantial progress, at least in our lower schools.
There is a further irony here. What the PISA data showed was that the goals of equity and achievement are not only not incompatible, they make very good companions. Many of the countries with top achievement are also among the nations with the highest levels of equity. Let me put a finer point on it. It turns out that systems designed to raise student performance across the board are more effective at improving equity than systems designed just to improve equity. There are good reasons for that, but that is a subject for another blog.
And here is yet another irony. Though A Nation at Risk got the diagnosis wrong, many of its prescriptions were right on the money. I pointed out at the beginning of this blog, for example, that the report said that "...our schools are sourcing too many of their teachers from the bottom quarter of high school graduates, pay very poorly, offer unacceptable working conditions to teachers, allow many teachers to teach subjects they have no expertise in, fail to demand teachers know their subjects and suffer from unending teacher shortages, especially in mathematics and the sciences.” Yes. And it is all still true (though the “bottom quarter” was and is an exaggeration).
This is one among many examples of issues identified in the report that the top-performing countries have been working hard on and we have failed to address over the more than three decades since the report was released. The report was right to call attention to the lax standards for course grading and course content in our high schools. But, here too, very little has changed since the report was released in 1983. The report complained that graduates were ready for neither college nor work. Yes, and yes again, 34 years later. That, too, is what I meant by sleepwalking through history. Would a country that is serious about addressing these issues have made so little progress on them in all this time, while one country after another zips by us?
Only one of the interventions that Mike mentioned—the Common Core—had any parallel or precedent as one of the key tools used by the world’s highest-performing education systems. And, in all of those systems, high standards of this sort are used to build instructional systems that also include well-developed curriculum frameworks that cover much more than English literacy and mathematics, detailed course syllabi and very high-quality examinations. Most of this is missing in the American formulation. But there is much more that is also missing.
Here is a little postscript on equity for you. Mike says, in effect, that this country’s responses to A Nation at Risk, while they did not do much to raise average achievement, at least improved equity. That is not so clear. The report itself did not focus on equity. As Dan Koretz pointed out, equity was on the rise before the report was released, but then plateaued. One of the responses to A Nation at Risk that Mike mentioned was No Child Left Behind. But equity, which had been improving before that legislation was passed, improved at of that legislation than it had been before it passed. It was only between the 2012 and the 2015 PISA surveys that equity resumed its in the United States. Given this history, I think it is hard to argue that equity was poor before the big policy interventions Mike cites and then improved in response to those interventions.
I wish we had in fact put together a well-designed attack on the issue of equity. But, instead, we appear to be picking out, almost at random, a few components of effective systems and expecting them to function as they would in a fully developed complete system. We pay very little attention to effective implementation planning. We lurch from one silver bullet to another with every change in administration. While report after report calls its readers’ attention to the importance of attracting capable high school graduates to teaching, sending them to good universities for deep grounding in the subjects they will teach and providing them with competitive pay and decent professional working conditions, we continue to do the opposite, lurching from teacher shortage to teacher shortage.
My point is simple. A Nation at Risk identified the wrong problem, looked in the wrong direction (the past) for solutions to the problem we actually faced and laid the groundwork for what has turned out to be a disastrous attack on the nation’s teachers. Nonetheless, the report identified some problems that were very real, like teacher quality, on which we have made no progress at all and some solutions, like the development of powerful, coherent instructional systems, which we have addressed in a piecemeal way that leaves us far short of the kind of systems our top competitors have been able to develop. We develop solutions to problems we don’t have and fail to develop solutions for the problems we do have. Most importantly, we fail to develop systemic solutions to our systemic problems. That is what I meant by sleepwalking through history.
I have, I confess, used Mike and Checker as foils to make my points. Checker speaks of this country “struggling, stumbling and fumbling” in the years following the release of A Nation at Risk. Mike laments our decision to concentrate on equity when we could have gone after both equity and quality, as our most successful competitors have done. These comments, too, get at what I meant by sleepwalking through history. The serious countries, the ones that have not been sleepwalking, are the ones that went after quality and equity and outpaced us on both. They were honest with themselves about what the data said. They put a major effort into analyzing the strategies used by the top performers. They took the time and made the effort to involve the public and the professionals in the search for a better way to do education. They built their futures on real system designs, not silver bullet solutions. They built their systems on a strong foundation of first-rate teachers and school leaders. And they put as much effort into implementation as they did into policy making and design. That is not sleepwalking.