I recently came across a rather acid attack on the methods used by people like me who argue that the U.S. education system is less efficient than the systems of other countries. Published by the Albert Shanker Institute, the piece in question is by Bruce Baker and Mark Weber, both professors at Rutgers University. The Shanker Institute has published many valuable contributions over the years, but this is not one of them.
The authors are not shy. Our assertions, they say, are “lacking in methodological rigor,” the models we use are “wholly unsuitable for drawing these or any other conclusions” and our arguments “offer little value in public discourse.” Hmmm....
The essence of their argument is that it is not possible to make apples to apples comparisons between U.S. states, let alone the United States and other countries. One would have to make sure that, when you were making such comparisons, such things as the levels and distribution of poverty among school children were the same, the general level of wealth in the countries or states being compared was the same, class sizes were the same, teacher compensation was the same (and included the same kinds and levels of benefits), the ratio between education personnel serving in schools and in all the other parts of the system was the same, the proportion of students in special education was the same, the proportion of homes where the language of instruction was not spoken was the same and on and on and on like that. One could in theory control for many if not all of these variables statistically, but one would have to have data on all of the relevant variables in forms that would permit such statistical manipulation. Since no one has it or is likely to get it, it is, they say, obvious on the face of it that statements about the relative efficiency or productivity of the U.S. system cannot be backed up by the facts or sound analysis.
This sounds like an objective critique made by sound methodologists on the basis of generally accepted scientific and mathematical principles. But it is not. It is an argument based on ideology and not much more than that. Leaving aside some really sloppy work—for example, they compare the United States sometimes to the top-performing countries and at other times to all the countries surveyed by the OECD but don’t tell you they are doing that—the authors’ points make sense only if their purpose is to show that American teachers cannot be held responsible for the manifest inefficiencies of our system. They have written a polemic masquerading as a statistical analysis.
Look at the list of items in the paragraph above in which I laid out the basis of their argument. They include, for example, such things as teacher wages, class size and the ratio of teacher wages to non-teacher wages in the education system as a whole. “Teachers’ wages in the U.S.,” they say, “are relatively non-competitive, both as a function of relatively low absolute wage and as a function of high (and more rapidly growing) non-teacher wage....The seemingly high per-pupil spending figure of the U.S. does not, therefore, translate into either competitive wages or small class sizes; instead, the U.S. has relatively noncompetitive wages and average to larger-than-average class sizes.”
First, the facts are wrong. This is a good example of the point I made above about the group to which the U.S. is compared. The only one of the top performers with significantly smaller class sizes than the U.S. average is Finland. Class sizes in the U.S. are roughly similar to class size in roughly half the top performers, and are smaller than the average in the other half. Over the last several decades, the U.S. has spent an enormous sum of money to reduce class size and gotten virtually nothing for that investment in terms of student performance, except in the early grades.
And yes, it is true that the wages of American teachers compare poorly to the wages of teachers in the top-performing countries, a point that we and our colleagues elsewhere have made many times, but someone should explain to me why this invalidates international comparisons. Seems to me it demonstrates the utility of such comparisons.
But that is an aside. What is important here is to follow the line of the authors’ argument. What they seem to be saying is that it is unfair to characterize the U.S. education system as inefficient because, for example, the ratio of total teacher compensation to total compensation in the system is lower in the U.S. than in the countries with higher student performance. We can’t blame the teachers for low student performance, they are saying, because so much of the money is spent on administrators and other non-teaching personnel. The whole piece is written that way, as if those of us who characterize the U.S. system as less productive and efficient than those of other countries with better student performance are attacking American teachers and the authors’ job is to defend American teachers with the tools of statistical analysis.
It is true that the U.S. spends less on teachers’ compensation and more on the compensation of non-teachers in the system than the top-performing countries. That is a very important feature of the American system. The authors appear to accept that feature as a given and suggest that we should be adding even more to our total costs by raising teacher salaries. Wouldn’t it make more sense to do what the top performers do: pay for higher teacher salaries, at least in part, by reducing the proportion of non-teachers on the payroll? But the authors are not interested in making our system more efficient: they evidently believe that acknowledging its obvious inefficiencies would expose our teachers to blame.
The authors do the same thing when they talk about special education. They want us to believe that we are not making apples to apples comparisons because the proportion of students in special ed in most of the top-performing countries is about half what it is in the United States and we all know that it costs much more to educate special ed students than it does to educate the other students.
But the OECD data also show that, in many of the top performing countries, a much smaller proportion of students wind up in PISA’s lowest performance category and, of those students, many more than in the United States go on to higher performance categories by the time they leave high school. Could it be that the very act of labelling students as special ed students overcomes all the good that special education services do for about half of our special education students? Instead of denouncing the comparison as invalid because we label so many more students as special education students, wouldn’t it make more sense to use the comparison to ask how we, like the top performers, could get better performance for those students who are performing least well, and do it for less money than we are currently spending?
The authors never mention the fact that though many top performers have pupil/teacher ratios much smaller than ours, their class sizes are often larger, sometimes much larger. And their student performance is higher. This is because they are using their “extra” teachers not to reduce class size, but to enable their teachers to work in teams to systematically improve instruction, make sure no students get lost in the system and provide tutoring whenever it is needed. What the data show is that, if countries invest enough money to put a really first-class teacher in front of every student, those teachers can handle larger class sizes than less-skilled teachers. So, larger class sizes can, at the margins, help pay for teachers to make as much as others in the high-status professions who have as much education as teachers have.
Insisting that these variables need to have the same values to make comparisons valid makes sense if you think you are defending American teachers against unfair comparisons. It makes no sense as a critique of international comparisons for the purpose of examining the productivity of the U.S. system versus the countries with the top student performance. Why? Because every feature of the American system just mentioned is the result of policy choices we have made. In every case, we can make a different choice. Don’t tell me, for example, that it is unfair to compare the results of our system to a system in which teachers’ compensation takes a higher proportion of the compensation dollar or a system in which only half of the proportion of students in the cohort are classified as special education students. Not only is it fair to make such a comparison, it is only by making such comparisons that we are going to learn how to get higher achievement overall for little more than we are currently spending—for example, by spending less on administration and more on our teachers.
And now we get to the point that the authors of this paper could and should have based their critique of the typical international comparisons on. It is true that the OECD comparisons show that the U.S. on average spends more per pupil on elementary and secondary education than the top performers and, the top performers, by definition, get better results in terms of both student achievement and equity. However, what the OECD is doing is comparing school expenditure. That does not take into account expenditures by government that are not in the school budgets.
The most important cluster of these expenditures has to do with the services that governments provide to families with young children. The types and amounts of expenditure in this realm vary widely from country to country among the top performers. In some top performing countries, especially in Asia, these expenditures are not greater than in the United States. But in others, they are far larger. I am speaking here of such things as child allowances, child bonuses, well-baby services, nutritional services, extensive paid family leave, high quality child care and high quality early childhood education. The extreme differences we now see between the incomes of our wealthiest families and our poorest combine with inequities in our school finance system and our demographics to produce a very large proportion of children entering our compulsory education system who may be homeless, malnourished, culturally deprived, ill, in need of dental care and severely traumatized.
Unlike the factors listed above that fall within the realm of school policy and can be changed by people who make school policy, these things do not fall in the realm of school policy, but they have a dramatic effect on the ability of the schools to do their job as well as the cost of doing it.
If the authors of this paper had been doing their job, they would have known that, when you compare what the United States spends per pupil to what the top performers spend per pupil, and take into account such things as the cost and source of things like fringe benefits, the U.S. spends about 17 percent more in the primary grades and about 8 percent more in the lower secondary grades. But this difference is more than compensated for by the cost of the services that many of these countries provide to their families with young children that we do not provide. I take that to be a very important datum, but then, what do I know? I am, after all, offering you information that has “very little value in public discourse.”
The model the authors used to present their data was designed to show that, when one takes into account all the relevant variables, our teachers can’t be blamed for the mediocre performance of American students because we pay so much to school administrators, have so many students in special education, have large class sizes, etc. If, they say, we accounted for all those things, our teachers couldn’t be blamed for our students’ poor performance.
Not one of the many colleagues we have around the world who have worked for years to build a sound system for comparing student performance have done it to blame teachers for the poor performance of their students. It was not American teachers who decided to pay themselves poorly, give themselves so little support when they first became teachers, saddle themselves with a bigger school bureaucracy than any top-performing country has, or deprive themselves of any opportunity to advance in pay or responsibility without leaving teaching. For a long time, I have believed, on what is now overwhelming evidence, that the United States has one of the least efficient education systems in the industrialized world. I do not blame our teachers for that. Indeed, I think the record shows that the teachers are the victims of a rather dysfunctional system, not the creators of it.
The right question is how to create an entire education system that is as efficient and productive as possible. Economists refer to this approach as “total factor” productivity.
If we are modelling the education system in the United States, we need to include in the model—not outside it—every feature of the elementary and secondary education system that is paid for by funds intended for the schools. That is what I just did. As I showed, when you do that, what gets highlighted is a big disparity in spending on families with young children.
The irony, of course, is that addressing these issues after these children arrive in school is the least efficient and least effective way to deal with the problems we have caused by delaying for so long the help we should have given these children and their families long before they first came to school. Perhaps the accomplished statisticians who wrote this paper could turn their attention to these problems next.