Around the world, school systems whose students have posted gains over time on international exams also appear to have embraced common clusters of interventions at particular phases of their improvement, concludes released last week by a prominent international consulting firm.
Such commonalities appear to transcend differences in nationality, size, demographics, and school spending, according to the analysis, which has drawn praise and skepticism from academics who study international comparisons.
Drawing from an analysis of nearly 600 reform strategies instituted across 20 international school systems over a quarter-century, the report by the London-based McKinsey & Co. suggests that school systems seeking to improve could do well by taking cues from the strategies used by those with similar performance trends—rather than trying to appropriate wholesale the techniques of the highest-performing countries.
Researchers from McKinsey & Co. evaluated 20 schooling systems around the world as they moved along a continuum of improvement from “poor” to “fair” to “good” and “great.”
SOURCE: “How the World’s Most Improved Systems Keep Getting Better”
It also takes a new tack on many current debates in education, eschewing the question of whether curriculum should be centralized or decentralized or whether teacher accountability should be formal or informal. Instead, the report indicates that all those strategies may be valid when deployed at the appropriate time on a system’s improvement trajectory.
Teachers’ Latitude
In general, the report finds that lower-performing school systems with weaker teaching forces, such as the education system now serving the Madhya Pradesh province of India, tend to provide teachers with prescriptive curricula and pedagogical techniques to ease the delivery of lessons and ensure consistency across classrooms and access for all students to achieve basic literacy and numeracy.
But systems that have mastered the basics and are striving for higher levels, like the Long Beach, Calif., district in the late 2000s, gradually give teachers and local schools more say over pedagogy and curriculum, transforming a tight central role into that of a supporting player that encourages local school personnel to use creativity and innovation to get students to reach ever higher.
The report also finds that growth in school systems can occur regardless of their starting point: Both the lowest- and highest-performing systems studied made improvements and narrowed gaps between students from different socioeconomic backgrounds, sometimes in as few as six years.
“It’s not the case that we have to wait 10, 15 years in order to see something happen in a school system,” said Mona Mourshed, a partner at McKinsey, during a Web presentation on the report. “It’s also not the case that rising performance means forsaking the achievement gap. Both of these are possible simultaneously.”
Common Interventions
The McKinsey report comes as the idea of “international benchmarking” education standards in the United States with those in other countries gains prominence. To an expanding base of literature on that topic, the McKinsey report indicates that the success of education interventions depends on when they are deployed, how they are adapted to fit the context of local circumstances, and how leaders go about gaining support for the initiatives.
For the report, McKinsey analysts examined gains on national and international exams since 1980 and identified two groups for study. The first consists of “sustained improvers” with five or more years of academic data points showing improvement, and the second of “promising starts”—developing countries with fewer years of improvement data.
From those groups, the authors selected 20 school systems—some encompassing an entire nation, others states or provinces—to study in depth. The list includes three systems in the United States: Boston; Long Beach, Calif.; and the Aspire Public 69ý charter network, in California.
Each system’s results on various international and national assessments, between 1995 and 2007, were translated onto a common scale. They were used to categorize systems’ performance trends in four stages: “poor to fair,” “fair to good,” “good to great,” and “great to excellent.”
Finally, the analysis catalogued a total of 575 interventions instituted in the 20 school systems during the period of their improvement and conducted field interviews with their leaders.
The researchers found that interventions fell in six different areas: revising curriculum and standards; establishing an appropriate reward and compensation structure for educators; building educators’ technical skills; assessing students; establishing data systems; and implementing laws and policies supporting the interventions. But the way those interventions manifested themselves at each performance stage differed.
For instance, systems that were poor-to-fair tended to rely on incentive funding for teachers and schools for meeting high performance targets. Those systems in the midlevel ranges had teacher salaries at a level comparable to gross domestic product per capita, while those in the great-to-excellent range tended to have salaries that far outpaced GDP per capita.
Similarly, lower-performing countries tended to use many more interventions based in accountability, like standardized student assessments, but systems in the good-to-great performance category and beyond, even those that had previously had formal accountability systems, reduced their frequency. And teacher evaluations gradually became less standardized as the teaching force got better and as teachers held one another accountable, through collaboration and demonstrations, for helping students to learn.
The report notes that context of the school system helps to shape the successful implementation of improvement strategies.
All the school systems studied share a focus on achievement data, for instance. But whereas the U.S. school systems and those in the United Kingdom and Canada have tended to make performance targets public, Asian and Eastern European countries tend to share data privately with schools for improvement.
The analysis also indicates that stable leadership appears to be a factor in all the improvements. Among the countries studied, education leaders oversaw reforms for an average of six years, at least two years longer than the average U.S. superintendent’s tenure.
Scale Concerns
Academics who have written about international benchmarking raised some questions about the universal-scale methodology used in the report. Created by Stanford University economist Eric A. Hanushek, the methodology has been used for other, less-extensive analyses of performance.
“I respect what they try to do in the study and I think it is certainly a move in the right direction as opposed to just saying [to education leaders], ‘Be like Finland; be like Singapore,’ ” said William H. Schmidt, a professor of statistics and education at Michigan State University, in East Lansing, who has compared content standards across countries. “But combining different subject matters across different tests and time points just poses enormous psychometric problems.”
Tony Wagner, an Innovation Education Fellow at the Technology and Entrepreneurship Center at Harvard University, raised questions about the different content of the exams on which the performance-trend data points were selected.
For instance, PISA consists mainly of open-ended questions designed to measure critical thinking, while the Trends in International Mathematics and Science Study, better known as TIMSS, is based heavily on multiple-choice questions that measure factual recall.
“When you dig deeper to figure out what’s being measured and how students are being motivated to succeed, there’s a difference of night and day in these systems,” said Mr. Wagner, who has written extensively about international comparisons.
McKinsey officials, however, said the universal-scale methodology used a control group of “mature and stable” systems to ensure accuracy.
The report’s implications for the United States are hard to tease out, given the country’s size, scope, generally decentralized nature, and disparities in income and population from district to district. Its more than 14,000 districts likely fall all along the report’s improvement continuum.
In the meantime, the nine-year-old No Child Left Behind Act, which requires states and districts to administer tests in grades 3-8 and once in high school, and intervene in low-performing schools, has been awaiting a rewrite since 2007. Proponents say that law has increased pressure to pay attention to neglected groups, but critics contend it has led to a focus on basic skills.
The authors plan to host an event to discuss what the report means for U.S. schooling on Dec. 7, the same day results from PISA are due.
