Seven years and $7 billion dollars later, the federal School Improvement Grants program seems in the final analysis not to have yielded much in terms of improved student achievement.
The Institute of Education Sciences on Thursday released its final evaluation of the program, which Congress killed last year.
鈥淥verall, we didn鈥檛 find evidence that schools implementing SIG-funded models significantly changed student outcomes more or less than other similar low-performing schools,鈥� said Lisa Dragoset, a senior policy researcher for Mathematica Policy Research, who co-wrote the evaluation.
She cautioned that the overall findings don鈥檛 鈥渘ecessarily mean that no student benefited from attending a SIG school.鈥� Maybe, but they鈥檙e still a far cry from the early results from the first two years of the program, in which about two-thirds of schools that began the improvement grants in the 2010-11 and 2011-12 school years showed boosts to their performance on state math and reading tests. By contrast, in the final evaluation, Mathematica and the American Institutes of Research found none of the program鈥檚 four school improvement models led to significant gains in students鈥� math or reading test scores, graduation rates, or college enrollment, when compared to similar struggling schools who had not participated in the grants. Researchers had compared student achievement at schools that barely did or did not make the cutoff to be identified for turnaround under the grant.
鈥淭here were such high expectations for the program when it first started, that these would become dramatically different and better schools,鈥� said Robin Lake, the director of the Center on Reinventing Public Education at the University of Washington at Bothell. 鈥淭hey didn鈥檛 hit dramatically different; unfortunately, they didn鈥檛 even hit better.鈥�
Tracking School Turnaround Practices
During the Bush administration, the SIG program funded state systems of support for improving schools. But with a massive infusion of cash from the federal economic stimulus in 2009, the Obama administration reinvented the grants as a means to push鈥攁nd evaluate鈥攆our models to turn around chronically failing schools:
- Transformation, which required replacing the principal and adopting a value-added evaluation system for teachers and leaders that included student growth, as well as overhauling the schools鈥� governance, instructional practices, and learning time, and incorporating community school practices;
- Turnaround, which called for similar changes to the transformation model, but instead of implementing a new teacher evaluation system, it called to replace the principal and at least half of teachers outright;
- Restart, which involved reconstituting the school under a charter organization; and
- Closing the school and enrolling its students in higher-achieving schools in the district.
Evaluators found that by 2013, SIG-funded schools used about three more of the 35 turnaround practices studied鈥攕uch as extended learning time and comprehensive instructional reform鈥攖han other, similar schools, but the difference wasn鈥檛 significant. Interestingly, nearly 9 in 10 of both the SIG-funded schools and other schools trying to improve made changes in instruction, like adopting a new curriculum. Fewer than half of SIG or other schools made 鈥渙perational flexibility鈥� changes, like giving principals more power over their budgets, hiring decisions, discipline, or school schedules.
That was in line with what Lake said she saw in an earlier study of turnaround practices at SIG schools in Washington. 鈥淣ot much had really changed in the schools,鈥� she said. 鈥淭hey were being asked to do different things, but the fundamental culture of the school, organization of the school, the fundamental design wasn鈥檛 reorienting toward dramatically higher intervention strategies, dramatically higher expectations, or dramatically better teacher training and support.鈥�
Moreover, schools that used SIG money were more likely than other schools to increase student learning time or create community-centered schools, but they also stopped using those practices faster than other schools did after a couple of years, suggesting they may not have been able to sustain the initiatives as well after the grant money ran out. In spite of general similarities, the study found schools that used SIG money to implement the turnaround practices tended to be poorer and in more urban districts than schools that implemented the practices on their own.
Dragoset suggested the lackluster results of the program might have been the result of more schools both in and out of the grant program adopting similar turnaround practices, or that the individual turnaround practices were not as well implemented in some schools.
In an interview with Time this week, former Secretary of Education Arne Duncan argued the SIG schools鈥� progress has been 鈥渆ncouraging鈥� and that grant-funded high schools 鈥�.鈥� The school improvement evaluation did find that middle and high schools that adopted a turnaround model had students with higher math scores than schools that adopted a transformation model, but Dragoset said it was not clear whether that meant the overall turnaround model was more effective than the transformation model. And there were no similar differences among the models in reading or for math in elementary schools.
鈥淎s schools and states consider evidence-based approaches for turning around low-performing schools, it鈥檚 important for researchers to continue conducting rigorous studies to find out 鈥榳hat works鈥� to improve student achievement,鈥� Dragoset said.
Lake agreed: 鈥淭he amount of money spent here was a lot, so contextually we鈥檙e talking about billions of dollars spent to little avail,鈥� she said. 鈥淭hat should give all of us pause when it comes to big new interventions. We need to have an evidence base before we ask everyone to implement them.鈥�
Related: