One of the most highly touted personalized-learning models in K-12 education had no significant impact on students鈥� scores on state math tests, according to a new, federally funded released this week.
But a second, less-rigorous of the 鈥淭each to One: Math鈥� program, funded by the Bill & Melinda Gates Foundation and also released this week, suggested the program helps accelerate students鈥� academic growth in ways that most state tests don鈥檛 measure.
The mixed results highlight a rising source of friction in a K-12 sector increasingly experimenting with new approaches to customizing the learning experience for each student.
鈥淭here is a tension between grade-level accountability and meeting kids where they are,鈥� said Joel Rose, the CEO of New Classrooms Innovation Partners, the nonprofit group behind Teach to One, in an interview.
鈥淭hat in and of itself is a hugely valuable insight for the field.鈥�
Focused on middle-school math, Teach to One is based on mapping out the skills and concepts necessary to learn algebra, regularly gauging which of those skills individual students have mastered, and then using algorithms to provide each student with a customized path through the remaining material. Thirty-nine schools across 11 states use the program. In 2017, Education Week reported that about a quarter of schools that started with Teach to One eventually stopped using the program.
Descriptive studies鈥攊ncluding the new Gates-funded report released this week鈥攈ave found signs that students鈥� math learning grew faster while they were using Teach to One. But more rigorous analyses have not been able to attribute such gains directly to the program.
鈥淭here is no causal evidence that Teach to One has either positive or negative effects on student outcomes,鈥� said Douglas D. Ready, the director of the Consortium for Policy Research in Education at Teachers College, Columbia University, in an interview.
Investing in Innovation?
CPRE conducted its evaluation of Teach to One as part of a .
At the time, the program was hailed as a promising new idea worthy of expansion and further study. Such luminaries as Bill Gates continue to tout the model as the 鈥渇uture of math.鈥�
But like many of the initiatives funded via the Obama administration鈥檚 $1.4 billion i3 effort, Teach to One has seen choppy implementation in schools, and the program has yet to yield clear, consistently positive results.
CPRE鈥檚 evaluation focused on deployment of the program in the Elizabeth, N.J., public schools during the 2015-16, 2016-17, and 2017-18 school years.
The study compared changes in scores on standardized state math tests over that period at five Elizabeth K-8 schools using Teach to One: Math with the test-score changes at 16 academically and demographically similar schools in the same district that did not use the program.
The researchers were 鈥渦nable to draw generalizable conclusions, positive or negative, regarding [Teach to One鈥檚] impact on student mathematics performance,鈥� according to the study.
One big reason: there was considerable variation in how the five Elizabeth schools chose to implement the program and the extent to which they chose to focus on providing students with grade-level content. Some schools changed their approach during the period being studied.
As a group, the CPRE researchers found, Elizabeth schools using Teach to One: Math scored slightly better than expected in the first year, slightly worse than expected the second year, and about as expected the third year.
But there was substantial variation in achievement among the five schools testing the model. Year-to-year performance at some of the individual schools also fluctuated considerably.
After comparing those changes in scores over time to the differences in test scores at the similar Elizabeth schools that did not use Teach to One, the researchers determined that the impact of the program was not statistically significant.
What should other K-12 administrators and policymakers who may be considering the model make of the results?
鈥淭here were no positive or negative effects in Elizabeth,鈥� said Ready, who led the evaluation. 鈥淔or me to recommend [Teach to One], I鈥檇 really love to see more causal studies in other places.鈥�
Focusing on Growth
For the time being, though, such robust, rigorous studies of any approaches to personalized learning remain quite rare.
It鈥檚 also important to remember that the movement takes many forms, said John F. Pane, a senior scientist at the RAND Corporation, who has that rely less on technology and don鈥檛 focus on a single subject and grade span, as Teach to One: Math does.
鈥淚t鈥檚 a trap to get focused on this idea of, 鈥楧oes personalized learning work?鈥欌�� Pane said.
Instead, he suggested, researchers and practitioners should focus on gathering lots of evidence about a variety of specific personalized-learning practices, then triangulate the results to determine what is (and isn鈥檛) effective.
The new i3 evaluation of Teach to One 鈥渋s a data point in that exercise,鈥� Pane said, with both a 鈥渇airly strong鈥� methodology and some important limitations the K-12 field should consider.
Most notable, Pane said, is that end-of-year state tests are not well-suited to capture the kind of academic growth that Teach to One may actually be generating.
That鈥檚 why the other new study of the program released this week, conducted by the private research firm MarGrady Research, is also important to consider, he said.
For that report, researchers focused on the same three-year span as the i3 evaluation. But there were important differences: MarGrady included a broader sample of 14 Teach to One schools located in multiple districts. The firm used a less rigorous methodology with a weaker comparison group. And most notably, the MarGrady study focused on a different outcome measure: The Northwest Evaluation Association鈥檚 Measures of Academic Progress (MAP) test, which is designed to capture learning growth across grade levels, as opposed to grade-level proficiency.
That study found that students who were consistently enrolled in Teach to One: Math over the three-year period saw about 23 percent greater math test-score gains than students in a national reference group. Gains were even sharper for schools whose accountability systems prioritized student growth. And the study found 鈥渟uggestive evidence鈥� that when Teach to One delivered content that was closely matched to what students knew and were able to do, those students鈥� scores improved dramatically.
The results could not be conclusively attributed to Teach to One.
For Rose and New Classrooms, though, the MarGrady findings represent a challenge to the 鈥渃onventional wisdom鈥� that schools should focus on teaching students grade-level content as much as possible.
They also highlight a major flaw with the nation鈥檚 approach to K-12 assessment, Rose said.
Focusing narrowly on grade-level proficiency doesn鈥檛 appear to be a good to measure innovative new models aiming to meet students where they are and help them grow from there, he contended.
鈥淧ersonalized learning isn鈥檛 pixie dust. You can鈥檛 sprinkle it on top of a school and watch all scores go up for all kids in a single year,鈥� Rose said.
鈥淲hen schools and districts play the long game, they see the benefits for kids.鈥�
Photo: As many as 67 math students might be working in a single Teach to One classroom at Nathan Hale Middle School in Connecticut. Source: Mark Abramson for Education Week
This post has been updated with the current title for John F. Pane, senior scientist at the RAND Corporation.
See also: