There’s new evidence to suggest that customizing instruction for every student can generate modest gains in math and reading scores, according to a report released today by the RAND Corp.
Despite the promising signs, though, the researchers behind the most comprehensive ongoing study to date of personalized learning describe their latest findings as a “cautionary tale” about a trend whose popularity—and backing from philanthropists, venture capitalists, and the ed-tech industry—far outpaces its evidence base.
“It’s important to set expectations,” John F. Pane, a senior scientist and the distinguished chair in education innovation at RAND, said in an interview. “This may not work everywhere, and it requires careful thought about the context that enables it to work well.”
Pane is a lead author of “,” the third and most recent study in a multi-year RAND analysis being funded by the Bill & Melinda Gates Foundation. (The Gates Foundation has provided support in the past for coverage of personalized learning in Education Week.)
How Personalized Learning Plays Out in Reality
In the world of K-12 education, personalized learning generally means using software and other digital technologies to tailor instruction to each student’s strengths and weaknesses, interests and preferences, and optimal pace of learning. The idea has gained significant traction with groups such as the Bill & Melinda Gates Foundation, which has given more than $300 million to related initiatives since 2009, and the Chan Zuckerberg Initiative, which says it intends to invest hundreds of millions of dollars annually into efforts to bring its own vision of personalized learning to scale.
Research evidence to date has been thin, however.
And on the ground, the RAND team found that personalized learning remains hard to distinguish, harder to implement, and even harder to expand and replicate across schools.
The researchers identify four strategies they consider central to personalized-learning models:
- Learner profiles, which provide a record of each student’s “strengths, needs, motivations, progress, and goals;"
- Personal learning paths, which allow for variability in the content and instructional approaches that each student experiences;
- Competency-based progressions, in which each student’s progress is determined by mastery of key skills and ideas; and
- Flexible learning environments, in which schools can adapt their use of staff, space, and time in order to support greater personalization.
RAND’s new findings are based on surveys, interviews, and focus groups with students, teachers, and principals at 40 schools that have embraced the idea of personalized learning and begun implementing at least some of those strategies. All the schools received funding from the Gates Foundation, via an initiative called Next Generation Learning Challenges.
The schools are mostly urban and mostly charter, serving a total of about 10,600 students. Nearly all were students of color, and 80 percent were eligible for a free or reduced-price lunch.
As part of its study, the RAND team drilled down to the details of how the schools operate. They found a wide variety of management and instructional practices, many of which closely resembled the practices employed at more-traditional schools used as a comparison group.
The result, the researchers wrote, was that it was “difficult to clearly identify what makes a school a [personalized-learning] school.”
One example: Teachers in both types of schools also used student data in similar ways and were similar when it came to “tailoring instruction to student needs.” Both personalized-learning schools and their more traditional counterparts also struggled to offer students choice in the topics and materials they studied.
Futhermore, RAND found, the personalized-learning schools wrestled with a lack of high-quality digital instructional materials. The 40 personalized-learning schools in the study used 62 different online or digital curricula and assessments, only eight of which were mentioned by more than one school.
And perhaps most significantly, teachers in personalized-learning schools reported major challenges. They frequently said they didn’t have enough time to develop customized lessons for each student. They wrestled with competing priorities, such as personalizing instruction vs. encouraging collaboration and addressing common standards. And teachers in the personalized-learning schools complained about a common challenge: When left to move at their own pace through classroom lessons, they said, many students went too slow.
Such findings are not exactly shocking, said Benjamin Riley, the executive director of Deans for Impact, an Austin, Texas-based nonprofit focused on teacher preparation, and the former policy director of the New 69ý Venture Fund, where he had a front-row seat to the early days of the personalized-learning movement.
“There’s a growing acknowledgement of the reality of how personalized learning actually plays out,” Riley said. “Even if it were a good idea, developing a personalized-learning path for every student, in a system that has to educate tens of millions of children, might not be realistic.”
Modest Achievement Gains
RAND did find reasons for optimism, however.
Most notably, the personalized-learning schools in its new study appeared to successfully make more time for one-on-one instruction between teachers and individual students. Teachers in the schools also reported that they adjusted the composition of small groups of students more frequently, based on data about where each student stood.
Test scores also went up modestly.
RAND determined that a student who would have had average test scores in a more traditional school performed 3 percentile points better than average as a result of attending a personalized-learning school. This was true in both reading and math, although only the math gains were statistically significant. 69ý in personalized-learning schools who started the year academically behind also made up slightly more ground than comparable students in traditional schools.
The researchers also found a cumulative improvement in student test scores after schools had completed their second year of implementing a personalized-learning model.
Phyllis Lockett, the CEO of LEAP Innovations, a Chicago-based nonprofit that helps schools build personalized-learning models, said that finding reflects what her group sees on the ground.
“The longer schools are at this, the better they do,” Lockett said. “One of the things in this study that gives me real encouragement is the two-year numbers. There’s a positive trend there.”
Still, Pane of RAND cautioned against making too much of the positive achievement results.
For one, the gains found in this cohort of personalized-learning schools were less substantial than the gains RAND had previously found in an earlier cohort of schools, many of which were part of well-established, high-performing charter school networks that helped pioneer the personalized-learning movement.
In addition, among the cohort examined in the new study, charters generally outperformed district-managed schools, many of which actually saw drops in student achievement after implementing a personalized-learning model.
“Scale-up is a challenge,” Pane said. “Conditions at other schools might not be exactly the same as they were in the early adopters.”
It’s also impossible to tell at this stage which specific personalization strategies and practices have the biggest impact on students, the RAND report says.
‘You Can’t Just Flip a Switch’
So what do the new findings mean for policymakers and school and district leaders?
Riley of Deans for Impact said the new RAND report should lead to some much-needed healthy skepticism.
“At the highest level, I think educators should be concerned about whether we are in the middle of yet another fad,” he said. “The researchers are saying, ‘Let’s at least take a deep breath, and listen carefully to some of the warnings that people in the field are giving us.”
For Lockett of LEAP Innovations, meanwhile, the takeaway was the importance of intentional strategies to improve schools’ implementation of personalized-learning models.
“Trying to overlay these kinds of personalization practices and ed-tech tools on top of an existing [way of doing things] is really tough,” she said. “You can’t just flip a switch.”
And Pane of RAND said that while there’s enough evidence to date to keep pursuing personalized learning, this will likely be a race won by slow-and-steady tortoises, rather than hares who rush out of the gate with the expectations of quick, dramatic transformations.
That analysis points to a central tension that is sure to play out in the coming months and years: given the limited research evidence to date to support personalized learning, to what extent should laws and policies be changed to further encourage more experimentation and wider implementations?
The new RAND report, for example, includes a wide variety of policy recommendations for those seeking to better support personalized learning, especially when it comes to providing schools with greater flexibility to move away from traditional grade-level content standards and accountability systems.
Riley is among those who worry that such a shift, and the investment of time and energy and money that it will require, may not be warranted by the available evidence.
Pane counters that schools are unlikely to know what personalized learning can ultimately amount to if some of the barriers to effective implementation—and to conducting more rigorous research—aren’t removed, at least in some places.
“Personalized learning could really be something big if it’s not impeded,” he said. “There’s promise here, but we have to do the scale-up in a way that’s cautious and thoughtful.”
See also: