Pandemic disruptions have both increased the need for schools to find faster and better ways to support students鈥� learning and reduced their capacity to do so.
Continuous improvement鈥攁 process by which groups of people or institutions set up feedback loops to identify problems, quickly test solutions, and evaluate their effectiveness鈥攈as long been used in manufacturing and health care but more recently gained traction in education when many states adopted the process as part of their plans to implement the Every Student Succeeds Act.
Education Week talked with Robert Balfanz, a research professor at the Center for the Social Organization of 69传媒 at Johns Hopkins University School of Education and a co-author (with associate professor Martha Abele Mac Iver) of the new book Continuous Improvement in High 69传媒, to look at how the process may help schools adapt more rapidly to the challenges of the last two years.
This interview has been edited for length and clarity.
Why should educators focus on variation when they are trying to solve problems of practice?
One of these really challenging problems [in translating education research to practice] is we tend to reflectively go to the average or the general finding, right? Average test scores or average daily attendance is that, or overall graduation rates are this. Those are all numbers made up of different components, and oftentimes, most of the story is in the components, not the average.
One of the classic examples was, until maybe a decade ago, the only way people looked at attendance was average daily attendance ... how many of our total kids were in the building on a given day. And, you know, except in rare circumstances, almost every school was, on that measure, falling up in the low 90s. [Principals said] I鈥檓 sure I could do better, but my literacy rates are in the 30s, and so clearly, I have to focus on literacy and not so much on attendance. But the truth was if you broke down that average, ... you can have average daily attendance in the low 90s and still have 20 percent of your kids missing a month or more school, like different kids on different days. ... Then if you look, that highly overlaps with the kids that are struggling in literacy in this school. So commonly, they would invest energy in a new literacy program and they would get very modest results. And they were like, oh, is that really worth all that effort? ... And often, they were not checking and so didn鈥檛 know that half the kids they aimed that intervention for didn鈥檛 come to school regularly enough to get it.
The pandemic has caused a lot of disruptions in the data schools use to track early-warning indicators and measure progress. Can education leaders still reliably use data to guide their schools鈥� improvement?
Yeah, it鈥檚 really done a number on many of our traditional metrics. It鈥檚 going to be a while before there鈥檚 a baseline of achievement data that people can have confidence in, and attendance became very problematic because, how do you measure virtual attendance? ... So one thing it鈥檚 done is make people think about what are we really trying to measure with something like attendance? I think it鈥檚 a combination of things like engagement, being present, and were you able to actually get your work and assignments done, regardless of where and when you did it. 鈥淎ttendance鈥� is a proxy for a lot of things that in a traditional setting are easy to measure this way. ... But there are other, very local ways you can get a sense of engagement like [student surveys and assignments completed]. If you don鈥檛 have normal attendance data, what data can you collect relatively easily that will still address the underlying issue?
Most schools鈥� systems of support use a structure in which a general education works for the large majority of students, with smaller groups of students needing increasingly intensive interventions. But right now, there are large swaths of students who have fallen behind academically in multiple subjects. How does that change how educators should think about interventions?
Yes, I saw some recent data from California that suggested it was the kids in the middle of the bell curve that are struggling much more, ... and that increases the total number of kids struggling. Another benefit of a continuous-improvement approach [is] it lets you be adaptive to the situation. So, say I depend on this bell curve for a tiered support system, but ... we can鈥檛 individually intervene our way out of half or more of our kids [academically struggling]. So that tells us we need to look at classroom-level activities with help with school-level or grade-level interventions. Instead of just grinding out a strategy that鈥檚 not going to work, [the continuous-improvement process] gives you a framework to shift your strategy and figure out what will work now.
The 鈥淧lan-Do-Study-Act鈥� cycle includes time for educators to identify potential causes of a problem, research, and experiment with potential solutions. How many different solutions can a school typically test in a given year?
That does get to some of the real-world limitations with this. In general, in our experiences, schools try out two different ideas at once, to find if A or B worked better. And they could maybe do three cycles a year.
In theory, [school improvement] networks should help you because if we all collectively retry and the network [schools] are similar enough, then what works for you should largely work for me. I would say again, in the real world, that probably works half the time, because schools are just hard-wired at some levels that ... they can鈥檛 take it at face value that if something worked over there, I can adopt it and have confidence it鈥檒l work. ... There鈥檚 some resistance still built into saying, no, we all can try this. They got the good results, so let me just give it a go.
Through necessity, a lot of schools have been experimenting with structural and format changes in the last few years, particularly in high schools. Are there any innovations that you think are the likeliest to stick in high schools?
I think what鈥檚 interesting is that during COVID, at the high school level in particular, students spend a lot more of their time with asynchronous learning. They typically had Zoom classrooms only for a couple hours a day, while elementary kids would have all-day Zoom. In high school, they were given assignments to do the rest of the time. And, you know, for some kids that failed spectacularly鈥攂ut for other kids, they ended up liking it better to be able to do their work in their time and also, say, working to help their family.
And in many ways that was much more a collegelike experience. It has struck the schools as well, since, as you know, there鈥檚 been this big push for decades to really make sure the high schools are preparing you for college. ... And perhaps the best way to do that is to actually make high school closer to college.
So I know a number of our schools are now experimenting with how could we keep elements of that, where kids are in classrooms some of the time, but other times they could work in groups, they can work with a supportive adult, and it captures a little bit of that more collegelike experience while they鈥檙e still in high school.
