The Every Student Succeeds Act gives states new responsibilities and wide latitude to rethink how they determine if a school is successful.
But amid enthusiasm for that broader vision for accountability come questions and challenges as state leaders seek public opinions and buy-in from the educators and district-level administrators who will carry out their plans.
The big question: When redesigning a state’s accountability model, how bold of a change is too bold?
In addition to traditional accountability measures—English-language proficiency, graduation rates, and scores on state achievement tests—the new federal education law requires states to incorporate at least one “other indicator” into their accountability systems. That indicator must be measured at the student level so that data can be disaggregated to show trends among groups of students, like racial groups and English-language learners.
The law lists a few examples of “other indicators,” including school climate and student engagement, but it gives states broad discretion about which, and how many, factors they select. Some examples suggested by state leaders and education policy watchers include college-going rates, access to arts and science education, and rates of chronic absenteeism.
Some advocacy and interest groups have pushed for state leaders to use the new measure to dramatically reshape their education systems to include more “whole-child” factors, like measures of social-emotional learning, student support, and schools’ ability to move the needle on student traits like grit and self-management.
Caution From All Corners
But the potential for big shifts in school accountability has also been met with caution from other corners of the education field: researchers who warn that some measurement methods aren’t ready to be used in high-stakes accountability; school leaders who say they lack the resources to meet some new, creative benchmarks; and the reality of the time and effort it takes to collect and assess new forms of accountability data.
Another concern: Data collection needs to be done in a way that is consistent across schools and districts.
“It has great potential to allow states the flexibility of looking at some of the things we know are important in terms of what really constitutes a high-quality education,” Kristen Amundson, executive director of the National Association of State Boards of Education, said of the new indicator. “That said, potential doesn’t always translate into actuality. The expectations for what [the other indicator] can do may be a little higher than what it can actually deliver.”
Amundson said she expects some states to start with less-dramatic new accountability plans, adding data they already measure, like school suspension rates, rather than incorporating a host of factors related to students’ relationship skills and well-being.
Education Week asked teachers to select the area they would most like to see their states measure for ESSA accountability purposes. The largest share of survey respondents (23%) pointed to students’ social and emotional learning.
But some states may begin collecting data in those social and emotional areas to get baseline information, to gauge its reliability, and to explore the possibility of incorporating it into accountability systems in the future, she said.
“Even under [No Child Left Behind], which was much more prescriptive, states changed accountability measures and metrics many times,” Amundson said. “I think the message states need to get out—and some are already—is what we’ve said for a long time: School is much more than a single test score. This is our first attempt at it.”
A survey of 634 teachers by the Education Week Research Center illustrates divisions in what states should track to gauge school quality and how they should measure those factors.
Social-emotional learning ranked the highest among a list of possible “other indicators,” with 23 percent of respondents saying they favored it. Also ranking among the top were student engagement, at 19 percent; college and career readiness, at 15 percent; and student mindset at 11 percent.
Respondents were also divided about how to measure their favored indicator. The highest preference, classroom observations specifically conducted for the purpose of measuring nonacademic factors, won support from 21.4 percent of respondents. Surveys of students won support from 13.4 percent, and surveys of teachers were favored by 10.4 percent of respondents.
While backers of social-emotional learning have suggested states use their new models to encourage it in schools, many have stopped short of encouraging them to directly measure students’ competencies in areas like self-management and growth mindset.
That’s because, well before President Barack Obama signed ESSA into law, high-profile education researchers warned that existing measurement methods of areas like social-emotional learning are prone to a host of biases that make them unreliable and unusable for accountability purposes.
Most measures of a student’s social and emotional skills development are done through self-report surveys, through which a student rates his or her own competencies.
Ideal, Not Reality
But some research has shown that students who’ve completed lessons on skills like self-control and relationship skills may actually rate themselves lower in those areas than their peers who have not completed such lessons because they have a better understanding of what the concepts mean and how much they have to learn. That means schools could actually see their social-emotional learning “scores” go down after they seek to assist students in these areas.
Currently, “perfectly unbiased, unfakeable, and error-free measures are an ideal, not a reality,” researchers Angela Duckworth and David Yeager wrote in a 2015 essay published in Educational Researcher that detailed an array of flaws with current measures.
That didn’t stop a group of California districts, known as CORE, from incorporating student surveys about social-emotional learning into their accountability system, which was developed before ESSA was approved.
Leaders of that multimetric effort say schools use the data as a “flashlight, not a hammer"—meant to illuminate what does and does not work in schools and not to simply identify schools in need of heavy handed interventions—and that the accountability plan could be adjusted as better measures are developed.
They encouraged California leaders to consider their model, which also includes factors like suspension rates and how quickly English-language learners transition into mainstream classrooms, as the state designs its ESSA model.
Education Week asked teachers to give their opinions on the best way to measure the performance area they would most like to see their states include in ESSA accountability systems. Roughly one-fifth of survey respondents said that classroom observations conducted to assess nonacademic factors would be the best method, making it the most commonly cited approach.
California’s new state-level accountability model, approved by the state board in the fall, does not incorporate measures of individual student traits. Rather, its list of school quality metrics includes measures of school climate that seek to gauge how safe and supported students feel at school.
Conditions for Learning
Boosters of social-emotional learning have said school climate measures, which are often also determined through student surveys, are much more reliable than self-report surveys of student traits because they measure students’ perceptions of their environment and, when gauging a school environment’s effects on learning, perception is reality.
A healthy school climate is necessary to help foster students’ social and emotional development, and it is also often the result of that work, said David Osher, a vice president at the American Institutes for Research.
“We need both the capacity to learn—let’s say to learn mathematics, which includes my ability to self-regulate and to handle stress—but I also need to be in an environment in which I am able to learn, where I do feel safe, supported, and so forth,” Osher said at a November panel on ESSA hosted by AIR in Washington. “Those two are always fundamental ... If we ignore the environment, we ignore what creates the capacity.”
Osher assisted the U.S. Department of Education in the creation of a free, online school climate survey tool that can be used by schools, districts, and at the statewide level. While the idea for that tool predated ESSA’s passage, the survey results may be useful as states consider climate as an indicator, he said.
Still others have encouraged the use of broader “proxy indicators” that can be affected by a variety of school policies and practices. Rates of chronic absenteeism could be affected by anything from school health services to student engagement to wraparound services, advocates for those fields say. And states are already collecting data on attendance, leaving fewer hurdles to incorporating it into their accountability plans, they say.
Public Input
State leaders say that, whatever shape their plans ultimately take, they are working to communicate their discussions with educators and members of the public at large. They aim both to seek a wide array of input and to win cooperation and support when new systems are put into place.
Virginia hopes to create its plan to comply with ESSA as an offshoot of an ongoing effort to update its school accountability and accreditation at the state level, said Cynthia Cave, the assistant superintendent for policy and communications at the Virginia Department of Education.
State leaders have met with teachers, superintendents, business leaders, and members of the public in their efforts to determine how to measure school success, she said. In the end, they intend to choose a broad array of factors.
In an online survey accessible to anyone in the state, which received nearly 16,000 responses, participants were asked to rank 16 possible indicators. The top three choices were college and career readiness, graduation rates, and a measure of the quality of the learning environment.
I think the message states need to get out is ... what we’ve said for a long time: School is much more than a single test score.
“We’re asking ‘How do you measure it? How do you define it? How will we decide what is good enough and what is not good enough?’ ” Cave said.
Amundson, of the national school boards group, said states are bound to face some resistance no matter what indicators they choose. She encourages state decisionmakers to be careful and deliberate and to anticipate some possible future adjustments.
“In this country right now, where people don’t trust institutions, if you only put into any system those things that would get 100 percent acceptance with no pushback, what are you left with? Puppies and cat videos on the internet,” she said. “I think state boards should do what is right.”
