A prominent state education group has recalled an online report that presented the number and percentage of schools that states have identified as needing improvement under the No Child Left Behind Act.
Saying that journalists had used the data to make “unwarranted negative assessments†of the quality of states’ schools, the Education Commission of the States said in a widely distributed e-mail on March 31 that it had removed the six-page document from its Web site and wouldn’t distribute it further.
The document warned against judging states based on the number and percentage of schools failing to make adequate yearly progress, or AYP, under the federal law. But the disclaimer didn’t explain specifically that judging the quality of state school systems by comparing the data isn’t valid because each state sets its own AYP standards.
Though ECS officials said the data were accurate, they withdrew the report because of the way the information was being interpreted.
“What I felt was not there, was the correct interpretation of what [the numbers] meant,†Piedad F. Robertson, the president of the bipartisan, Denver-based clearinghouse on state education policies said in an interview last week.
‘Bottom-Feeder Ranking’
The incident is the latest example of the way newly accessible data are changing how the public views schools. It also demonstrates why state officials are wary of efforts that produce state-by-state data on student achievement.
For example, a new Web site produced by Standard & Poor’s and state officials allows users to compare schools’ student-achievement data within their own states. That kind of comparison is valid, state officials say, because those schools administer the same tests and are expected to hold students to the same standards. (“Online Tools For Sizing Up 69´«Ã½ Debut,†March 30, 2005.)
But if the site’s users want to compare a state’s student achievement with another state’s, they are able to look only at scores on the National Assessment of Educational Progress and college-entrance exams—the only testing programs given in every state.
The ECS report, which was released online last month, documented the number and percentage of schools in every state that are in various stages of school improvement under the federal law. Those stages range from the mandate that schools in the first year of an improvement plan offer students a transfer to another public school, to the implementation of a state-approved school improvement plan in the fifth year.
The data were presented in a state-by-state table at the end of the report.
The report explained that the numbers and percentages of schools not making AYP vary between states because of a variety of differences in the ways states are implementing the 3-year-old law, a reauthorization of the Elementary and Secondary Education Act. It said states’ standards differ, as do the difficulty of their tests and their short-term goals for making adequate progress.
But some journalists glossed over such warnings in their reporting, ECS officials said.
In Hawaii, for example, the Honolulu Star-Bulletin ran a story headlined: “Isle 69´«Ã½ Fare the Worst.†The story highlighted the ECS report’s finding that 10 percent of Hawaii’s schools were in their fifth and last year of needing improvement—the highest percentage in the nation.
Hawaii has 28 of its 284 schools at that stage because it has set ambitious AYP targets, a state spokesman said. It was unfair to label the state’s schools as the worst in the nation because those schools might actually have higher student achievement than in states with lower AYP goals, said Greg Knudsen, the spokesman for the state department of education.
State-to-state comparisons of AYP numbers are unfair “because states have such different standards and different timetables,†Mr. Knudsen said.
In the only other news story cited by ECS officials, a March 29 story in the Arkansas Democrat-Gazette said Arkansas “has some of the highest percentages in the nation of public schools categorized as academically troubled.â€
Two days later, an editorial said the report gives “Arkansas its usual, bottom-feeder ranking†on school quality.
The information collected does include worthwhile data, Mr. Knudsen of the Hawaii education department and Todd Ziebarth, the author of the report, agreed.
Useful Information
State officials could look at the number of schools each state is identifying as failing to achieve AYP, said Mr. Ziebarth, a policy analyst for Augenblick, Palaich and Associates, a Denver-based consulting firm that conducted the research for the ECS.
In doing so, he suggested, they can determine if they are labeling too many or too few schools as not reaching their goals. He said states also could find other states that have had experience with the interventions available for schools needing help, such as restructuring of those schools’ programs and staffs. “It helps to provoke states to think about how their system compares to others,†he said.
But Ms. Robertson said that while such information is useful, the ECS plans to produce reports that highlight states’ progress in complying with the law.
“What we need to concentrate on is working with every individual state,†said Ms. Robertson, a former president of Santa Monica College near Los Angeles who became president of the ECS Feb. 1.
