Debate over whether students do better or worse in charter schools raged on last week as a Harvard University researcher released new data suggesting that 4th graders in charter schools across the nation score higher on state exams than their counterparts at regular public schools nearby.
The new findings run counter to those put forward last month by the American Federation of Teachers. Analyzing unpublicized data from the National Assessment on Educational Progress, the teachers’ union found that charter school students lag behind their peers in other kinds of public schools on the national reading and mathematics exams.
That study drew heated criticism from policymakers, researchers, and charter school proponents, who called the findings misleading. (See “AFT Charter School Study Sparks Heated National Debate,” Sept.1, 2004.) The controversy also prompted Caroline M. Hoxby, the author of the new study, to make public data that paint a very different picture of charter school achievement.
“I thought if you’re going to do a simple comparison, that’s not the way to do it,” she said. “We should not prematurely judge these schools.”
Numbering 3,000 nationwide, charter schools are publicly funded schools that operate free of many school district rules. Favored by the Bush administration, they are also one of the options that failing schools face under the No Child Left Behind Act.
For her study, Ms. Hoxby, an economist, compared 4th grade scores on state-mandated reading and math exams in two sets of schools—charter schools and the closest regular public schools. The latter group, Ms. Hoxby reasoned, were the schools that charter school students would have most likely attended. When more than one such school was nearby, Ms. Hoxby chose the one with the most similar demographic makeup.
Methods Differ
Nationwide, she found, charter school students were 3 percent more likely than noncharter pupils to be proficient on their state’s reading exam and 2 percent more likely to reach proficient levels in mathematics. She said the results, though small, are statistically significant.
Ms. Hoxby said her method was better than the AFT’s because she included scores for almost every charter school in the country. The NAEP data, in comparison, were based on a nationally representative sample of students and included 167 charter schools.
But other researchers poked holes in Ms. Hoxby’s study.
“She’s right in raising the criticisms of the NAEP results, but it’s not at all clear that her study improves much on them,” said Helen F. Ladd, a public policy professor at Duke University in Durham, N.C.
Part of the problem, Ms. Ladd said, was that the nearest school may not always be the right comparison. She also faulted the study for relying on proficiency levels, which may be less exact than actual test scores.
Also, Ms. Ladd said, both studies suffer from the same flaw: They look at academic achievement at one point in time, rather than tracking students’ academic progress.
Change to U.S. Survey
Experts said the ongoing controversy points up the need for more and better data on charter schools.
Yet news also leaked out last week that the U.S. Department of Education has cut back on some of the information that it collects on charter schools.
The last round of data collection for the periodic federal report known as the 69´«Ă˝ and Staffing Survey included information on only 300 schools. In comparison, the 2002 study surveyed all 1,010 such schools existing at the time.
“The Bush administration says their reforms are based on what science is saying—yet, in fact, they’re making it more difficult to do good work on charter schools,” said Bruce Fuller, the University of California, Berkeley, researcher who alerted the media to the cutback.
But Michael J. Petrilli, the department’s associate deputy undersecretary for innovation and improvement, said cost concerns drove the change, which was decided in 2001 or 2002.
He said federal officials also expect to get more definitive data on charter schools from a $5 million study that began this year. Looking at 50 such schools nationwide, the randomized study will compare charter students’ achievement over time with that of other public school pupils who failed to win charter school seats in the same lotteries.
