For some, the results of the National Assessment of Educational Progress released this week show that the sustained local, state, and federal efforts to improve instruction in reading and mathematics are paying off.
Others offer a more sobering appraisal, suggesting the gains are meager, given the significant investment of time and money in those subjects. And some say it is still too early to tell if the latest gains in reading will turn into a steady upward trend, as has occurred in mathematics.
“Moving [these] scores is like turning the proverbial ocean liner. It goes slow,” said David W. Grissmer, principal scientist at the Center for the Advanced Study of Teaching and Learning at the University of Virginia in Charlottesville.
Joan L. Herman, an expert on testing who is the director of the National Center for Research on Evaluation, Standards, and Student Testing, said any statistically significant increase on NAEP is important. Judging what a 1- or 2-point gain in reading or math means, in terms of students’ overall academic knowledge, is tougher to quantify.
SOURCE: National Center for Education Statistics
“It’s sort of unknowable—is it practically significant?” said Ms. Herman, whose CRESST organization is based at the University of California, Los Angeles
Fourth grade math scores on “the nation’s report card” rose from 238 to 240 from 2005 to 2007, while 8th grade performance climbed from 279 to 281.
Those gains continued an overall upward trend in NAEP math scores in both grades that dates to the early 1990s, while reading scores have been more stagnant over that time. While the gains in math were smaller than in some previous testing cycles, they were still statistically significant, as were the increases in reading.
A total of about 700,000 students across the country participated in the reading and math exams, which were given from January through March. The new test results show not only nationwide trends in reading and math, but also state-by-state scores in both subjects
While the impact of a 2-point math gain on a 500-point NAEP scale should not be exaggerated, the overall steady climb in achievement is impressive, said Andrew C. Porter, the dean of the University of Pennsylvania graduate school of education and a member of the National Assessment Governing Board, which sets policy for NAEP.
In particular, he noted that the increase in test scores among the lowest 10th percentile of 4th graders in math—from 171 in 1990 to 202 today—represented the most headway in any performance group.
“It’s an amazing success story,” Mr. Porter said. “Over time, those gains are pretty tremendous.”
In reading, the subject that has seen the greatest investment of federal and state education spending over the past several years, 4th graders’ scores rose to 221 from 219 in 2005. Eighth graders’ average mark increased from 262 to 263, a statistically significant gain, though that figure dipped slightly from the reading test given five years ago.
Cause and Effect
The latest results emerge as Congress considers various ideas for reauthorizing the No Child Left Behind Act, which requires schools to test students annually in reading and math in grades 3-8 and once in high school.
U.S. Secretary of Education Margaret Spellings, as she has in the past, drew a connection between NAEP advances in reading and math and NCLB requirements that schools make adequate yearly progress in those subjects. Federal lawmakers are hearing from critics of the law who say that it has reduced teaching in other areas, and from those who want schools to be allowed to be judged by measures beyond reading and math test scores.
But the secretary said the new NAEP results show that changing the current focus of the law would be a step backward.
“No Child Left Behind is working,” Ms. Spellings declared in a statement. “It’s doable, reasonable, and necessary. Any efforts to weaken accountability would fly in the face of rising achievement. … To those who would suggest that [the law] is not working,” she said, “our nation’s 4th and 8th graders just proved the naysayers wrong.”
Mr. Porter and other experts, however, are skeptical of attempts to link rising or falling NAEP scores to the nearly 6-year-old law, or any particular program. Those who seek to establish that connection, he said, ignore the potential positive or negative impact on test scores brought by myriad state and local education efforts, many under way before the federal law took effect.
Less than 9 percent of nationwide education spending comes from the federal government; the rest comes from state and local sources.
“The problem is, you’d have to say, â€The only thing in play here has been No Child Left Behind,’ ” Mr. Porter said. “You’re trying to establish cause and effect … [but] all kinds of different causes happen simultaneously.”
Some critics of the NCLB law have challenged Ms. Spellings’ interpretation of the NAEP scores, suggesting instead that the trend in improved assessment results appears to have slowed since the law was implemented.
Richard Rothstein, a former education columnist and a research associate at the Economic Policy Institute, a Washington think tank, said the “one or two-point changes (about .03 of a standard deviation) since 2005 are meaningless.”
“The steady increase in math scores has been just that—steady since 1990,” he wrote in an e-mail. “Indeed, the rate of gain was greater from 1990 to 2003 than from 2003 to 2007. Therefore, any claims that the recent gains are attributable to NCLB are hard to defend.”
Others have noted that different NAEP tests, given to different student groups in recent years, show mixed results, and have criticized the the administration for singling out the positive data. (“Bush Claims About NCLB Questioned,” March 9, 2007.)
Population Shift
The improvements in reading and math occurred despite a steady demographic shift in the NAEP test-taking population since the early 1990s, noted David W. Gordon, a member of the National Assessment Governing Board.
For instance, Hispanic students’ share of the pupils taking the 4th grade math exam rose from 6 percent in 1990 to 20 percent today, with similar growth among those tested in reading. The proportion of students eligible for free lunches also rose at both grade levels and in both subjects.
Mr. Gordon, speaking at a Sept. 25 event here outlining the assessment results, suggested that improvements in reading and math scores were not so much the result of any single academic program or curricular approach used in states or schools. The strides were more a matter of states’ and districts’ commitment to a consistent academic-improvement strategy over time, said Mr. Gordon, the superintendent of the Sacramento County, Calif., office of education.
While the gains are encouraging, said the University of Virginia’s Mr. Grissmer, the rate of improvement and the persistence of the achievement gap are “disappointing,” given the intense focus on improving reading and math skills over the past few years.
“If we were going to expect something unusual to happen, we would start to see it now,” he said. “But we’re not seeing anything unusual in the trends.”
Eric A. Hanushek, a senior fellow at the Hoover Institution, a think tank at Stanford University, took a more positive stance.
“We’ve been spending more [on education] for the last 40 years and doing it in ways that haven’t had a very distinct payoff,” he said. “So the fact that we have coupled both our resource investment in schools with these other policies of accountability and gotten something that looks positive, we should be happy.”
State Initiatives
But Mr. Hanushek cautioned that the improvements, particularly on the 4th grade reading test, where scores have remained about the same for more than a decade, do not yet indicate an upward trend.
“If we could get two points every year for the next 10 years, that would be a lot,” he said. “Two points by themselves don’t mean anything if that’s all that happens.”
In some states that saw gains in both 4th and 8th grade math, officials attributed the improvements partly to the adoption of stronger, more specific standards for what students should know by each grade in those subjects—and to making those expectations clearer to teachers.
Kentucky recently revised its state math standards and a separate document that describes the material students are expected to know on the statewide exams. New expectations were set at each grade, rather than simply a few of them, from elementary through high school.
With clearer grade-level guidance, “a 3rd grade teacher can say, this is what I’ve been doing in class, and this is what I need to do to get to 4th grade material,” Joe McCowan, the manager of the math and science division for the Kentucky education department, said. “It’s consistency, coherence.”
Those more specific standards and curricula were crafted in part to help schools better prepare for the state’s annual tests in math and reading, as required by the No Child Left Behind law.
State officials in Missouri, which also improved its 4th and 8th grade NAEP math scores, made expectations for students in math and other subjects more detailed three years ago, said Stan Johnson, an assistant commissioner with the Missouri education department. “It’s about specificity—how specific you can be in giving teachers information on how to instruct and assess kids,” he said.
In addition, the state has staged workshops for teachers in math and other content areas on how they can make more effective use of in-class tests to identify students’ areas of weakness and prepare them for state assessments, Mr. Johnson said.
Massachusetts on Top
New Jersey, another state that saw its 4th and 8th grade math scores rise, has set new requirements for districts to align their curriculum and textbooks in math and other subjects with the state standards.
Education Commissioner Lucille E. Davy suggested that those changes are having an effect in math and reading. Ensuring that school systems follow a uniform curriculum is crucial, she said, because it increases the odds that students who move between schools do not repeat material or miss out on it.
In the past, “we knew we had several districts where the math curriculum in place was not the same from school to school,” Ms. Davy said. A recent audit of low-income districts, serving a combined 320,000 students, she noted, confirmed those inconsistencies.
While New Jersey’s performance in 8th grade reading was relatively flat, its 4th grade reading score jumped from 223 to 231. Ms. Davy attributed those gains to a statewide undertaking to improve literacy at early grades.
In reading, just Florida, Hawaii, Maryland, and the District of Columbia showed improvement at both grade levels. Eighteen states improved at the 4th grade level, while six did so on the 8th grade test. Thirty states had no significant improvement at either grade level. About half the states have progressed on NAEP reading since 1992.
Massachusetts came out on top in all categories. The state added 5 points to its average score on the 4th grade reading test, outscoring all other states with 236 points.
It was the top performer on the math test at both grade levels as well, with 252 points at the 4th grade and 298 points at 8th grade. The Bay State tied for the top spot on the 8th grade reading test with Vermont and the U.S. Department of Defense 69´«Ă˝, with 273 points. The state’s achievement gap between black and Hispanic students and their white peers, however, persists.
The results, “serve as proof positive that Massachusetts’ long-standing reform strategy of setting and holding to high expectations for all students is right on target,” said S. Paul Reville, the chairman of the state school board.
