Is the No Child Left Behind Act working?
President Bush says it is, pointing to student-achievement results from a single subsection of the National Assessment of Educational Progress and tentative 69传媒 First data. But the evidence available to support his claim is questionable.
鈥淔ourth graders are reading better,鈥� the president said during a March 2 visit to a school in New Albany, Ind. 鈥淭hey鈥檝e made more progress in five years than the previous 28 years combined.鈥�
In mathematics, he said, elementary and middle school students 鈥渆arned the highest scores in the history of the test.鈥�
The data Mr. Bush cited at that event are from just the 鈥渓ong-term trend鈥� NAEP in reading and math, researchers say. All available data, they add, show modest improvements that can鈥檛 be attributed to the 5-year-old law. Instead, progress in achievement is more likely a continuation of trends that predate the law.
鈥淭here鈥檚 not any evidence that shows anything has changed,鈥� said Daniel M. Koretz, a professor of education at Harvard University鈥檚 graduate school of education.
Other researchers suggest that the standards and accountability system of the NCLB law is drawing attention to achievement gaps and other inequalities and is causing educators to change their practice. But it鈥檚 too early to say whether the federal law will result in achievement gains, they contend.
The law鈥檚 鈥渕echanisms are just coming into play, and not enough time has passed to establish a trend,鈥� said Adam Gamoran, a professor of sociology at the University of Wisconsin-Madison.
鈥業鈥檓 Lobbying Congress鈥�
Portraying the No Child Left Behind law as a success is a critical element in President Bush鈥檚 argument that Congress should renew it on schedule this year. The president signed the legislation, an overhaul of the Elementary and Secondary Education Act, with much fanfare in January 2002 and has cited it as his most important accomplishment in domestic policy.
鈥淚鈥檓 not only speaking to you, I鈥檓 lobbying,鈥� Mr. Bush said at the Silver Street Elementary School in New Albany earlier this month. 鈥淚鈥檓 lobbying Congress. I鈥檓 setting the stage for Congress to join me in the reauthorization of this important piece of legislation.鈥�
Congress is laying the groundwork for reauthorizing the measure. This week, the Senate education committee held a hearing on the law鈥檚 teacher-quality requirements. Next week, the House and Senate education committees plan to hold a joint session on an overview of the law.
Rep. George Miller, D-Calif., and Sen. Edward M. Kennedy, D-Mass., the chairmen of the education committees and two of the architects of the bipartisan law, say they hope to renew it this year. But many observers expect the process will be delayed until next year or even after Mr. Bush leaves office in 2009.
At the New Albany school, Mr. Bush highlighted the gains on the national assessment鈥檚 long-term-trend tests in reading and mathematics. Secretary of Education Margaret Spellings pointed to the same NAEP data on the law鈥檚 fifth anniversary in January, and during several other recent speeches.
President Bush likes to cite the 鈥渓ong-term-trend鈥� NAEP as proof that the No Child Left Behind Act is working. The gains are significant only for 9- and 13-year-olds in math and 9-year-olds in reading. What鈥檚 more, the gains fall into a five-year testing window, and only two of those years occurred after the law took effect.
*Click image to see the full chart.
Between 1999 and 2004, the reading scores of 9-year-olds climbed from 212 to 226 on the test鈥檚 500-point scale. The gap between African-American and white students that age narrowed to 26 points in 2004, compared with 35 points five years earlier. The gap between Hispanic 9-year-olds and their non-Hispanic white peers tapered from 24 points to 21 points in that same time period.
On the 鈥渘ational鈥� NAEP, meanwhile, researchers say the advances in math reflect a continuation of student-achievement progress since 1990. Fourth graders are dead-even with where they were in reading when the law took effect in 2002. The slight decline in 8th grade reading scores is not statistically significant.
*Click image to see the full chart.
Note: Lighter shading indicates years when testing accommodations weren鈥檛 allowed.
SOURCE: National Center for Education Statistics
On the math test, 9-year-olds鈥� scores rose by 9 points, and the gaps between Hispanics鈥� and African-Americans鈥� scores and whites鈥� scores narrowed slightly as well.
Although the results for 9-year-olds on the reading test are positive, researchers say they can鈥檛 be linked to the law. The testing window extends back to 1999鈥攖hree years before President Bush signed the NCLB legislation into law and even before he was president.
鈥淲ith some of the claims that Spellings has made, for most of the time period there was no NCLB, so she can鈥檛 really say [any improvement] is because of the law,鈥� said Gerald W. Bracey, the author of 69传媒 Educational Research: How to Avoid Getting Statistically Snookered,who runs a LISTSERV, or e-mail forum, tracking what Mr. Bracey calls the administration鈥檚 鈥渄isinformation.鈥�
Mr. Bracey, a frequent critic of testing programs, points out that implementation of the law began in 2002, but didn鈥檛 start to fuel significant change in schools until the 2003-04 school year. 鈥淪o I guess [the Bush administration] should be sharing some of the credit with the Clinton administration,鈥� he said.
In math, the gains since 2002 are the extension of an upward trend that dates back more than 20 years, researchers say.
鈥淭hey just pay attention to what happened after NCLB,鈥� said Jaekyung Lee, an associate professor of education at the State University of New York at Buffalo. 鈥淧art of it is just a continuation of a trend from pre-NCLB.鈥�
The administration appears to ignore other data that suggest the law has had little or no positive effect on achievement.
On a different NAEP exam, gains haven鈥檛 been as significant, Mr. Lee said. What is known as the 鈥渘ational鈥� NAEP, as distinguished from the long-term-trend tests, shows 4th grade reading scores the same in 2005 as three years earlier, when the law was signed. Math scores rose 1 point between 2003 and 2005. While that increase was statistically significant, it was smaller than the 9-point gain between 2000 and 2003.
The scores on the 鈥渘ational鈥� NAEP demonstrate that the NCLB law鈥檚 impact is incomplete, said Katherine McLane, the U.S. Department of Education鈥檚 press secretary.
鈥淭he secretary is the first to say we have more work to do,鈥� Ms. McLane said in response to the criticisms. 鈥淭hat is one of the issues we have to look at in education.鈥�
Regardless of whether NAEP scores go up or down, it鈥檚 almost impossible to link those changes to the NCLB law without a well-designed research study, said Mr. Koretz of Harvard. That would compare a group of students who were exposed to NCLB policies against one that hadn鈥檛 participated in the testing and accountability measures in the law.
Those are the types of studies that the Bush administration says must be presented as evidence to select reading materials for the 69传媒 First program and to win approval for research grants from the department.
Also, scores in the upper grades on both versions of the national assessment are for the most part unchanged from before the law鈥檚 passage.
NAEP is given to a sampling of students nationwide. Scores on states鈥� own tests, however, are used to determine whether schools have made adequate yearly progress under the federal law. Mr. Gamoran of the University of Wisconsin said the debate over NAEP scores is probably irrelevant. Even in 2005, the law鈥檚 most significant policies weren鈥檛 fully phased in. Those include the requirements that all teachers be 鈥渉ighly qualified鈥� and that all states annually assess math and reading achievement in grades 3-8 and once in high school, said Mr. Gamoran, the director of the university鈥檚 Wisconsin Center for Education Research.
鈥�69传媒 First鈥� Results
In addition to speeches citing the NAEP long-term-trend data, members of the Bush administration have lauded the success of the $1 billion-a-year 69传媒 First program, the largest new initiative in the NCLB law.
In the administration鈥檚 blueprint for the reauthorization of the No Child Left Behind Act, unveiled in January, the Education Department described 69传媒 First as 鈥渢he largest, most focused, and most successful early-reading initiative ever undertaken in this country.鈥�
Few disagree that it is the largest and most focused. The initiative, which requires that participating schools use 鈥渟cientifically based鈥� materials and assessments, includes more than 5,600 schools in 1,600 districts. An estimated 100,000 teachers have had some kind of professional development associated with the program, according to the blueprint.
But there is scant empirical evidence showing the program鈥檚 effect on student achievement. An independent interim study on 69传媒 First implementation, released last year, included survey results from state officials. It showed that the program had led to significant increases in the time participating schools spent on reading instruction, as well as more substantive professional development and support for teachers, and the use of assessment data to inform instruction.
A later survey, conducted by the Center on Education Policy, a Washington-based research and advocacy group, indicated that states were generally pleased with the program, with most claiming some improvement in student scores on state tests.
President Bush鈥檚 blueprint includes preliminary results showing some gains in students鈥� reading fluency. 鈥淔or the 2004-05 school year, students in 69传媒 First schools demonstrated increases in reading achievement across all performance measures,鈥� Education Department officials wrote in the blueprint.
鈥淭he percentage of 2nd grade students who met or exceeded proficiency in reading on 69传媒 First outcome measures of fluency increased from 33 percent in 2003-04 to 39 percent in 2004-05 for economically disadvantaged students; from 27 to 32 percent for [limited-English proficient] students; from 34 to 37 percent for African-American students; from 30 to 39 percent for Hispanic students; and from 17 to 23 percent for students with disabilities,鈥� the document adds.
Those gains, however, are based on a compilation of all test results in annual state reports for 69传媒 First.
That compilation includes results from the DIBELS assessment, or Dynamic Indicators of Basic Early Literacy Skills, developed by researchers at the University of Oregon and used in more than 35 states to monitor student progress on fluency and other measures. But they also include results from a variety of other assessments, including the Iowa Test of Basic Skills and Terra Nova.
鈥淭he results show that more kids in the early grades are making great progress on learning the basic components of reading under 69传媒 First,鈥� Ms. McLane, the department鈥檚 press secretary, said of the data reported in the blueprint.
Although such an assemblage of test scores can provide a general view of student progress, some researchers question whether the compilation says much about reading proficiency.
鈥淚f the goal is just to see if students are improving, I think there is nothing wrong with using different tests as long as it is established that the tests are reliable and valid, and reasonably comparable,鈥� Stephen D. Krashen, an education researcher and linguist at the University of Southern California, in Los Angeles, wrote in an e-mail. However, 鈥渕any [researchers] feel that DIBELS is not valid.鈥�
Critics of DIBELS cite the tendency of some educators to teach to the tests or give the measures too much weight in judging reading ability. They also question whether a test that gauges how many words a student can read accurately in a minute, as DIBELS does, is a valid indicator of their proficiency. (鈥淣ational Clout of DIBELS Test Draws Scrutiny,鈥� Sept. 28, 2005.)
According to Mr. Bracey, fluency鈥攖he ability to read a text accurately and quickly鈥攊s not a good indicator of reading mastery, which requires comprehension.
鈥淜ids can be very fluent and not have a clue about what they just read,鈥� he said.
Success of Standards
While most researchers say it鈥檚 too early to measure the NCLB law鈥檚 impact on achievement, many are beginning to see evidence that educators are changing their behavior as a result of both the federal law and policies that took root in the 1990s at the onset of the movement for higher standards and greater accountability in education.
鈥淭he big success of No Child Left Behind so far is to galvanize attention to the challenges we face, particularly the challenges of inequity,鈥� Mr. Gamoran said.
But critics of the law question, in any case, the central place it gives to test scores. They say it puts too much emphasis on the negative consequences of failing to meet annual student-performance targets and glosses over the professional development and other interventions needed to improve struggling schools and get to the heart of elevating student achievement.
鈥淲hat鈥檚 troublesome about it is the idea that you can eliminate [achievement] gaps by putting pressure on schools and nothing else,鈥� said Gary A. Orfield, the director of the Civil Right Project at Harvard and the University of California, Los Angeles. 鈥淚t鈥檚 making a bad situation worse.鈥�
