Is the No Child Left Behind Act working?
President Bush says it is, pointing to student-achievement results from a single subsection of the National Assessment of Educational Progress and tentative 69´ŤĂ˝ First data. But the evidence available to support his claim is questionable.
âFourth graders are reading better,â the president said during a March 2 visit to a school in New Albany, Ind. âTheyâve made more progress in five years than the previous 28 years combined.â
In mathematics, he said, elementary and middle school students âearned the highest scores in the history of the test.â
The data Mr. Bush cited at that event are from just the âlong-term trendâ NAEP in reading and math, researchers say. All available data, they add, show modest improvements that canât be attributed to the 5-year-old law. Instead, progress in achievement is more likely a continuation of trends that predate the law.
âThereâs not any evidence that shows anything has changed,â said Daniel M. Koretz, a professor of education at Harvard Universityâs graduate school of education.
Other researchers suggest that the standards and accountability system of the NCLB law is drawing attention to achievement gaps and other inequalities and is causing educators to change their practice. But itâs too early to say whether the federal law will result in achievement gains, they contend.
The lawâs âmechanisms are just coming into play, and not enough time has passed to establish a trend,â said Adam Gamoran, a professor of sociology at the University of Wisconsin-Madison.
âIâm Lobbying Congressâ
Portraying the No Child Left Behind law as a success is a critical element in President Bushâs argument that Congress should renew it on schedule this year. The president signed the legislation, an overhaul of the Elementary and Secondary Education Act, with much fanfare in January 2002 and has cited it as his most important accomplishment in domestic policy.
âIâm not only speaking to you, Iâm lobbying,â Mr. Bush said at the Silver Street Elementary School in New Albany earlier this month. âIâm lobbying Congress. Iâm setting the stage for Congress to join me in the reauthorization of this important piece of legislation.â
Congress is laying the groundwork for reauthorizing the measure. This week, the Senate education committee held a hearing on the lawâs teacher-quality requirements. Next week, the House and Senate education committees plan to hold a joint session on an overview of the law.
Rep. George Miller, D-Calif., and Sen. Edward M. Kennedy, D-Mass., the chairmen of the education committees and two of the architects of the bipartisan law, say they hope to renew it this year. But many observers expect the process will be delayed until next year or even after Mr. Bush leaves office in 2009.
At the New Albany school, Mr. Bush highlighted the gains on the national assessmentâs long-term-trend tests in reading and mathematics. Secretary of Education Margaret Spellings pointed to the same NAEP data on the lawâs fifth anniversary in January, and during several other recent speeches.
President Bush likes to cite the âlong-term-trendâ NAEP as proof that the No Child Left Behind Act is working. The gains are significant only for 9- and 13-year-olds in math and 9-year-olds in reading. Whatâs more, the gains fall into a five-year testing window, and only two of those years occurred after the law took effect.
*Click image to see the full chart.
Between 1999 and 2004, the reading scores of 9-year-olds climbed from 212 to 226 on the testâs 500-point scale. The gap between African-American and white students that age narrowed to 26 points in 2004, compared with 35 points five years earlier. The gap between Hispanic 9-year-olds and their non-Hispanic white peers tapered from 24 points to 21 points in that same time period.
On the ânationalâ NAEP, meanwhile, researchers say the advances in math reflect a continuation of student-achievement progress since 1990. Fourth graders are dead-even with where they were in reading when the law took effect in 2002. The slight decline in 8th grade reading scores is not statistically significant.
*Click image to see the full chart.
Note: Lighter shading indicates years when testing accommodations werenât allowed.
SOURCE: National Center for Education Statistics
On the math test, 9-year-oldsâ scores rose by 9 points, and the gaps between Hispanicsâ and African-Americansâ scores and whitesâ scores narrowed slightly as well.
Although the results for 9-year-olds on the reading test are positive, researchers say they canât be linked to the law. The testing window extends back to 1999âthree years before President Bush signed the NCLB legislation into law and even before he was president.
âWith some of the claims that Spellings has made, for most of the time period there was no NCLB, so she canât really say [any improvement] is because of the law,â said Gerald W. Bracey, the author of 69´ŤĂ˝ Educational Research: How to Avoid Getting Statistically Snookered,who runs a LISTSERV, or e-mail forum, tracking what Mr. Bracey calls the administrationâs âdisinformation.â
Mr. Bracey, a frequent critic of testing programs, points out that implementation of the law began in 2002, but didnât start to fuel significant change in schools until the 2003-04 school year. âSo I guess [the Bush administration] should be sharing some of the credit with the Clinton administration,â he said.
In math, the gains since 2002 are the extension of an upward trend that dates back more than 20 years, researchers say.
âThey just pay attention to what happened after NCLB,â said Jaekyung Lee, an associate professor of education at the State University of New York at Buffalo. âPart of it is just a continuation of a trend from pre-NCLB.â
The administration appears to ignore other data that suggest the law has had little or no positive effect on achievement.
On a different NAEP exam, gains havenât been as significant, Mr. Lee said. What is known as the ânationalâ NAEP, as distinguished from the long-term-trend tests, shows 4th grade reading scores the same in 2005 as three years earlier, when the law was signed. Math scores rose 1 point between 2003 and 2005. While that increase was statistically significant, it was smaller than the 9-point gain between 2000 and 2003.
The scores on the ânationalâ NAEP demonstrate that the NCLB lawâs impact is incomplete, said Katherine McLane, the U.S. Department of Educationâs press secretary.
âThe secretary is the first to say we have more work to do,â Ms. McLane said in response to the criticisms. âThat is one of the issues we have to look at in education.â
Regardless of whether NAEP scores go up or down, itâs almost impossible to link those changes to the NCLB law without a well-designed research study, said Mr. Koretz of Harvard. That would compare a group of students who were exposed to NCLB policies against one that hadnât participated in the testing and accountability measures in the law.
Those are the types of studies that the Bush administration says must be presented as evidence to select reading materials for the 69´ŤĂ˝ First program and to win approval for research grants from the department.
Also, scores in the upper grades on both versions of the national assessment are for the most part unchanged from before the lawâs passage.
NAEP is given to a sampling of students nationwide. Scores on statesâ own tests, however, are used to determine whether schools have made adequate yearly progress under the federal law. Mr. Gamoran of the University of Wisconsin said the debate over NAEP scores is probably irrelevant. Even in 2005, the lawâs most significant policies werenât fully phased in. Those include the requirements that all teachers be âhighly qualifiedâ and that all states annually assess math and reading achievement in grades 3-8 and once in high school, said Mr. Gamoran, the director of the universityâs Wisconsin Center for Education Research.
â69´ŤĂ˝ Firstâ Results
In addition to speeches citing the NAEP long-term-trend data, members of the Bush administration have lauded the success of the $1 billion-a-year 69´ŤĂ˝ First program, the largest new initiative in the NCLB law.
In the administrationâs blueprint for the reauthorization of the No Child Left Behind Act, unveiled in January, the Education Department described 69´ŤĂ˝ First as âthe largest, most focused, and most successful early-reading initiative ever undertaken in this country.â
Few disagree that it is the largest and most focused. The initiative, which requires that participating schools use âscientifically basedâ materials and assessments, includes more than 5,600 schools in 1,600 districts. An estimated 100,000 teachers have had some kind of professional development associated with the program, according to the blueprint.
But there is scant empirical evidence showing the programâs effect on student achievement. An independent interim study on 69´ŤĂ˝ First implementation, released last year, included survey results from state officials. It showed that the program had led to significant increases in the time participating schools spent on reading instruction, as well as more substantive professional development and support for teachers, and the use of assessment data to inform instruction.
A later survey, conducted by the Center on Education Policy, a Washington-based research and advocacy group, indicated that states were generally pleased with the program, with most claiming some improvement in student scores on state tests.
President Bushâs blueprint includes preliminary results showing some gains in studentsâ reading fluency. âFor the 2004-05 school year, students in 69´ŤĂ˝ First schools demonstrated increases in reading achievement across all performance measures,â Education Department officials wrote in the blueprint.
âThe percentage of 2nd grade students who met or exceeded proficiency in reading on 69´ŤĂ˝ First outcome measures of fluency increased from 33 percent in 2003-04 to 39 percent in 2004-05 for economically disadvantaged students; from 27 to 32 percent for [limited-English proficient] students; from 34 to 37 percent for African-American students; from 30 to 39 percent for Hispanic students; and from 17 to 23 percent for students with disabilities,â the document adds.
Those gains, however, are based on a compilation of all test results in annual state reports for 69´ŤĂ˝ First.
That compilation includes results from the DIBELS assessment, or Dynamic Indicators of Basic Early Literacy Skills, developed by researchers at the University of Oregon and used in more than 35 states to monitor student progress on fluency and other measures. But they also include results from a variety of other assessments, including the Iowa Test of Basic Skills and Terra Nova.
âThe results show that more kids in the early grades are making great progress on learning the basic components of reading under 69´ŤĂ˝ First,â Ms. McLane, the departmentâs press secretary, said of the data reported in the blueprint.
Although such an assemblage of test scores can provide a general view of student progress, some researchers question whether the compilation says much about reading proficiency.
âIf the goal is just to see if students are improving, I think there is nothing wrong with using different tests as long as it is established that the tests are reliable and valid, and reasonably comparable,â Stephen D. Krashen, an education researcher and linguist at the University of Southern California, in Los Angeles, wrote in an e-mail. However, âmany [researchers] feel that DIBELS is not valid.â
Critics of DIBELS cite the tendency of some educators to teach to the tests or give the measures too much weight in judging reading ability. They also question whether a test that gauges how many words a student can read accurately in a minute, as DIBELS does, is a valid indicator of their proficiency. (âNational Clout of DIBELS Test Draws Scrutiny,â Sept. 28, 2005.)
According to Mr. Bracey, fluencyâthe ability to read a text accurately and quicklyâis not a good indicator of reading mastery, which requires comprehension.
âKids can be very fluent and not have a clue about what they just read,â he said.
Success of Standards
While most researchers say itâs too early to measure the NCLB lawâs impact on achievement, many are beginning to see evidence that educators are changing their behavior as a result of both the federal law and policies that took root in the 1990s at the onset of the movement for higher standards and greater accountability in education.
âThe big success of No Child Left Behind so far is to galvanize attention to the challenges we face, particularly the challenges of inequity,â Mr. Gamoran said.
But critics of the law question, in any case, the central place it gives to test scores. They say it puts too much emphasis on the negative consequences of failing to meet annual student-performance targets and glosses over the professional development and other interventions needed to improve struggling schools and get to the heart of elevating student achievement.
âWhatâs troublesome about it is the idea that you can eliminate [achievement] gaps by putting pressure on schools and nothing else,â said Gary A. Orfield, the director of the Civil Right Project at Harvard and the University of California, Los Angeles. âItâs making a bad situation worse.â
