69传媒 taking part in the federal 69传媒 First program are showing significant progress in boosting students鈥� reading fluency and comprehension, according to state-reported data compiled by the U.S. Department of Education and released last week.
In releasing for the first time detailed, multiyear data on how 69传媒 First schools are performing on key measures, federal officials hailed the results as solid evidence that the $1 billion-a-year initiative is working.
鈥淲e are very pleased with the outcomes we鈥檝e seen from this data,鈥� said Amanda Farris, a deputy assistant secretary for elementary and secondary education in the department. 鈥淭his shows tremendous gains for our neediest students in our neediest schools.鈥�
But some observers questioned whether the data should be used to generalize about the program鈥檚 impact on students鈥� reading skills. They also noted the timing of the report鈥檚 release, on the eve of what was expected to be a contentious congressional hearing late last week into allegations of mismanagement and conflict of interest in the program.
The analysis of test results from about half the states鈥攖hose that reported baseline data on participating schools鈥攕hows about a 15 percent improvement in the proportion of 1st, 2nd, and 3rd graders who can read fluently, meaning accurately and at an appropriate rate. In measures of reading comprehension, those states averaged about a 12 percent increase in the number of 3rd graders who were deemed proficient. 69传媒 in most subgroups also saw gains.
The report does not include information on the progress of schools that are similar to those in the program, making it difficult to attribute the gains to the federal initiative.
View the , posted by the .
鈥淭here are some small gains, yes. But are they larger than gains in non-69传媒 First schools?鈥� said Richard A. Allington, a professor of education at the University of Tennessee in Knoxville. 鈥淲e don鈥檛 know whether improvements are related to the 69传媒 First model or to general improvement trends across all schools.鈥�
Improvement Reported
The thick binder of state-level summaries released by the Education Department on April 19 includes test results for 1st through 3rd graders from 2002 to 2006. Results are broken down into subgroups by race and socioeconomic status, as well as for English-language learners and students with disabilities.
The federal 69传媒 First program requires states to submit annual performance reports, which include student-achievement data in reading fluency and comprehension for grades 1-3. States use various tests to gauge students鈥� proficiency, so the results are not comparable across the states. Below are selected state results:
*Data from 2005
SOURCE: U.S. Department of Education
To measure reading fluency, most of the states relied on an assessment called the Dynamic Indicators of Basic Early Literacy Skills, or DIBELS, which scores how many words in a grade-level passage students can read in one minute. States set their own benchmarks for determining proficiency, so the scores are not comparable across states. For assessing comprehension, or students鈥� understanding of what they read, states used a variety of tests, including the Iowa Tests of Basic Skills, the Stanford Achievement Test, the TerraNova, and state exams. In most states, students showed significantly more improvement in fluency than in comprehension.
Some states reported that most of their 69传媒 First districts showed improvements of at least 5 percentage points in the two categories. California, for example, reported that nearly three-fourths of its 69传媒 First districts showed significantly better rates of proficiency in comprehension among 2nd graders, while 37 percent of such districts did so for 3rd graders. Oregon officials, meanwhile, reported that all their participating districts had shown improvements in reading fluency among 2nd graders, while nearly 80 percent did for comprehension at that grade level.
69传媒 First was authorized as part of the No Child Left Behind Act, which President Bush signed into law five years ago, to ensure that struggling schools had access to research-based programs, assessments, and teacher professional development in the subject. Most states began receiving their share of annual funding in 2002 and 2003.
The program has been under scrutiny by federal auditors, who have been responding to complaints from several commercial vendors that federal program officials and consultants favored particular reading textbooks, assessments, and approaches over others and directed state officials to use certain products, which the NCLB law prohibits.
The Education Department鈥檚 inspector general largely substantiated those claims, as did a separate review by the Government Accountability Office, the investigative arm of Congress. (鈥�69传媒 Probe Will Continue on Capitol Hill,鈥� April 4, 2007.)
The House Education and Labor Committee conducted an April 20 hearing into the program, with witnesses who included Christopher J. Doherty, the former 69传媒 First director in the Education Department, and Edward J. Kame鈥檈nui, a prominent former consultant to 69传媒 First who is now the commissioner of the department鈥檚 National Center for Special Education Research.
The department鈥檚 inspector general, John P. Higgins Jr., who oversaw the six recent reports his office has released on the program, also testified.
Timing Questioned
69传媒 First has earned praise, however, from state officials for providing the kind of money, other resources, and technical assistance they say is necessary to fuel significant changes in reading instruction and achievement.
Two reports released last year, one commissioned by the Education Department and another by an independent policy group, found that most states were satisfied with the program and reported achievement gains in participating schools.
鈥淧articipating schools and districts have made many changes in reading curriculum, instruction, assessment, and scheduling,鈥� the report by the Washington-based Center on Education Policy said. 鈥淢any districts have expanded 69传媒 First instructional programs and assessment systems to non-69传媒 First schools.鈥�
Neither of the reports, though, included test-score data or other empirical information to show the program鈥檚 impact on students鈥� reading skills.
Such data have generally been unavailable because many states did not start reporting test results until 2004. The Institute for Education Sciences, the Education Department鈥檚 research arm, is in the midst of a study of 69传媒 First that will analyze the state data and compare those findings with test results from a control group of other schools that are also in the federal Title I program for disadvantaged students. The results of that study are due out next year.
The state data鈥攔eported annually as required under the 69传媒 First program鈥攈ave not been easily accessible until now. In previous years, the data were closely held by Education Department officials and not released to the public at large. Over the past several years, Education Week has been allowed to review the state performance reports and lengthy data summaries only after repeated requests.
Members of Congress have apparently also found it difficult to gain access to such information. In response to last week鈥檚 data release, Rep. George Miller, D-Calif., sent a letter to Secretary of Education Margaret Spellings asking why the department had not released information on state 69传媒 First programs until this month, even though it was requested by the House education committee several months ago.
The committee, which Rep. Miller chairs, held its investigative hearing last Friday into charges that 69传媒 First was mismanaged, and that federal officials overstepped their authority in directing states on their choice of reading texts and assessments in participating districts.
鈥淚t is obvious that a great deal of time went into preparing and formatting this report,鈥� Rep. Miller wrote in his letter to Ms. Spellings. 鈥淚t is therefore clear that much of the information requested by committee investigators was available earlier than the date it was provided.鈥�
Education Department officials said last week that they have been working to make more information on 69传媒 First available to the public and chose to release the state performance results as a first step.
But while some of the raw results, along with a fact sheet and press release drafted by the department, appeared to strengthen officials鈥� contentions that 69传媒 First is working, some experts said they were not convinced that such conclusions could be drawn from the available data.
鈥淭he information that they are reporting doesn鈥檛 really support the notion that this program has had an effect, but that鈥檚 not to say that it hasn鈥檛鈥攖here just isn鈥檛 the data to support that claim,鈥� said John A. Nunnery, an assistant professor of education leadership and counseling at Old Dominion University, in Norfolk, Va., who teaches graduate courses on research methods and program evaluation. 鈥淚f you look at the statewide results in some of those states, they had similar or better gains as a whole. So everyone is going up.鈥�
