Includes updates and/or revisions.
A federal advisory panel said yesterday it will urge Congress to 鈥渞efrain from eliminating funding for 69传媒 First,鈥� as House and Senate appropriations subcommittees have recommended, because the panel鈥檚 members believe a recent evaluation has caused confusion and misperceptions about the program鈥檚 effectiveness.
鈥淲e have concerns about the limitations of the study that preclude drawing firm conclusions,鈥� Susan Brady, a member of the 69传媒 First Advisory Committee, said at the meeting. The statement will be forwarded to members of the Senate Appropriations Committee, which is expected this week to make its recommendation for funding the program in the fiscal 2009 federal budget.
It outlines three concerns: 鈥渓imitations surrounding the design, methodology, and analysis of the 69传媒 First Impact Study; unwarranted inferences drawn from the interim report [of that study]; and lack of consideration of other sources of information about the effectiveness of 69传媒 First.鈥�
鈥淐ertainly, any decisions about funding,鈥� the statement continues, 鈥渟hould consider multiple sources of data.鈥�
The House subcommittee鈥檚 version of the budget would kill the program, based in part on the interim report, which found that the federal funding has had no significant impact on students鈥� reading comprehension. The Senate subcommittee that handles education appropriations also voted to end funding for 69传媒 First, but that panel didn鈥檛 cite the interim report. (鈥淪enate Panel Also Votes to Kill Funds for 鈥�69传媒 First鈥�,鈥� June 24, 2008.)
The independent advisory panel of reading researchers, which was appointed by U.S. Secretary of Education Margaret Spellings last year, is charged with reviewing ongoing issues related to the program and making recommendations to the Department of Education.
After a briefing on the here, the members lamented that the findings have been broadly interpreted to mean that, after a $6 billion investment, the program has failed to raise reading achievement. The preliminary results were based on a single measure of reading comprehension, and not the basic, early skills that are the main targets of instruction in the early grades. (鈥溾��69传媒 First鈥� Research Offers No Definitive Answers,鈥� June 4, 2008.)
鈥淚 don鈥檛 think we should misinform the public and lead them to believe that this was a solid study, because for whatever reason, it wasn鈥檛,鈥� said panel member Frank Vellutino, a prominent reading researcher at the State University of New York at Albany. 鈥淚鈥檓 not trying to spin the data, ... but to allow people to spin their own tales, including members of Congress who want to scuttle the program,鈥� is not acceptable, he said.
Past Criticisms Cited
Rep. David R. Obey, D-Wis., the chairman of the House Appropriations Committee, has pushed to slash the program over the past year, citing a series of reports in 2006 and 2007 by the Education Department鈥檚 inspector general. Those reports found that 69传媒 First was mismanaged and that federal officials and consultants involved in the program appeared to have used their influence to promote programs and assessments that they favored.
69传媒 First 鈥渉as been plagued with mismanagement, conflicts of interest, and cronyism, as documented by the inspector general,鈥� Rep. Obey said during the subcommittee meeting. 鈥淢oreover, a scientifically rigorous study released by the Department of Education found that the program has no discernible impact on student reading performance.鈥� (鈥淗ouse Panel Would Kill 鈥�69传媒 First鈥� Funding,鈥� June 19, 2008.)
The long-awaited 69传媒 First Impact Study Interim Report, released May 1, caused a backlash among supporters of the program, who point to district and state test results as evidence that it is having an effect. Much of that data is not comparable across states and has not been subjected to rigorous analysis.
Positive Changes?
Several independent surveys have found that most participating schools have noticed positive changes in instruction and students鈥� test scores. An , conducted by the American Institutes for Research and released by the department last week, shows that a majority of states have seen gains in reading fluency and comprehension in 69传媒 First schools.
The gains were across grades 1, 2, and 3, and most subgroups of students. The analysis is not a rigorous study, however, and averages data that is generally not comparable because all states do not use the same tests and they may set different benchmarks for defining proficiency. The data are from annual performance reports submitted by each state and have not been verified as accurate.
The unusual design of the $31 million interim impact study鈥攁 regression, discontinuity analysis鈥攈as drawn criticism for comparing 69传媒 First schools across 18 sites with nonparticipating schools in the same districts. Some experts have argued that the sample is 鈥渃ontaminated,鈥� because the non-69传媒 First sites have adopted many of the principles and practices of those that are in the program, but have not benefited from annual federal grants.
鈥楬it the Pause Button鈥�
Many of the panel members suggested that Congress needs to have additional information, including data that paint a more positive picture of how the program is going. But they also stated that they wanted their input to be neutral, and to maintain the panel鈥檚 advisory role without crossing over into advocacy. Several members pressed for immediate action, however.
鈥淐an we march over to the Senate鈥� and present the committees with additional information, asked Ms. Brady, a panel member and a senior scientist at the Haskins Laboratory in New Haven, Conn.
鈥淯nless Congress gets different information, this program is going to go by the wayside,鈥� said Laurie E. Cutting, the director of the education- and brain-research program at the Kennedy Krieger Institute in Baltimore. She noted the the committee had to prepare its statement quickly so that it might have an influence on the budget discussions. 鈥淐an鈥檛 we say, 鈥楬it the pause button please; it鈥檚 not really appropriate to be making these decisions yet.鈥� 鈥�
Several members said they were frustrated that the interim study was having such an effect on decisionmakers, while a trove of state data has yet to be mined. Moreover, they said, a draft of the final impact report, which includes additional data on student achievement and teacher practice, is complete and going through peer review, yet not set for release until the fall.
鈥淭he timing of this is so off,鈥� said Nonie K. Lesaux, a professor of human development and urban education advancement at Harvard University. 鈥淚t鈥檚 crazy to think that the final report is drafted and sitting somewhere while its fate is being debated.鈥�
All the members agreed it was appropriate to alert federal lawmakers that they think it is premature to make significant decisions about the program until more evidence is weighed.
The panel will form subcommittees to draft an in-depth analysis of the study鈥檚 findings, and to clarify for Congress and the public what it does and doesn鈥檛 say about the program鈥檚 effectiveness.
