With political change coming soon to the nation’s capital, policymakers and national groups are trying to divine what the shifts might mean for the U.S. Department of Education’s long-running, and sometimes controversial, campaign to transform education into an “evidence based” field.
The movement to promote more scientifically rigorous research in education predates President Bush’s tenure by a year or more.
But experts agree that the Bush administration and Congress gave it a mighty push, first by enacting the No Child Left Behind Act, which calls more than 100 times for basing education decisions on “scientifically based research,” and then by reorganizing the Education Department’s research arm into the Institute of Education Sciences, which has led the charge for toughening research standards.
Now, both the No Child Left Behind law and the , the 2002 law that gave birth to the IES, are up for renewal. The Oval Office will change hands next January, and the six-year term of Grover J. “Russ” Whitehurst, the IES’ first director, ends in November.
“I think we’re in the midst of a perfect storm brewing,” James W. Kohlmoos, the president of the Knowledge Alliance, a Washington-based organization that represents a wide range of research groups, said at a here on the topic. The conference, sponsored by the American Enterprise Institute, a Washington think tank, was one of two held in recent weeks to focus on what’s ahead for federal education research.
Redefining Proof
One sign that changes may be afoot is the proliferation of proposals to revise the way the federal government defines scientifically based research in education. The NCLB definition emphasizes randomized experiments over other kinds of research, prompting some critics to complain that it promotes too narrow a view of education scholarship. (“‘Scientific’ Label in Law Stirs Debate,” Oct. 17, 2007.)
Widely used in medicine, randomized controlled trials are experiments in which subjects are randomly assigned to either the experiment group or a business-as-usual condition. Experts consider such studies the “gold standard” for seeing whether an intervention works. But randomized trials are rarer in education and hard to conduct in some school settings.
By comparison, newer definitions for “scientifically valid” research say that studies that try to make strong cause-and-effect claims may “include but not be limited to random-assignment experiments.” Such definitions popped up in legislation for renewing the Higher Education Act and the federal Head Start program for disadvantaged preschoolers, as well as in an early discussion draft for reauthorizing the NCLB law.
“We can’t be constrained solely by quasi-experimental and random-assignment studies in education,” Roberto Rodriguez, a senior education adviser to U.S. Sen. Edward M. Kennedy, D-Mass., the chairman of the Senate Health, Education, Labor and Pensions Committee, told participants at a Feb. 21 panel discussion on Capitol Hill. “I also think we have to be more conscious about it and ask ourselves how we’re using that standard and in what context. We may need to have different standards in different environments.”
However, an Education Department official who spoke at the same forum cautioned against going too far in loosening up federal research standards. He said such a step could leave educators to choose programs and products on the basis of fad and anecdote, rather than solid research.
“Unless we bring in rigor, we’re not going to bring in really scientific advances,” said Williamson Evers, the assistant secretary for planning, evaluation, and policy development at the Education Department.
Congress Steps In
Another sign of a brewing backlash to the evidence-based movement came recently when Congress succeeded in scuttling a $5 million random-assignment evaluation of the federal Upward Bound program, which provides college assistance to students from disadvantaged families. Program advocates opposed the random-assignment study, which was already under way, because it would deny services to students after they had undergone an elaborate application process.
The congressional action came in the fiscal 2008 omnibus spending bill that President Bush signed into law in late December, which effectively bars any funding of the evaluation in this fiscal year. Furthermore, both the House and Senate versions of bills to renew the Higher Education Act contain language that would prohibit such an evaluation in the future. Those bills, which differ on other issues, are awaiting reconciliation.
The Education Department, which had originally signaled it would either retool the study or press on with it, notified grantees on Feb. 21 of its plans drop the study.
Proponents of the evidence-based movement, including Mr. Whitehurst of the IES, see the congressional ban on the randomized study as a dangerous precedent. “To have a program put outside the realm of collection of data on its effectiveness ultimately puts that program at risk,” Mr. Whitehurst said at the Feb. 7 meeting, which took place before the House voted that day to pass its version of the HEA bill.
“It is unfortunate, but is that more unfortunate than having the federal government dictate how scientific research should be done?” said Gerald E. Sroufe, the director of government relations for the Washington-based American Educational Research Association, which favors a broader conceptualization of education research than the language used in the NCLB law.
Though his 25,000-member group quibbles with some existing definitions of “scientifically based” research, Mr. Sroufe said the organization would like to see the current IES structure remain mostly intact when Congress gets around to renewing the federal authorization for it—a task that lawmakers are not likely to take up until after the NCLB law is reauthorized.
“There’s a lot of value to the system we have now,” he added. “I do not want to start over and have another authorization like NIE [the National Institute of Education] or OERI [the office of educational research and improvement],” referring to previous incarnations of the federal education research agency.
Other groups favor strengthening some ties between the Education Department and the IES. The research agency was set up to be more independent than its predecessor to protect against potential political bias. Mr. Kohlmoos of the Knowledge Alliance, however, said greater cooperation could help make education research more usable and relevant to educators and policymakers.
“There’s a moat between the Department of Education now and IES, and I think the department needs to build some bridges there,” he said at the American Enterprise Institute meeting.
Innovation Eyed
If any changes are in store for the federal education research enterprise, most experts hope one of them will be an increase in federal support for innovation and for developing applications for research that educators can put into practice fast.
“There has been a real lack of federal support and energy around innovation for the last seven years,” Glenn Kleiman, the executive director of the Friday Institute for Educational Innovation at North Carolina State University, in Raleigh, said at the Feb. 21 discussion. The event was organized by the Knowledge Alliance, the Software & Information Industry Association, and the Association of American Publishers, three Washington groups with a big stake in the debate on evidence-based research.
“We have to figure out how to remove barriers that are keeping schools from participating in new programs,” Mr. Kleiman said.
One of those barriers, educators at that conference said, has been the accountability provisions of the NCLB law, which mandate sanctions for schools that fail year after year to raise students’ test scores.
“In schools that do not perform well, … principals can lose their jobs,” said Wesley L. Boykin, the executive director of research, accountability, and assessment for the Baltimore County, Md., public schools. “Would you want your superiors to fire you if you were innovative and it failed?”
The research-and-development budget of the U.S. Department of Education falls far short of the total R&D budgets, shown below in millions of dollars and including facilities, of many other federal agencies.
SOURCE: American Association for the Advancement of Science
Educational innovation is also hampered by a lack of federal funding, some experts say. According to the American Association for the Advancement of Science, the Education Department’s $321 million budget for research and development in fiscal 2008 is a small fraction of those of most other federal agencies.
“One of the greatest contributions the federal government can make toward innovation would be an increase in funding for research and development in education,” said Nina S. Rees, a former assistant deputy secretary for innovation and improvement at the Education Department who is now the senior vice president of strategic initiatives for Knowledge Universe Education, a for-profit education services company based in Menlo Park, Calif.
Trials Gain Traction
Yet to assume that pending political changes could undo the evidence-based movement is to ignore the inroads that it’s made during this decade, proponents have noted. In a report to Congress last May, for instance, the IES listed 23 large-scale, federally sponsored evaluations under way, 18 of which are randomized studies. That is a significant increase from 2000, the report said, when just one of the evaluations under way was a randomized study.
“These are not only showing things that don’t work,” said Jon Baron, the executive director of the Washington-based Coalition for Evidence-Based Policy. “A few are also showing things that do work, that do produce important educational outcomes, and positive examples are critical to keep the momentum going.”
The omnibus 2008 spending bill included language strongly supporting federal evaluations “using rigorous methodologies, particularly random assignment, that are capable of producing scientifically valid knowledge regarding which program activities are effective,” and endorsed the IES as the lead agency handling those evaluations.
“Policymakers have passed a tipping point in understanding the need for education to be built on good evidence and research,” Mr. Whitehurst said in an e-mail.
Although nothing bars President Bush or his successor from nominating Mr. Whitehurst to another six-year term as IES director, he has not publicly expressed interest in continuing in the job.
The research agency is also underwriting interdisciplinary research-training programs in the education sciences for 160 predoctoral students at 10 universities, in an effort to build some future capacity in the field.
“It is the work of a generation that we’re involved in,” Mr. Whitehurst told participants at the AEI forum last month, “not the work of a few years.”
