A just-launched project will review education reports released by private think tanks for the quality of their research, methodology, and conclusions, using expert academic reviewers.
The Think Tank Review Project is a joint collaboration of the Arizona State University Education Policy Studies Laboratory and the University of Colorado’s Education and Public Interest Center. It has pledged to provide “expert reviews” to policymakers, the news media, and the public within two weeks of a report’s release. The first such reviews—of three reports released since January—were posted online last week at .
The project reflects a growing concern among researchers that studies on hot-button issues increasingly appear in public without having undergone an independent peer review by scholars. The project’s creators particularly hope to provide a counterpoint to what they view as a proliferation of education reports coming from think tanks that were founded to advance particular ideological points of view. (“Research: Researching the Researchers,” Feb. 20, 2002.)
View from Arizona State University’s .
“Despite garnering media attention and their influence with many legislators, reports released by private think tanks are often of very poor quality,” Alex Molnar, the director of the Education Policy Studies Laboratory, said in a press release announcing the project.
“Many think tank reports are little more than ideological argumentation dressed up as research,” he said. “We believe that policymakers, reporters, and the public will greatly benefit from having qualified social scientists provide reviews of these documents in a timely fashion.”
Neutrality Debated
Ellen Condliffe Lagemann, a professor of education at Harvard University’s graduate school of education and a former president of the Chicago-based Spencer Foundation, which focuses on education research, called the project a “much-needed development.”
“It’s very difficult for laypeople to react critically to the claims made in reports published by these organizations,” she said in an e-mail. “We all need help in understanding some of the complicated design issues that arise in evaluating programs, judging the significance of different test scores, and the like.”
But the project itself has raised eyebrows about whether it will be bias-free. Its funding comes from the Great Lakes Center for Education Research and Practice, an East Lansing, Mich.-based group founded by the Michigan Education Association, an affiliate of the National Education Association. (“Union-Funded Study Finds Fault With High-Stakes Testing,” Sept. 28, 2005.)
While it’s useful and appropriate for researchers and advocates to monitor and critique each other’s work, said Frederick M. Hess, the director of education policy studies at the conservative American Enterprise Institute in Washington and the executive editor of Education Next,the Arizona State scholars “are more than a little ideological in their own right.”
“Even aside from the issue of funding, these are individuals who have clearly articulated policy preferences,” he said. “While that’s fine, they’re certainly not neutral authorities. I’d hope that media treat these ‘critiques’ accordingly and carefully weigh their claims.”
Kevin Welner, the director of the Education and Public Interest Center at the University of Colorado at Boulder, said: “Each of our reviewers is an independent scholar. I do not think the reviewers even know the source of the funding. There is absolutely no pressure being exerted by the Great Lakes Center on the content of the reviews.”
The project’s first three reviews cover a report by the Cato Institute and the Milton & Rose D. Friedman Foundation, on the District of Columbia voucher program; a report by the Manhattan Institute for Policy Studies evaluating Florida’s plan to end social promotion, which later appeared as an article in Education Next; and a report by the Wisconsin Policy Research Institute on the status of high school education in that state.
