Although sharing is one of the first lessons children learn in school, a few prominent education scholars are suggesting some of their colleagues ought to take a refresher course in the subject.
The issue: Investigators are often reluctant to share data from their studies so other researchers can reanalyze the information, replicate the studies, or expand on the findings.
鈥淲e need cooperation and collaboration and sharing in a way we haven鈥檛 in the past,鈥� said Barbara Schneider, a sociologist at the University of Chicago. 鈥淲e鈥檙e in the heart of education reform, and we have tremendous amounts of resources being funneled into a lot of experiments.鈥�
|
Ms. Schneider, who is a senior social scientist at the university鈥檚 National Opinion Research Center, has led some of the calls for researchers to be more forthcoming with the data they collect鈥攑articularly when the federal government is paying for their work.
She has been joined by other scholars, foundation officials, and policymakers, all of whom contend that their fellow researchers need to 鈥渟hare and share alike鈥� for the field to amass a credible store of knowledge on what works in education.
The calls for collaboration still do not represent a consensus, however. Some researchers have cautioned against pushing too hard and fast to make data-sharing a standard practice. That鈥檚 especially true, they add, when research subjects鈥� privacy is at stake.
鈥淚t would absolutely kill ethnography,鈥� said Mary Haywood Metz, a University of Wisconsin-Madison researcher who specializes in qualitative studies looking at the internal social and emotional workings of school communities. While it鈥檚 admirable that some individual researchers have taken it upon themselves to make their data public after their papers are published, she said, 鈥渢o set that up as a standard for everybody to follow is quite another matter.鈥�
Privacy High Wire
A panel organized by the National Research Council, an arm of the National Academies of Sciences, is exploring ways to make education research more 鈥渟cientific.鈥� The group, known as the Committee on Research in Education, was interested enough in the issue of data-sharing to include it in a workshop last summer on accumulating a knowledge base in education.
鈥淭hey spend many years collecting their data, and they want to make sure they have first crack at it,鈥� Kay Dickerson, a Brown University professor of community health and member of the panel, said of researchers鈥� attitudes. 鈥淥n the other hand, sometimes researchers take a long time getting to it, and that鈥檚 frustrating.鈥�
Sometimes, too, information goes unused because researchers鈥� grants do not cover the cost of 鈥渃leaning up鈥� the data for public use. That means stripping away any information that could be used to identify study subjects and adding in enough context to make the data understandable.
Protecting subjects鈥� privacy, though, remains the main reason most researchers avoid opening up their files to outside inspection. In qualitative studies and small-scale quasi-experiments, researchers say, it鈥檚 hard to disguise everyone鈥檚 identity. A school might have just one art teacher, for instance. Or a small city might have a single high school principal.
Also, if subjects know that their views and actions will be widely shared, the University of Wisconsin鈥檚 Ms. Metz said, they might be less forthcoming鈥攐r refuse to participate in the study altogether.
鈥淧eople won鈥檛 tell you what鈥檚 going on unless they can trust you,鈥� she noted.
On the other hand, she said, when researchers remove too much identifying information, they risk losing some of the context that outside scholars need to understand what鈥檚 being studied.
Such debates are not unique to education.
Just last year, the federal National Institutes of Health, after vetting the opinions of researchers in the health fields, issued new rules requiring grant recipients to make all their research materials available to the public once their studies are accepted for publication.
Likewise, the Washington-based American Sociological Association鈥攁 group that also includes education researchers鈥� revised its ethics code in the mid-1990s to explicitly encourage its members to share their data with colleagues.
|
鈥淚 don鈥檛 necessarily see educational research as behind in this,鈥� said Felice Levine, the executive director of the American Educational Research Association, which also is located in Washington. Ms. Levine was the executive director of the sociological association when its ethics code was updated, and she served on a National Science Foundation panel on data-sharing.
In her current organization, she said, such policies are implicit in the 鈥渘ormative fabric鈥欌�� of the ethics code. 鈥淐ould a code be more explicit?鈥� she said. 鈥淚t might be.鈥�
Publication Pressures
Regardless of the status of data-sharing in other fields, Ms. Schneider of the University of Chicago and others suggest that the practice could be more ingrained in education research than it is now.
They offered this case in point: Researchers who publish in the journals Nature, Science, the American Sociological Review, the American Economic Review, and Demography all must agree to share their work when their articles are published. Researchers said few, if any, education research journals have similar requirements.
Similarly, the U.S. Department of Justice鈥檚 National Institute of Justice requires the researchers it supports to make their data public.
The U.S. Department of Education has no such blanket requirements for its researchers. That鈥檚 not to say that no department-financed studies are made widely available. All the data collected by the department鈥檚 National Center for Education Statistics or one of its contractors have long been open to the public. And the department鈥檚 Institute of Education Sciences, newly required by federal law to do the same, is making plans to follow NCES鈥� example.
Some of those federal data sets, though, are still a year or two from being widely available online or on CD-ROMs, according to Grover J. 鈥淩uss鈥� Whitehurst, the institute鈥檚 director.
Likewise, Ms. Schneider is spearheading a project to develop a common database from all the information gathered from a federally funded, cross-agency research initiative on education.
Mr. Whitehurst said he doubts, however, that the Education Department will ever be able to demand that the individual researchers it subsidizes through the field-initiated-studies grant program or other smaller-scale research efforts put up their data for all to use.
While data-sharing may be a good way to get the 鈥渂iggest bang for the buck鈥� out of federal research expenditures, he said, 鈥渢here are, in most cases, severe confidentiality issues that constrain distribution of that data.鈥� But when it comes to informing public policy, some researchers say timely sharing of data becomes a moral obligation.
鈥淚f researchers want to hoard data, that鈥檚 fine, but I think the rest of the policy community should ignore the results until someone else has had the opportunity to analyze the data,鈥� said Alan B. Krueger, a Princeton University economist who has sought access to data from school voucher experiments in New York City, the District of Columbia, and Dayton, Ohio.
Mr. Krueger said his own attempts to reanalyze the data from the three-city school choice experiments have been 鈥渆ye-opening.鈥� Although results from those experiments were released last year, the Princeton economist could not persuade Harvard University researcher Paul E. Peterson and his colleagues to part with their data from Dayton and Washington.
But Mathematica, the Princeton, N.J.-based research group that had worked with Mr. Peterson on his New York study, complied. Mr. Krueger reanalyzed the data using a different method for categorizing students by race. And he came up with some distinctly different results. (鈥淪tudy: No Academic Gains From Vouchers for Black 69传媒,鈥� April 9, 2003.)
In an e-mail message, Mr. Peterson explained that he had declined to share the data because members of his research team are still publishing more papers from the results.
In its earlier analysis, the Mathematica-Peterson research team found that the privately financed tuition vouchers had led to sizable learning gains for the African-American students who used them to transfer to private schools. They outperformed a similar group of disadvantaged students who had also sought the vouchers, but didn鈥檛 win them in the lottery. Mr. Krueger鈥檚 refiguring, however, found those gains statistically insignificant.
That reanalysis did not end the debate on the benefits of school choice. Mr. Peterson continues to dispute Mr. Krueger鈥檚 conclusions. Experts say the example does, however, point up the need to cast a second eye on controversial studies that might be used to inform public policy.
Stepping up cooperation among researchers is important, they argue, because it offers another possible route for improving the quality of educational research.
One such proponent is Marshall S. Smith, the director of educational programs for the Menlo Park, Calif.-based William and Flora Hewlett Foundation, who was a top education official in the Clinton administration. The foundation has been pushing to make research data and other information resources from higher education more widely available.
鈥淚t introduces transparency, which creates a level of accountability which hasn鈥檛 existed before,鈥� Mr. Smith said.
When someone is looking over their shoulders, he added, 鈥測ou make people much more rigorous about carrying out their analyses.鈥�
Coverage of research is underwritten in part by a grant from the Spencer Foundation.
