69传媒

Emerging research shows the science school-age children learn in informal settings鈥攆rom museums and clubs to online communities and television shows鈥攃an have a big impact on their lives. Yet the open format and distinct structures of informal science make it next to impossible for researchers to evaluate the quality of those experiences in the same way they can gauge formal schooling.

School assessments generally focus on cognitive measures, such as what a student knows and can demonstrate about particular content. In contrast, informal learning is dominated by noncognitive measures such as motivation, interest, and identity, according to Larry E. Suter, the National Science Foundation鈥檚 program director for informal science education. Moreover, traditional 鈥済old standard鈥� research methods such as randomized controlled trials can be detrimental to activities that base their strength on people鈥檚 choosing to participate, rather than being assigned.

Such research raises the risk, Mr. Suter said, that 鈥渋f you touch it, you鈥檙e going to kill that thing you鈥檙e trying to study.鈥�

That has led to some creative research alternatives. Alan J. Friedman, a former director and chief executive officer of the New York Hall of Science and the editor of the NSF鈥檚 2008 framework for evaluating informal science education, recalls judging the effectiveness of an astronomy exhibit by the number of visitors who chose an astronomy poster over a different prize. Barbara N. Flagg, the director of the Multimedia Research consultant group in Bellport, N.Y., said she has used smudged museum walls indicating where visitors have touched exhibits, changes in Google and Amazon search terms over time, and phone interviews with parents and children.

鈥淚f you鈥檙e comparing this to other education research, you鈥檝e got to turn back the clock 30 or 40 years,鈥� said Kevin J. Crowley, the director of the University of Pittsburgh鈥檚 Center for Learning in Out-of-School Environments and an associate professor of education and psychology. 鈥淲e are just now in the Wild West frontier, and people are just starting to gear up the longitudinal studies on how this will coalesce into a coherent narrative of how people learn science.鈥�

Now, the tools being born of that creativity show the potential to link children鈥檚 education across a lifetime of different experiences, and in the process uncover more of how and what children learn than has ever been measured in a school test alone. Children, after all, spend more than 80 percent of their waking hours outside the classroom.

鈥淭he research has evolved,鈥� Mr. Crowley said. 鈥淚n the past, the great flaw of the informal-learning science was we looked at it in situations; we weren鈥檛 really looking at a lifelong trajectory in science learning. For the first time, we鈥檙e asking questions about how learning and participation are moving across place and across time.鈥�

鈥淏renda鈥檚鈥� school performance, for instance, didn鈥檛 really show her scientific achievements or level of interest in the subject. Teachers reported the first-generation Haitian daughter of a single mother in Seattle had little interest in science and consistently 鈥渇ailed to engage鈥� with the chemical-mixing tasks in her school lab.

Yet because researchers at the University of Washington, in Seattle, had been observing the 4th grader across formal and informal settings for more than 2,000 hours, they knew that school didn鈥檛 tell Brenda鈥檚 whole story. Not only did she regularly measure and mix chemicals and record the results for her perfume-making hobby, but she also had told the researchers she was considering becoming a chemist when she grew up.

鈥淪chool science underrepresents her developing expertise,鈥� Philip Bell, an associate professor of learning sciences at the university and the director of ethnographic and design-based research at the Everyday Science and Technology Group there, said during a recent lecture. 鈥淛ust in terms of how people learn, our literatures don鈥檛 do justice to the varied pathways that people take through their experiences to make progress on things they care about.鈥�

Interest vs. Grades

That鈥檚 a dangerous disconnect, experts say, because mounting evidence shows that early engagement, even through informal pathways, eventually can lead to careers in the STEM fields of science, technology, engineering, and mathematics more surely than top grades in school.

In a published in the journal Science, Robert H. Tai, an associate professor at the University of Virginia鈥檚 Curry School of Education, in Charlottesville, tracked thousands of students via the National Educational Longitudinal Study. He found that students who had only average grades in middle school but expressed interest in science were two to three times more likely to earn bachelor鈥檚 degrees in a science or engineering field 12 years later than high-achieving students who did not voice interest.

The landmark 2009 study that Mr. Bell co-wrote, for the National Research Council, synthesized a vast array of research, including Mr. Tai鈥檚, to conclude that 鈥渁bundant evidence鈥� exists that informal experiences can effectively support learning. What鈥檚 more, it found that those free-choice, low-stakes pathways can engage children from groups underrepresented in science fields: students in poverty, those from ethnic and language minorities, and women.

Still, while the number of research proposals submitted to the NSF about informal education has doubled in the past five years, from 300 to 600 annually, Mr. Suter said, the vast majority have been and continue to be from practitioners evaluating their individual programs, clubs, and exhibits, often using self-made assessments.

Better Alignment

In 2008, the Program in Education, Afterschool, and Resiliency at Harvard University and McLean Hospital in Boston found that of 64 widely used informal science evaluation tools, none met all five of the NSF鈥檚 five domains of informal learning: engagement and interest, attitude toward science and behavior, content knowledge, competence and reasoning, and career knowledge and acquisition.

Program founder Gil G. Noam and Rick Bonney, the director of program development and evaluation at the Cornell University Ornithology Lab, in Ithaca, N.Y., and the creator of several nationwide citizen-science projects, are working to change that. Mr. Noam has established the Assessment Tools in Informal Science database of existing tests. He said that he and Mr. Bonney are about a year away from using the tests to develop a single, unified assessment of informal science learning.

鈥淥ur big goal in all this is to bring into this field some proof that the informal science work makes a difference,鈥� Mr. Noam said.

Likewise, Mr. Crowley鈥檚 organization, CAISE, has gathered five years鈥� worth of evaluation reports from NSF informal science projects鈥攎any never published鈥攁nd has started to cull best practices and other lessons learned from across the many types of informal learning.

鈥淥nly recently has the learning theory caught up with those hunches, and now we鈥檙e at a place where we really can test design hypotheses based on what we know about the science of learning,鈥� he said.

Ultimately, John H. Falk and Lynn D. Dierking, both professors in free-choice learning at Oregon State University, in Corvallis, hope the field will move toward more 鈥渟ynergy鈥� studies, like Mr. Bell鈥檚 longitudinal work, that can explore how people learn science across the formal and informal experiences of their lives.

The Oregon State researchers now are monitoring how children in Portland, Ore., come to learn scientific concepts across the city鈥檚 many formal and informal science offerings during a four-year period. Portland provides a perfect microcosm for science learning, Mr. Falk said, because it has one science center, zoo, and public-broadcasting station and a few school districts that will allow the researchers to track individual students over time.

鈥淚n order to see real systemic improvement, you have to have a whole greater than the sum of its parts, and now the whole is less than the sum of its parts because there鈥檚 no alignment among these pieces,鈥� Mr. Falk said. 鈥淲e need to develop ways to better integrate practitioners across formal and informal education. We don鈥檛 understand how people navigate, make sense of, and make use of the resources in their community.鈥�