69传媒

A new evaluation of the Bill & Melinda Gates Foundation鈥檚 campaign to improve U.S. high schools offers a decidedly mixed picture of the early returns the foundation is getting from the roughly $1 billion it has invested in the initiative so far.

The in-depth study, commissioned by the foundation and scheduled for release this week, identifies several shortcomings in the high schools that have been started or redesigned with the philanthropy鈥檚 support, especially in instruction and student performance in mathematics. It also cites signs of progress in reading and language arts, and concludes that many Gates-funded small schools, particularly those built from scratch, offer a positive learning climate for students.

on the 鈥榮 high school initiatives are posted by the organization.

鈥淚n summary, the first returns are promising for English/language arts but worrisome for mathematics,鈥� the researchers say in the report.

The study鈥檚 authors caution, though, that the achievement data available to them were insufficient to draw definitive conclusions about student performance. They supplemented the data with analysis of teacher instruction and student schoolwork to provide a more meaningful portrait.

Among the most disheartening findings of that analysis鈥攁nd one the researchers said also applied to comparison schools in their study that do not receive Gates support鈥攚as the lack of rigor in teacher assignments and student work, especially in math.

鈥淸W]e concluded that the quality of student work in all of the schools we studied is alarmingly low,鈥� the evaluation says. 鈥淭his is not surprising, however, because students cannot demonstrate high-quality work if they have not been given assignments that require 鈥� deep understanding鈥� and higher-order thinking skills.

The study, the third in a series financed by the foundation, was conducted by researchers at the Washington-based American Institutes for Research, or AIR, and the Menlo Park, Calif.-based SRI International. It comes as the foundation is awarding new grants this fall aimed at helping districts with high school improvements, including the school systems in Portland, Ore., Los Angeles, and Atlanta, with more expected in the coming weeks.

Tom Vander Ark, the executive director of education initiatives for the Gates Foundation, said the evaluation鈥檚 findings were consistent with information he鈥檚 gleaned over the past couple of years, and he said the Seattle-based philanthropy had taken steps to adjust accordingly.

He cited, for instance, a greater emphasis on districtwide measures intended to improve the quality of curriculum and instruction, as well as an emphasis on using proven school models.

鈥淚 didn鈥檛 find anything surprising in the report,鈥� he said in an interview last week. 鈥淚 think it鈥檚 fair to say we鈥檝e learned a number of important lessons, and it鈥檚 quite clear that over the last six years I created a number of grant programs that were well-intentioned but had some weak assumptions.鈥�

Math Found Lagging

The Gates Foundation has committed about $1 billion since 2000 to support the start-up of small high schools or the restructuring of large existing schools into smaller units.

Gates officials often emphasize that school size is not an end in itself, but a tool to build strong relationships between teachers, students, and families, and to deliver a rigorous curriculum that is relevant to students鈥� lives.

Foundation officials say that, in all, the philanthropy has supported more than 850 new schools and nearly 700 existing high schools. Funding has gone to schools in nearly 275 districts. (Education Week also receives funding from the foundation.)

In general, the nearly 300-page study found more positive outcomes, from achievement and student work to academic climate, in schools that were newly created than in those that had undergone redesign.

鈥淏ased on the data currently available, it appears that new schools have been successful with respect to attendance, test scores, and the quality of students鈥� work in English/language arts,鈥� the researchers write.

Math appears to present the biggest problem, in student test scores as well as instruction and student schoolwork.

鈥淚n general, the math achievement level of students attending new schools is on par with or lagging behind other schools in the same district,鈥� the report says.

The report provides standardized-test data from one or two school years from four unidentified urban school systems. The researchers also examined results from four other districts for purposes of analysis.

Attendance rates at new schools were generally higher than at other schools in the district. Average reading scores were lower than the district average in some systems, but that finding was reversed, the study found, after statistically controlling for the level of students鈥� prior achievement.

In two of the three districts where trends on state test data could be examined, the study found larger improvements in reading and English language arts achievement over time in foundation-backed schools than elsewhere in the district. The third district saw gains in that area 鈥渙n a par with the rest of the district.鈥� The reading gains were larger in the system with new schools than in the district with redesigned schools.

And the researchers stress that the Gates-funded schools tend to serve largely disadvantaged students who enter high school well behind grade level.

鈥淲hen we look at 8th grade achievement levels, we find almost universally 鈥� students start substantially behind other students going elsewhere in the district,鈥� said David A. Rhodes, an AIR researcher involved in the study.

鈥楢 Huge Challenge鈥�

For redesigned schools, the researchers were still more cautious about drawing conclusions on test-score data.

They found slightly lower-quality student work than in the Gates-funded start-up schools. But in examining the rigor of teacher assignments in math, the report says the results were 鈥渟imilarly poor鈥� across the two school types.

鈥淗alf of the assignments at both types of [Gates-funded] schools exhibited little or no rigor,鈥� the report says.

Mr. Vander Ark cited several factors that he believed contributed to the problems in math.

鈥淔ive years ago, the nation鈥檚 attention was 鈥� more on literacy than on mathematics,鈥� he said. He also noted that in small schools, there may well be extra faculty support for addressing students鈥� language arts difficulties, since those areas overlap with several disciplines.

鈥淭he same is less true for mathematics,鈥� he said. 鈥淚n a small school, you may have two math teachers working on their own with students that are a number of years below grade level, and they face a huge challenge with limited support.鈥�

As for student attendance, it was relatively strong at new schools but fell short at the redesigned campuses.

The researchers cautioned that the redesigned schools have a history of underperformance that must be changed.

鈥淵ou really need to think of the redesign reform as on a longer time frame,鈥� Mr. Rhodes of AIR said. 鈥淎nd that鈥檚 going to emerge out of an existing stream of data that, to be blunt, is going to be dismal to start with, because a school isn鈥檛 going to reinvent itself if it鈥檚 already doing a good job.鈥�

Some of the most promising findings had to do with what the researchers termed 鈥渃reating a culture for learning.鈥�

Still, the redesigned schools were seeing what the researchers described as 鈥渟lower progress as they work to change existing structures, cultures, and beliefs.鈥� Those schools, too, showed signs of improvement over time, most notably in fostering the creation of a school culture in which students feel known by their teachers and supported by them both in academics and personally.

Strategic Shifts

Mr. Vander Ark of the Gates Foundation said the philanthropy has been shifting its grantmaking approach, particularly in the past past two years, to help address many of the problems the research spells out.

First, he said, 鈥渨e try to approach school improvement in the context of a system, a school district. ... One way to think about it is, we took school-as-the-unit-of-change too far.鈥�

Second, the foundation is focusing its school efforts on 鈥渨ell-specified school models that provide really strong support,鈥� including a well-developed curriculum and instructional approach, Mr. Vander Ark said. Models he cited included the networks of schools affiliated with the Providence, R.I.-based Big Picture Company and the San Francisco-based Knowledge Is Power Program, or KIPP.

Mr. Vander Ark said the foundation has much more explicit expectations for school-level change than it initially did.

鈥淎nother thing I got wrong at the beginning was autonomy,鈥� he said. 鈥淚 visited 100 great schools and made the observation that they were all small, autonomous, and assumed that was a path to school improvement. It turns out giving a failing school autonomy is a bad idea.鈥�

On the subject of change throughout whole districts, he cited the foundation鈥檚 $2.3 million grant last spring for a planning effort to improve Chicago鈥檚 districtwide approach to curriculum and instruction.

Another grant just issued, $1.4 million to the Atlanta public schools, will pay for district plans to redesign high schools鈥� curriculum and instruction, and will consider the role of new schools in that effort.

鈥淲ith an investment of $2 million in a plan, you can help a city make better use of a billion-dollar budget,鈥� he said. 鈥淭hose plans typically incorporate small schools and small learning communities, but they pay more attention to curriculum and instruction.鈥�

The AIR-SRI evaluation offers a series of recommendations, including a call for the foundation to support professional development, technical assistance, coaching around math content and instruction, and efforts to provide curricular materials.

It also focuses on the need for sustainability, noting that new and redesigned schools are 鈥渧ulnerable organizations, with limited internal capacity and numerous external challenges.鈥�

鈥淭he foundation and its grantees may want to focus more of their energy and resources on protecting the schools that have already been started,鈥� the report says, 鈥渆ven if means starting fewer new schools.鈥�