Urban school systems are large businesses, charged with running a wide range of noninstructional functions that typically don’t garner them much national notice.
But now, thanks to the work of a coalition of big-city districts, their leaders are gathering data on how those operations are run, in the hope of improving their business practices.
The Performance Measurement and Benchmarking Project, led by the Washington-based , aims to help districts create benchmarks for operational performance and learn from other districts that are delivering services in the most efficient and effective way.
Districts’ timeliness in completing work orders varies widely. Those with lower completion times were more likely to have a management system in place with funding to address repairs. They were also more likely to have higher rates of customer satisfaction than those with longer wait times.
SOURCE: Council of the Great City 69ý
“We thought it was important to launch this project so big cities could compare themselves to each other on operations that were critical to their mission, and we’d have some way of building some confidence and improving our overall noninstructional operations,” said Michael D. Casserly, the executive director of the council, which represents 67 urban school districts. “We really are extremely determined to improve academically and operationally, and we need good and comparable data to do both.”
The push for more data reflects a growing trend among district leaders to import methods used in private business, in an effort to show efficiency and progress to a skeptical public accustomed to reading about cases of waste and mismanagement in urban schools.
Beverly L. Hall, the superintendent of the 49,000-student Atlanta public schools, said the project also is a compliment to the council’s ongoing work on measuring and evaluating academic achievement.
“Even before the economic climate we are in, there are always people who are skeptical about whether the urban systems in particular were being efficient and effective,” said Ms. Hall, who chairs one of the council’s task forces that launched the data-gathering effort five years ago. “Now that we can benchmark, I think it will help us to do our own self-checking about our efficiency and effectiveness.”
3,000 Data Points
Through the work of the member districts and the council’s staff, more than 3,000 data points were collected to examine districts’ performance in four areas: business operations, finance, human resources, and information technology. The work is being done on a volunteer basis by school district executives, with no outside funding.
Since the latest round of data, a 264-page report called “,” was presented to members this past fall, district leaders have been using it for comparison purposes.
The project also produced four initial case studies, which looked at a handful of measures in procurement, maintenance operations, financial management, and food services. Known as the “essential few,” the measures were picked from a set of “key performance indicators,” considered important for superintendents and school board members to have for a quick understanding of the operational health of their districts.
“No one district has indicators that are exemplary,” Mr. Casserly said, “but there are some districts that seem to do uniformly better than others. We are doing case studies on their practices and what allows their indicators to be as good as they are.”
The goal, he said, is to transform those performance indicators into a discussion of policy and resources by superintendents and school board members.
One measure, for example, examined the number of teacher vacancies that had been filled by the first day of school. Districts reported between 46 percent and 100 percent of their vacancies filled, with the median at 64 percent.
In examining performance metrics in transportation, the council looked at the weighted average age of school bus fleets. Districts have to consider the capital costs of buying new buses against the maintenance costs of repairs for an older fleet. The report recommends using a “careful life-cycle cost analysis” to balance the two factors.
The data analysis revealed trends that reflect different climates. For example, the three districts with the oldest bus fleets were in balmy Southern California, where buses last longer, while Northeastern school districts were well represented among those with an average fleet age less than the median level of 6.7 years.
Ms. Hall said when her team uses the performance indicators to help set benchmarks for the Atlanta school system, they examine whether policy or local preferences contribute to higher or lower costs.
“When there are variations, we look at these as opportunities to improve our practices,” she said. “And we look at whether it is an intentional variation because of policies, or if there are things we should all be doing differently.”
Close Look
For example, she found that Atlanta spent more money busing students than some other systems did, which she explained as a legacy of an earlier time of busing for desegregation. Parents and communities, she said, now have become accustomed to having buses to transport children.
Atlanta also doesn’t benefit from a wide-ranging transportation system like that in New York City, where Ms. Hall has spent a majority of her career.
As the 408,000-student Chicago school district prepares its budget for the 2010 fiscal year, its operational teams are working on creating performance metrics for each of the district’s “cost centers,” such as transportation and food service. The benchmarking data from the council’s reports make up a key element of the process, providing information for staff members to use as they begin establishing goals, Chicago officials said.
“It’s a good check for us. You start the conversations of ‘Why is our performance at this level?’” Pedro Martinez, the district’s chief operating officer, said. “What that drives is also conversations with other districts to say ‘What are you doing better to reduce your cost?’ Sometimes it can be replicated, but sometimes it can’t.”
Mr. Martinez said the value of the data makes the labor-intensive undertaking worthwhile.
“We are seeing people welcome this. On the instructional side, they have always been under a lot of pressure to measure things,” he said. “Now that we are putting this on the noninstructional side, it changes things. People realize that it’s not just the instructional side that has to be accountable for performance measurement. We are all trying to get better.”
Mr. Martinez, who is the benchmarking project’s leader on financial measures, said finding comparable data on the business operations of school districts was often difficult in the past. Studies have examined government and private industry for benchmarking purposes, he noted, but not large school districts.
“Generally, we have found there is a lot of support from the superintendents and the chief financial officers and the chief operations officers,” Ms. Hall said of the project. “They want to really have these indicators against which to benchmark their work. Gone are the days where you could just say, ‘We believe we are efficient.’ You have to have metrics to measure it against.”
The Council of the Great City 69ý used Six Sigma, a business-management strategy that includes complex statistical calculations, to help verify the data.
As part of the council’s goal of “fostering a safe environment” during the project, the raw data collected are shared only among staff of the council and the technical teams. All public reports of the data identify each district by a number, not by name. To further preserve confidentiality, the number is shared only with the school district for its staff to use in comparisons with other districts.
As the project continues, the council plans to do more case studies to examine other areas of performance, and it may send technical teams to look more deeply at districts that are performing at a higher rate.
Mr. Martinez said that for the next round of financial data, there are plans to break out the performance of districts by enrollment, which will allow for more-direct comparisons between the larger districts, such as Chicago and Los Angeles. One human-resource measurement will look at the rate at which teachers leave the profession within the first five years, for example.
Another next step for the project will be allowing districts to make real-time performance measurements through the council’s Web site. Mr. Casserly said members have tested a prototype, with the hope that a system will be up and running within a few months.
With the recession causing district budgets to take a hit, Ms. Hall said, the need for operational efficiency has become even more of an imperative.
“We have been doing this kind of work on the achievement side,” she said. “Now we have something similar in operations looking at our business practices. There’s nothing like a crisis to really drive it home.”
