Educational technology companies trying to do business with schools often have only a vague sense of district officials’ specific buying needs, according to a new study that delves into vendors’ and K-12 leaders’ frustrations with the procurement process.
The study, based on surveys and interviews of district leaders and ed-tech company officials, finds that many business officials regard the district buying process as cumbersome and confusing.
Districts administrators, for their part, say they’re often overwhelmed by the vast number of ed-tech products peddled to them, which they lack the time and resources to evaluate thoroughly, the study says.
The overall findings are likely to resonate for many executives in the K-12 industry who have long regarded the K-12 procurement as bureaucratic and particularly unwelcoming to startup companies with potentially cutting-edge products and ideas.
But the report also provides details on which aspects of the buying process cause the most difficulty for vendors and K-12 administrators—and it offers suggestions for how the process could be improved to help buyers and sellers, and, presumably, students and schools.
The study found that ed-tech vendors, on the whole, are much more frustrated by the procurement process than are K-12 officials such as superintendents, curriculum directors, chief technology officers, and school business officials.
There’s a price to be paid when companies lose faith in districts’ procurement processes, said Karen Cator, the president and CEO of Digital Promise, a Washington- and California-based nonprofit that was one of the organizations to sponsor the procurement study.
She said entrepreneurs and others offering digital products and tools that could help schools may end up avoiding K-12 systems, and instead look to markets they regard as easier to work in and more lucrative.
“Education is not necessarily the market they’re going to jump into,” said Ms. Cator, a former director of the U.S. Department of Education’s office of educational technology. “The consequence is a lessening of innovation and new ideas.”
Vague Priorities
The study was conducted by researchers at Johns Hopkins University, and was commissioned by Digital Promise, a congressionally authorized organization that focuses on improving education through technology and research, and the Education Industry Association, a Vienna, Va.-based group with more than 200 members, including small and large technology companies. Both Digital Promise and the EIA helped design the study.
The university researchers used a mixed-methods approach, based on surveys of nearly 300 district officials who have some connection to procurement, including superintendents, curriculum directors, business officials, technology directors, and principals. The researchers supplemented that data with interviews with survey participants.
The survey offers myriad examples of shortcomings that vendors and school officials see in the procurement process. Among the findings:
• A majority of ed-tech vendors, 66 percent, were dissatisfied or very dissatisfied with district procurement procedures. By contrast, most district officials—between 60 and nearly 80 percent, depending on their job titles—said they were satisfied or very satisfied with the buying process, the survey found. (Superintendents were more likely to be content with the procurement process than other K-12 administrators, such as school business officers and principals.)
• Nearly all districts acquire ed-tech products based on some kind of “needs assessment.” But those assessments are often informal and don’t provide vendors with information on the specific educational challenges that districts hope to conquer.
• Districts struggle with “discovery,” or trying to find the best ed-tech products to meet their needs within the crowded and competitive landscape of companies touting their goods. As a result, school officials tend to regard the buying of tools and systems as a “hit-or-miss” enterprise.
• Similarly, many companies, especially less-proven ones, report that they don’t have the capacity to market their products to large numbers of districts, and so their ed-tech goods remain unknown to many potential K-12 buyers.
• District officials see “cooperative purchasing” with other K-12 systems as an appealing but often unrealistic strategy, because they regard districts as having specialized needs.
• Superintendents and other central-office administrators see decentralized, or school-based purchasing, as undesirable, because it leads to a loss of quality control and fragmented instructional practices. School principals, as one might expect, are much warmer to the concept.
• With limited, reliable information on what to buy, district officials tend to rely on recommendations from peers in other districts. They also rely on pilot studies—though such pilot studies tend to be conducted using very different standards, which may not provide useful information about the strengths and shortcomings of products, the study found.
• Principals and teachers are typically only moderately involved in district needs assessments in determining which educational technologies to buy, despite those employees’ key roles in making digital approaches work in classrooms.
The study was funded by the Bill and Melinda Gates Foundation, which also provides support for Education Week‘s coverage college- and career-ready standards. Education Week retains sole editorial control over the content of this coverage.
As it now stands, many district needs assessments are often so vague that developers are often unclear if their ed-tech product are what those districts covet, said Steven M. Ross, a professor of educational psychology and a senior research scientist at Johns Hopkins, who is a co-author of the study.
For instance, in requests for proposals and other public statements districts will lay out their broad goals for increasing student achievement in math or language arts without laying out what types of ed-tech interventions or instructional strategies most appeal to them, Mr. Ross said.
Simply saying “‘we need a math program for high school,’” without providing other relevant details, he observed, is “sort of like saying, ‘We need a car with four doors.’”
Teachers’ lack of involvement in determining districts’ ed-tech needs is especially troubling, Mr. Ross said. The complex nature of many ed-tech products, and the uncertainty about how students will respond to them, means they need to be vetted with educators in ways that textbooks and other printed materials do not, he said.
Mr. Ross was not surprised by districts’ reliance on pilot tests of ed-tech products. But he pointed out that pilot research often means different things to different districts.
In some districts, a pilot is treated as more of an informal “demo” to get teachers and students exposed to the tech tool, he noted. A better approach is to have groups of teachers testing out ed-tech tools under “real conditions” and “actively engaging with products,” Mr. Ross said, yet “very few districts are doing that.”
Marketplace ‘Noise’
Steven Pines, the executive director of the Education Industry Association, said the study results echo the complaints he often hears from vendors, many of whom struggle to make it in the K-12 market and tell him they feel “invisible,” with little hope that districts will ever become aware of their products.
The survey made it clear districts are flummoxed by several parts of the procurement process, too.
Both sides believe “there’s noise in the marketplace, and that they have difficulty navigating through that clutter,” Mr. Pines said.
John Musso, the executive director of the Association of School Business Officials International, in Reston, Va., said the study offered valuable perspective on school and company officials’ views of procurement.
But he was skeptical of the notion that large numbers of promising ed-tech startups are being shut out of the school market. Entrepreneurs are securing a foothold in K-12 systems all the time, he pointed out. The problem is that many companies don’t deliver on marketing promises, which makes K-12 leaders reluctant to test new ones.
“69ý have a really hard time distinguishing the good, the bad, and the ugly,” Mr. Musso said.
The study didn’t account for other factors that can disrupt K-12 procurement of educational technology, he added, such as the turnover in district leadership. “When you get a superintendent forced out because of low test scores, clearly the new person will come in with different [priorities]” in buying, he said.
Mr. Musso agreed with the study’s conclusion that vendors could benefit from getting better information about districts’ specific buying needs.
But he said districts face very different challenges in gauging their needs—and they evaluate their needs in different ways, as a result. It’s typically much easier for smaller districts to get a sense of teachers’ and schools’ ed-tech demands than it is for larger school systems, he said.
“There’s a difference between the vendor’s view of helpful and the district’s view of helpful,” Mr. Musso said.
The study’s authors put forward a number of recommendations for improving the buying process. Some of that advice is meant to increase the flow of information about ed-tech products and districts’ demands.
One recommendation is to build an online “Ed Tech Product Information Exchange” that would list and describe digital tools, report research on their effectiveness and consumer reaction to them, and allow for sharing of that information across K-12 districts. Mr. Musso said he liked that idea—as long as that online resource was feeding K-12 officials independent, reliable information, not merely spin provided by companies.
Another idea is to establish guidelines for conducting assessments of district needs, for staging pilot tests of products that can help better align tests to the real needs of teachers, produce reliable information on school officials’ satisfaction and student outcomes, and allow for the analyzing of data for product evaluation and development.
Pilots should be a tool “for discovery and evaluation,” the authors say. When they are structured and rigorous, pilots “generate evidence [about digital tools] that is not only useful locally, but also to other districts.”
