It took less than a year for leaders in the Utica City School District in New York to regret spending $3.7 million on artificial intelligence systems designed to keep weapons out of schools. They quickly reversed course when the technology failed at the one thing they expected it to do.
Last October, a high school student stabbed and injured his classmate, using a knife that the scanners apparently failed to detect.
What followed was a debate about the machines鈥� effectiveness, complaints about the limits of the technology, and a decision to phase them out.
The AI systems, made by a company called Evolv Technology, remain in the district鈥檚 elementary and middle schools. But officials have replaced them with 鈥渁irport style鈥� metal detectors and bag scanners at the high school. It takes students longer to pass through, but administrators also have more confidence they will detect weapons like knives and cans of pepper spray, acting Superintendent Brian Nolan said.
鈥淚t may stop someone with a bomb or a rifle, anything like that,鈥� Nolan said of the Evolv system. 鈥淏ut the practical application for a high school, the primary weapon of choice for a high school student is a knife. They didn鈥檛 catch that.鈥�
The Utica machines also missed a gun, Nolan said. A staff member at another school told him that a police officer, who was also a parent, passed through one of the scanners with his service weapon, which went undetected.
As districts around the country respond to concerns about community violence, student behavior, and school shootings, a growing number have to keep weapons out of their buildings.
Makers of the systems, which use a combination of AI software, cameras, and electromagnetic sensors to detect possible weapons as students walk past, sell their products as a highly effective alternative to the traditional metal detector that allows users to avoid long lines. The systems are also used by major sporting venues to screen fans attending professional sporting events.
But their adoption has not been without frustration. Some districts, like the Urbana, Ill., school system, have reported consistent with common items like water bottles and Chromebooks, creating new challenges at building entrances. Officials in the Charlotte-Mecklenberg, N.C., schools recorded videos showing students how to hold frequently misidentified objects in their outstretched arms so staff monitoring the scanners can easily see what triggered the sensor.
Reducing the machine鈥檚 sensitivity will reduce false alarms, but it may also reduce the technology鈥檚 effectiveness at detecting legitimate weapons, like guns, administrators told Education Week.
Administrators who support the use of AI scanners say that, while they may not be a definitive solution to concerns about school violence, they have helped reduce weapons reports and served as a deterrent. Part of the job, they say, is adapting to the quirks of the technology so that it can be more effective.
鈥淭echnology has limitations, and we have to understand that and work around those limitations,鈥� said Ronald Applin, director of safety and security for Atlanta Public 69传媒, which installed Evolv detectors this year.
Growing use of AI weapons detection in schools
A growing number of school districts, including eight of the 100 largest in the country, have installed AI weapons detection systems in the past two years. Local news reports suggest some school systems have used federal COVID-19 relief funding to cover the expense. The machines rely on ongoing software fees to operate, which means the districts will shoulder continuing operations costs when that time-limited federal aid expires.
Evolv, one of the largest providers of AI security technology for schools, said recently that the K-12 sector represented half of its business in the fourth quarter of 2022鈥攁longside clients like Atlanta鈥檚 Mercedes-Benz Stadium, where the city鈥檚 NFL team, the Falcons, play.
K-12 鈥渃ontinues to be our No. 1 end market,鈥� Evolv CEO Peter Gustav George said in . 鈥淲e partner with our customers to create safer zones in over 400 school buildings, up from 200 at the end of the third quarter.鈥�
Technology has limitations, and we have to understand that and work around those limitations.
School security vendors often see a surge of interest after school shootings, like the May 2022 attack at Robb Elementary School in Uvalde, Texas. Conversations about 鈥渉ardening schools鈥� with physical security measures, including AI and other weapons detections systems, started anew after six people died in a March 27 shooting at a private elementary school in Nashville, Tenn.
About 2.7 percent of all public schools and 7.7 percent of high schools reported in the 2019-20 school year, according to the most recent federal data. Six percent of all schools and 14.8 percent of high schools reported random sweeps with metal detectors.
While metal detectors and AI scanners may help keep some weapons out of schools, they are less likely to prevent a mass shooting, Ohio-based school safety consultant Kenneth Trump said. Attackers intent on committing mass violence are known to shoot through secure doors, as the Nashville shooter did, or openly carry weapons into buildings. (In Denver, a high student shot and wounded two staff members on March 22 as they conducted a routine weapons search.)
An increase in school weapons reports
While mass shootings drive school safety conversations, some school districts that use systems like Evolv鈥檚 point to day-to-day concerns about weapons in their buildings as a motivation for installing them.
Atlanta schools installed Evolv scanners this year in some building entrances, particularly those where large numbers of bus riders pass through at once, to supplement the use of traditional metal detectors at other entry points, Applin said. By this time last school year, the district had confiscated 30 guns from students, he said. This year, they鈥檝e confiscated seven so far.
It鈥檚 not a foolproof system. Some students managed to bypass scanners by passing the guns through windows or side doors their peers opened for them, Applin said.
Charlotte-Mecklenburg schools installed Evolv scanners at its high schools after finding a record 31 firearms on students during the 2021-22 school year, Chief Operations Officer Brian Schultz said.
鈥淐oming out of COVID, we saw increased physical aggression, increased firearms,鈥� he said. 鈥淭his is a problem around the country.鈥�
The district鈥檚 use of Evolv scanners was the subject of , which relied on 2,000 internal emails to reveal problems with its rollout. Officials at another school district had warned Charlotte-Mecklenburg administrators that it had been able to pass a Glock handgun through the scanners without detection, Vice reported.
Administrators also complained of long lines at the scanners, error messages, and false alarms that detected 鈥渨eapons of mass instruction,鈥� like Chromebooks.
The process has improved in the time since, Schultz told Education Week. Administrators made videos to show students how to pass through the machines and to hand off frequently flagged items like umbrellas to supervising staff beforehand.
The district has found four firearms on students this school year, Schultz said, although some were located by word-of-mouth after students brought them into school through another entry point.
鈥淥ur students have become a lot more vigilant about reporting those kinds of things,鈥� Schultz said. 鈥淩elationships trump any system you have.鈥�
鈥楴o perfect solution鈥�
Independent school safety consultants warn about the effectiveness of the scanners and have called for more data about their effectiveness.
In November 2022, that a private evaluation of Evolv machines showed that, in 24 trial walkthroughs, they failed to detect knives 42 percent of the time. That detail was among those omitted from a shorter version of the evaluation that Evolv released publicly, leading some attorneys to claim the company had participated in deceptive marketing tactics.
At EdWeek鈥檚 request, Evolv initially scheduled an interview with one of its executives to respond to questions about the technology. But it later cancelled the call, citing a busy travel schedule, and a spokesperson responded to questions by email instead.
鈥淭here is no perfect solution that will create a completely sterile environment and catch all weapons for schools or any venue,鈥� Anil Chitkara, Evolv鈥檚 co-founder and chief growth officer said in an emailed statement. 鈥淭his is why a layered approach of people, process, and technology is used in security planning and execution鈥攁nd schools are no different.鈥�
Evolv acknowledged that its machines flag some everyday items that have the 鈥渃omposition and profile of component parts鈥� of weapons. But, as the AI technology continues to 鈥渓earn鈥� to better distinguish everyday objects from weapons, the company offers software updates to its customers, Chitkara said. In the fourth quarter of 2022, Evolv machines around the country 鈥渢agged nearly 70,000 weapons鈥攊ncluding 36,000 guns and 33,000 knives,鈥� he wrote in the email.
The company also denied misleading marketing. Evolv doesn鈥檛 want to provide a 鈥渂lueprint鈥� of vulnerabilities to bad actors by revealing too much, Chitkara said in the email response.
While 鈥減ublic-facing marketing materials are intentionally not specific,鈥� he said the company communicates all 鈥渓imitations and鈥痗apabilities鈥� of the system with potential customers.
Most district administrators who spoke with EdWeek would not discuss the settings they used on their Evolv systems or possible error rates, citing security concerns.
In Atlanta, Applin acknowledged that no machine is perfect. The district, like Charlotte-Mecklenburg, also relies on student tiplines, security cameras, and student awareness campaigns to prevent violence and determine if there are weapons on campus.
鈥淚f someone wants to get something into the building, they are going to find a way to do it,鈥� Applin said.
鈥淲e sometimes have the impression that this is going to solve all of our problems, but it doesn鈥檛 work that way,鈥� he continued. 鈥淏ut when I look at the results from last year to this year 鈥� we know that the numbers [of gun reports] have been reduced significantly.鈥�
After the stabbing incident in Utica, administrators determined a previous superintendent had erred when he told the school board the Evolv system would detect all weapons, including knives, said Nolan, the acting superintendent.
The machines still in place in the district鈥檚 elementary and middle schools are now turned up to the highest sensitivity level in hopes they will be more effective at finding harmful objects. But that means they also alarm on some benign items, too, Nolan said. This year, a mother called him to say her young daughter was in tears after her metal lunch box triggered an alarm and a subsequent search.
AI systems may be a good solution for schools in the future, after years of development and refinement, Nolan said. But for now, he鈥檚 determined they are not the right solution for Utica.
鈥淚t鈥檚 a tough situation when you think about the money you are spending, that it doesn鈥檛 give you the assurance of safety,鈥� Nolan said. 鈥淧eople who are considering it should understand what it will and won鈥檛 do for you.鈥�
