A report from the state’s Office of the Inspector General for Education determined that an artificial intelligence weapons detection system did not show any bias when it mistakenly flagged that a Kenwood High School student may have been carrying a gun, but called for more training with the technology.

The report released Tuesday comes after police drew guns last month on the student who was only holding a bag of chips. The school district’s Omnilert, an AI-powered weapon detection system, flagged that he could have been carrying a gun.

The IG’s report found that it wasn’t the bag of chips that was the perceived threat, but rather the student’s hand movement.

The incident was upsetting to the student, Taki Allen, and his family. Allen’s grandfather, Lamont Davis, has questioned whether racial bias played a part in Omnilert flagging his grandson, who is Black.

He said on Wednesday that he still belives racial bias played a part.

The incident also led County Councilman Julian Jones to call for Baltimore County Public Schools to review its procedures for use of the technology.

School officials said at the time that Omnilert was working as intended. About a week after the incident, the district announced it would conduct Omnilert retraining.

Maryland’s Office of the Inspector General for Education conducted an investigation of the incident and submitted its findings to the county’s superintendent on Tuesday. The school district has to submit a written response by Jan. 9, the report states.

“The OIGE report affirms the information we provided to Team BCPS following an internal review of the incident at Kenwood High School,” the school district wrote in a statement. “We believe our multi-tiered approach for managing school safety helps ensure that we protect and preserve safe learning environments and foster positive student growth.”

The district said they will adopt the report’s recommendations and noted that all administrators and Baltimore County Police Department partners were retrained in November, that reminders about the Omnilert process were provided during an annual safety training and that school resource officers were reminded of after-school procedures.

The investigation was triggered after the IG office received multiple complaints alleging the school system is “operating an ineffective and inefficient artificial intelligence.”

“Additionally, OIGE received claims that the software violated students’ civil rights because its detection algorithm targets disadvantaged students and students of color more than other students,” the report stated.

Baltimore County Public Schools has a three-year, $2.6 million contract with Omnilert that ends in 2027. The technology searches image frames of all the school cameras — 9,500 of them, according to the IG report — for people and the objects surrounding them. If it detects a potential gun, it alerts principals and safety assistants inside the building, as well as security staff in the school system’s central office.

Thousands of alerts have been sent to staff since the district bought the technology in 2023. Fewer than 1% of the alerts required alerting law enforcement, the report stated.

Fifteen people, five from the school system and 10 from the Baltimore County Police Department, received the Kenwood Omnilert notification, the IG reported.

A company spokesperson said the lighting on Allen’s Doritos bag and other issues with the image made the bag look like a gun. Allen disagreed when police showed him the image.

Here’s a timeline of the Kenwood High incident on Oct. 20, according to school officials:

• 7:04 p.m. — Omnilert sent an alert for review
• 7:05-7:06 p.m. — School system safety personnel and the Maryland Safe Schools facilitator reviewed the alert and canceled it.
• 7:17 p.m. — Kenwood’s principal reported the alert to a school resource officer. “Because this alert was received after school hours, the SRO called the local [police] precinct,” school officials stated.
• 7:23 p.m. — Police arrived at Kenwood after receiving a report of a suspicious person with a weapon.

School officials said the principal notified the school resource officer without knowing the security office had cleared the incident. Administrators are expected to work with central office security staff directly when an incident occurs.

Trained school system safety professionals review the images and either validate or cancel alerts based on what they see. Canceling means that no action is taken. Validating means that safety personnel notify police.

According to the inspector general’s office, Omnilert “did not show any bias” when flagging a potential threat. The software doesn’t consider a person’s gender or race but solely focuses on a moving object.

Davis said he’s skeptical of the technology’s use on Black people and he questioned whether the principal would’ve alerted authorities if Allen and the teammates he was sitting with were white.

The incident still impacts his family. His wife, whose first child died from a shotgun wound, can’t talk about it without crying. Allen is now in therapy and steers clear of the benches he was sitting on when apprehended by police last month, said Davis.

In-person Omnilert training was given to administrators, principals and assistant principals before Omnilert was activated, investigators were told. However, the IG office couldn’t verify that claim, since no attendance logs were provided. Only a few meetings were held on the alert system, the report stated, and no individual training was provided.

Staff did have a safety update meeting on Nov. 5, according to the report, and an internal Omnilert training is available online for staff to access.

“The Omnilert system did not detect a single blue potato chip bag as a threat,” the report stated. “The perceived threat was based on the student’s hand movement, including its position and angle.”

Investigators reported that Allen was holding an “unidentified item that extended slightly beyond the fingers and was held at a slight angle near the hip or pocket level.”

Looking at the image on a phone screen and zooming in on the student’s hand “could cause the user to overlook the cancellation text message” that was sent after staff determined it was a non-issue.

The Kenwood principal viewed the alert in a text, clicked the link that led to the video of the event and paused to enlarge the image.

“The principal said that it wasn’t until after they watched the video, tried to contact safety staff, and spoke with BCoPD [Baltimore County Police Department] personnel — who informed them that the police had cleared the school grounds — that they noticed a cancellation had been issued,” the report stated.

The inspector general’s office is recommending that the school district’s safety office review its protocols when it comes to how many people receive alerts.

Staff should use the Omnilert app instead of relying on Omnilert text message notifications so everyone can view the alert in the same format, investigators recommended. And the school system should have biannual training sessions for executive staff and the Baltimore County Police Department on the school system’s protocols for Omnilert alerts.

About the Education Hub

This reporting is part of The Banner’s Education Hub, community-funded journalism that provides parents with resources they need to make decisions about how their children learn. Read more.