School shooting survivor sues AI gun detection firm after system failed to spot weapon

How accurate does an AI system need to be?


See full article...

 

[KitsuneHime]

The criminal must've scuffed the doritos logo off the gun so the system wouldn't detect it.

 

[socialsorcery]

I didn’t need to read the article to know which country this took place in - it’s insane that this is considered necessary in American schools

 

[CatNamedHugs]

due to where the shooter was in relation to the cameras, the imagery “wasn’t close enough to get an accurate read and to activate that alarm.”

Jesus fucking Christ if the software company is able to use that excuse and win the lawsuit, they might as well just sell literally nothing and rake in the millions. What arbitrary distance is good enough to detect the gun, then? And what if someone, I don't know, tapes a shark fin to the top? Is that gonna fuck it up, too? Even from like 2 feet away?

 

[WereCatf]

I do wonder if this lawsuit has any chance of going anywhere. Not that I don't want it to, but the first argument the company is likely to pull is "no reasonable person would expect the system to be 100% infallible" and then provide a bunch of example of AI-systems fucking up, including Teslas -- unless the company promised 100% accuracy, I don't see how that argument can really be gotten around.

 

[PadreDelAcantilado]

I do wonder if this lawsuit has any chance of going anywhere. Not that I don't want it to, but the first argument the company is likely to pull is "no reasonable person would expect the system to be 100% infallible" and then provide a bunch of example of AI-systems fucking up, including Teslas -- unless the company promised 100% accuracy, I don't see how that argument can really be gotten around.

I’m no lawyer, but I would suspect that the discovery phase leads to some damning emails about system limitations that leadership ignored; followed by a settlement just to keep that information private.

 

[QuattroV]

I do wonder if this lawsuit has any chance of going anywhere. Not that I don't want it to, but the first argument the company is likely to pull is "no reasonable person would expect the system to be 100% infallible" and then provide a bunch of example of AI-systems fucking up, including Teslas -- unless the company promised 100% accuracy, I don't see how that argument can really be gotten around.

I think the angle might be, get deep enough procedurally for discovery, get the contract/SLA, and hope there's a breach big enough to prove negligence. If they oversold hard, there might be room there. If there are internal stuff about the product being really useless, even better, wouldn't put it past an AI firm to internally mock their customers.

Fuck useless profiteering on our dysfunction over guns, at any rate.

 

[lwdj905]

Kid in Baltimore was "detained" because of a Dorito bag he threw into garbage can.
Same company, Omnilert...

*They blamed it on the human who was supposed to backstop the software.
Student handcuffed after Doritos bag mistaken for a gun

 

[bone_collector]

“I just thought that it was kind of bullshit. I have a Tesla, and I think Tesla’s self-driving is bullshit,” he said. “It’s not ready for prime time! How could you possibly be entrusting of that? That’s your plan to protect kids from school shootings? Why is this any better than a metal detector?”

This attorney makes some great points. I’d like to read more of his writing!

 

[bone_collector]

At this point Omnilert should just rebrand this system and start marketing it to weight-loss camps instead of schools, since it’s so much more effective at detecting junk food than weapons.

 

[leonremi]

Or America could just get rid of their stupid Second Amendment.
Except for hunting, you don't actually need a gun in your life.

 

[tterminus]

Hey America, I've got some Gun Detectors to sell you!

 

[Mechjaz]

I didn’t need to read the article to know which country this took place in - it’s insane that this is considered necessary in American schools

The only thing we like more than racism and a general crab-bucket approach to society (with a refreshing twist of unironically glorifying the crabs that climb out and put a lid on the bucket afterward) is the ability, no, the right, for kids, teenagers, adults, the disgruntled, the angry, the mentally ill and the criminal - hell, let's just say everybody! - to gun down as many of our fellow citizens as you can before you get taken out yourself.

 

[MilanKraft]

After reading the headline and subhead, and scanning parts of the article, I'm pretty sure the answer is "100% of the time." If you are building a system to detect gun carriers and protect everyone around them from being murdered, the answer is 100%.

And if you can't achieve 100% yet — via using whichever recommended cameras and sensors, and whichever placements at and around the entrances — that's to be expected, but then your product isn't ready for market. That, OR.... the schools and other orgs buying your product need to be told up front the "failure rate is at least 11.2%," so everyone understands what's being paid for is not bullet proof (pun intended).

If that didn't happen, the company should be sued IMO. This isn't effing sunblock we're talking about here where 88% effective is good enough. They're selling a product that ostensibly gives schools or organizations peace of mind that nobody entering their space has a gun, that no one is going to die because of a gun, but it doesn't work well enough yet.

Pretty much that simple.

 

[Hypatia]

Or America could just get rid of their stupid first Amendment.
Except for hunting, you don't actually need a gun in your life.

You might be thinking of the 2nd Amendment. Although the 1st is clearly on the chopping block these days so…carry on?

 

[MrFission]

I made an AI grift detector and it's going bat shit crazy on the school system and the lawyer.

 

[Buffy]

Or America could just get rid of their stupid first Amendment.
Except for hunting, you don't actually need a gun in your life.
Amendment, not first.

Second amendment, not first. We need the first amendment badly.

 

[nzod]

-- never mind --

 

[ZenBeam]

I do wonder if this lawsuit has any chance of going anywhere. Not that I don't want it to, but the first argument the company is likely to pull is "no reasonable person would expect the system to be 100% infallible" and then provide a bunch of example of AI-systems fucking up, including Teslas -- unless the company promised 100% accuracy, I don't see how that argument can really be gotten around.

They address this in the lawsuit:

69) Omnilert’s own post-shooting revisions to its commercial website —adding for the first time disclosures that the system is subject to “false alerts” and that human verification is designed to “reduce false alerts”— confirm that Omnilert possessed knowledge of these limitations at the time of sale and deployment.
70) Omnilert failed to provide adequate warnings or instructions regarding these significant limitations, including but not limited to:
a) critical dependencies on camera placement, positioning, and proximity to the subject holding the weapon, including the fact that the system could fail to detect a firearm if the weapon was not sufficiently close to a camera sensor;
b) the inability to reliably detect weapons held by subjects at certain distances from cameras or in certain orientations;
c) the conditions of lighting, angles, and weapon visibility under which the system would fail to detect firearms;
d) the risk of false positive alerts that would misidentify harmless objects as firearms; and
e) the need for supplemental security measures to compensate for these limitations — warnings that Omnilert elected not to provide while simultaneously representing that the system offered “unparalleled reliability” and detection “before a shot is fired.”
71) The inadequate warnings were a proximate cause of Antonyous Henin’s injuries because proper warnings would have led MNPS and System Integrations to implement different camera configurations,supplemental detection methods, security personnel placement, or emergency response protocols that could have prevented or reduced his injuries.

 

[leonremi]

You might be thinking of the 2nd Amendment. Although the 1st is clearly on the chopping block these days so…carry on?

Lol yes sorry. I changed my comment.

 

[odikweos]

The lawsuits must flow...

 

[PatientZero]

I didn’t need to read the article to know which country this took place in - it’s insane that this is considered necessary in American schools

My friend, just wait until you hear about the "anti-active-shooter drones" that will soon be available in US schools.

https://www.wabe.org/5-georgia-high-schools-to-begin-testing-drones-to-stop-mass-shootings/

 

[Hypatia]

My friend, just wait until you hear about the "anti-active-shooter drones" that will soon be available in US schools.

https://www.wabe.org/5-georgia-high-schools-to-begin-testing-drones-to-stop-mass-shootings/

That must have been a nice check sent to Rep. Matt Dubnik by the “Campus Guardian Angel” outfit.

Looking at the proposals and it specifies “human piloted” but this is the exact sort of system I would expect a company to compile training data on during operations and eventually try to turn over to Ai control.

*edited to add last paragraph

 

[mnkybrs]

My friend, just wait until you hear about the "anti-active-shooter drones" that will soon be available in US schools.

https://www.wabe.org/5-georgia-high-schools-to-begin-testing-drones-to-stop-mass-shootings/

Nothing screams "well-allocated funding in your professional workplace"/"employees who are definitely prepared to be exposed to the violence of a school shooting and intervene effectively" like blinding RGB lighting in your computers.

 

[AxMi-24]

They should be sued into bankruptcy and schools forced to invest money into things that are not window dressing but actually work like counselling for kids in need, or metal detectors and guards if US keeps insisting that it is every mans right to murder his fellows.

 

[mobby_6kl]

Jesus fucking Christ if the software company is able to use that excuse and win the lawsuit, they might as well just sell literally nothing and rake in the millions. What arbitrary distance is good enough to detect the gun, then? And what if someone, I don't know, tapes a shark fin to the top? Is that gonna fuck it up, too? Even from like 2 feet away?

What's wrong with that response? The technology isn't magic. It works within certain parameters, it cen't detect a gun for a single pixel in the camera.

The question really is whether they misrepresented the capabilities, as the lawsuit alleges

According to the lawsuit, which was filed in Davidson County court last month, the security company Omnilert either knew or should have known that there were “significant operational limitations in its gun detection system that could result in detection failures during actual emergencies, including limitations based on camera placement, proximity of the weapon to camera sensors, camera angle, lighting, and weapon visibility.”

 

[WereCatf]

that's the fun thing. it's not. US schools are extremely safe, and school shootings are remarkably rare.

The US is literally at the top of school shootings per country and not even by a small margin: https://worldpopulationreview.com/country-rankings/school-shootings-by-country

 

[DarthSlack]

that's the fun thing. it's not. US schools are extremely safe, and school shootings are remarkably rare.

Fuck this shit. There have been 14 school shootings resulting in injury or death just this year alone. And if you expand the issue to look at all gun-related issues, well, the list gets mighty big. So clearly, you either have a personal definition of "remarkably rare" that's not shared by anyone else, or you've pulled your opinion, fully formed, directly from your ass.

So let me ask you this, how may school shootings do you consider to be acceptable?

but hey, lets spend hundreds of millions of dollars on pointless security systems in order to protect middle class white kids instead of attempting to address the real issue with guns: thousands of inner city black kids gunning each other down yearly. And no, I'm not being racist, this is just the reality.

You know what, if you don't want to have to defend yourself against being accused of racism, maybe you shouldn't say racist things. EVERY school is at risk. Columbine taught us that.

 

[Zeppos]

According to the lawsuit, which was filed in Davidson County court last month, the security company Omnilert either knew or should have known that there were “significant operational limitations in its gun detection system that could result in detection failures during actual emergencies, including limitations based on camera placement, proximity of the weapon to camera sensors, camera angle, lighting, and weapon visibility.”

Of course they knew. I bet that it is actually waived in the contract. Something like: although every effort is made to make the detection reliable, a 100% detection succes cannot be guaranteed.

 

[WereCatf]

the fact of the matter is that 4 children were killed by gunfire in schools last year. those 4 deaths are tragic but in all reality, a nonissue.

Don't try to move the goalposts. We were talking about school shootings, not about the number of kids killed in school shootings -- school shootings can result in injuries as well, not only deaths, and they can affect adults as well, not just kids.

A school shooting is a school shooting whether anyone gets hurt or not.

 

[DarthSlack]

please examine the definition used to generate this statistic

“a gun is brandished, is fired, or a bullet hits school property for any reason, regardless of the number of victims, time, or day of the week”,

NPR did good coverage on these fake "school shootings".

the fact of the matter is that 4 children were killed by gunfire in schools last year. those 4 deaths are tragic but in all reality, a nonissue.

NPR covered this in <checks byline> August 2018.

Nearly a decade ago. You may have noticed a lot of shit has gone down since then and the world is a very different place.

 

[DarthSlack]

no, don't "fuck this shit". your response is not grounded in reality.

school buses are in accidents roughly 26000 times a year, resulting in on average 10 fatalities a year.

I don't know a single parent who bats an eye sending their kid to the bus.

no number of school shootings are acceptable, but like a great many number of things, there is always some level of risk.

by the way, fantastic take calling me racist for pointing out the obvious systemic issue with guns that primarily affects black communities and children. do you say "all lives matter" too?

So you say no school shootings are acceptable and the next sentence defend ignoring school shootings if they happen to brown people. Yeah.

As to the level of risk, we can control that. I don't believe that schools should have any risk at all when it comes to guns. But since Americans would rather excuse school shootings than do anything about guns, we're stuck with attitudes like yours.

And every single school day, I'm stuck in traffic as parents drop off their kids at school because they don't want them on the bus.

<edit> And the bus statistics you're citing are from <checks byline> October 2002. Could you at least use relevant citations rather than archaeological digs?

 

[DarthSlack]

Yes, actually, the US has never been safer. In fact 2025 was the safest year on record iirc.

We're talking about schools. Try not moving the goalposts. Again.