Schools are using AI surveillance to protect students. Sometimes arresting them

45 djoldman 75 8/7/2025, 11:17:32 AM apnews.com ↗

Comments (75)

Atreiden · 2h ago
This is utterly dystopian. We say some stupid things as kids, because they're just words and we're missing greater context at that age.

Immediately and automatically engaging law enforcement, and even the FBI, is horrific. Kids have always had greatly restricted freedoms in schools, but transcending the classroom and monitoring their digital lives is just training them to accept the surveillance state.

worldsayshi · 2h ago
The fact that this technology exists might make decision makers feel compelled to always to add as much surveillance as possible and acting on it as diligently as possible. Because it's their responsibility to create safety. And the most short term solution is to always enact more control over everything.
1718627440 · 14m ago
The fact that weapons exist, might make some people feel compelled to always shoot as much as possible.

Sure, these people exist. They are dangerous.

doctorwho42 · 1h ago
Adding more and more safety and control is always a self-defeating policy. It always ends with creating uncontrollable societal discord.

Same thing with unfettered capitalism, the systems only work if we continue to support said systems. When the rules break down, so do the desires of the collective to maintain said systems.

worldsayshi · 56m ago
It seems that a deeper cause of this is lack of trust in long term solutions and the ability to come up with a plan that beat the short term knee jerk solutions - which could make the problems worse.

If we can somehow win back trust in our collective ability to democratically solve problems... that should solve the problem.

I think that involves some creative solutions to collective decision making.

kolektiv · 1h ago
> training them to accept the surveillance state

From the perspective of those pushing this kind of technology and political movement, is that a bug or a feature?

doctorwho42 · 1h ago
Yeah, it's a feature. If anything, its a basement bargain feature. For any large system, indoctrination early and often is the best way for systemic change.

We already see it with the modern surveillance state, post 9-11 the US citizenry has lost so many freedoms and if you ask random people on the street about it they would be perplexed. Hell, even my friends give me a bit of the "ahh so this is your conspiracy theory" look when I mention them. Growing up through 9-11 and the forever war was pretty dystopian, or at least a March into the dystopia's that I only read about in books.

koakuma-chan · 1h ago
> We say some stupid things as kids, because they're just words and we're missing greater context at that age.

I think the problem is that people send kids to public schools and just hope for the best. Imagine you have a brand new child, and you send it to school, and the child ends up saying something offensive, is this the child's fault? I think not. The child was trained on harmful data, it's not surprised the child exhibited undesirable behaviours.

jacquesm · 1h ago
My kid went to public school in Canada. He learned English pretty quickly and one day got suspended from school for a week. When I asked what had happened he said the other kids asked him to pronounce f.u.c.k. and then, after he complied ran to the teacher to say that he'd used a 'bad word'.

So, the principal, one Roman Peredun calls me up and says that my son used a bad word. I asked him what word. He wouldn't say it. So I asked how am I supposed to know how 'bad' my son is if you can't even repeat the word. He then spelled the word. I said, oh, 'fuck'. Yes, that's not in dutch however so he must have picked it up in your school. Peredun hung up and I sent my kid back to school the next day.

giantg2 · 1h ago
I can confirm this. Sent a 5yo to public school and they came home with the new saying "I don't give a fuck!". Not what I want a 5yo to learn. We have massive overreactions by the schools for comments like in the article, but they're culpable in this sort of behavior by creating a largely undisciplined environment for basic in-person behavior. Uncontrolled classrooms and busses lead to all sorts of problems because nobody gets punished on the low end. My kid got in trouble for something and the punishment was to play alone at recess. Really? I told the school I think they should have been in detention at recess, possibly for a couple days. Of course they got punished at home too, which I suppose doesn't happen in some households.
mr90210 · 1h ago
> The child was trained on harmful data, it's not surprised the child exhibited undesirable behaviours.

Your comment reduces children to entities that will behave as expected provided they get fed “good” data.

Humans are not LLMs.

giantg2 · 1h ago
You're taking the inverse of the commentor said. Being fed good data and being good is a different thing than being fed bad data and expecting them to be good.

There are plenty of studies on formative environments, especially on how negative environments can lead to negative behaviors.

koakuma-chan · 1h ago
I think children will behave properly given proper values and education.
giantg2 · 1h ago
I don't think that's universally true, but I do generally agree. It at least raises the probability substantially.
koakuma-chan · 45m ago
Correct. According to Socrates, corruption of majority is unavoidable.
tyleo · 2h ago
I don’t like this article. It mixes two things together:

1. Schools running and monitoring their own communication platforms. This seems fine.

2. The US government monitoring private communication platforms like Snapchat and arresting kids on school grounds. This seems bad.

The way it’s written though mixes these together and makes it look like schools are monitoring the private communication of students. I think it’s fine for schools to monitor there own platforms but weird for the government to monitor all platforms haphazardly.

chasd00 · 1h ago
US schools have way too much authority over children. A student can goto a party on Saturday night and get a ticket for MIP or some other infraction and then face punishment at school in addition to the fine. US schools are quasi authoritarian regimes for teenagers. I don’t believe that is right at all.
cogogo · 1h ago
I mostly agree with you. But the same is true with your employer in the US. You do something stupid outside of work like an OUI and there will likely be consequences with employment - either current or future. Not terrible to internalize that early when the consequences are relatively forgettable.
giantg2 · 1h ago
"1. Schools running and monitoring their own communication platforms. This seems fine.

2. The US government monitoring private communication platforms like Snapchat and arresting kids on school grounds. This seems bad."

Public schools are government.

1718627440 · 10m ago
Public schools get money from the government. That doesn't mean they're are the private arm of the government.
AnimalMuppet · 1h ago
Public schools are not the US government. They're usually city or county.
giantg2 · 1h ago
There are many federal laws and policies enforced in the schools. The police arresting kids on school grounds are also usually state or local.
kayodelycaon · 2h ago
Schools have already monitored student’s social media outside of school and punished them for breaking rules outside of the school.

How long before they use AI to do that?

Permit · 2h ago
AI has nothing to do with this. It’s trivial to build a keyword based system to detect someone threatening/joking that they will kill all the Mexicans at their school.

AI hype is at play here as well, not only in the breathless press releases from AI companies.

chasd00 · 1h ago
I don’t believe a school has a right to monitor students outside of school but getting wind of a mass murder threat and turning it over to the authorities is the right thing to do.
abtinf · 2h ago
You have no problem with the opening example in the article?
tyleo · 2h ago
I have a problem with the degree of punishment but not that the student was punished.

A student said, “on Thursday we kill all the Mexico’s,” on a schools private communication platform. The school should correct that behavior.

Unfortunately they involved law enforcement. Thats where I see the problem. A better solution would be detention and informing the parents.

worldsayshi · 2h ago
> Unfortunately they involved law enforcement

Sounds like they could've reasoned that they face the least chance for liability if they pushed the responsibility to law enforcement.

chasd00 · 1h ago
A threat like that has to be reported to the authorities. In 99.9% of cases it’s nothing but you never know. You can’t threaten the president without getting investigated, you can’t threaten mass murder at school without getting investigated.
giantg2 · 1h ago
"A threat like that has to be reported to the authorities."

This forced reporting necessarily creates false reports. Under the law, things like terrorist threats are required to be credible or incite panic. Reporting things that aren't credible is arguably a violation of law under any other context, yet they choose to ignore that with these mandatory reporting laws. Basically it creates a situation where nobody is allowed to use their brain - automated conveyer to the criminal system.

FpUser · 1h ago
Investigation is ok given how much shooting happens. Punishment however is way out of proportion. It was a bad joke. Does not deserve any real punishment except maybe couple of hours in a class that explains why jokes like this are bad.
giantg2 · 1h ago
Inappropriate comments or jokes could easily be handled with parent-teacher intervention and some detention. If it's possibly credible, then reporting it to the police to kick off an investigation is good. It seems that nobody really used their brain in the example given - everyone just trusted some automated system's determination instead of making their own.
kayodelycaon · 2h ago
When I was a kid, detention over a stupid joke would have been an extreme overreaction and would result in the teacher being pulled into the principal’s office, not the kid.
JumpCrisscross · 2h ago
> detention over a stupid joke

When you were a kid gun laws were stricter.

giantg2 · 1h ago
Not exactly true. You don't know how old the person is nor their jurisdiction.

Some states have fewer laws, others have more. At the federal level, there are more laws and rules than there were in the 60s or 70s (overall more than at any prior time).

And of course enforcement varies. I remember many people coming to school with guns in their cars during hunting season even though it's not legal.

JumpCrisscross · 2h ago
> Unfortunately they involved law enforcement. Thats where I see the problem

Eh, terminal violence was potentially threatened. Calling the cops seems fine if no teacher or administrator can vouch for the kid. (Particularly if, as is true in this case, the law requires “any threat of mass violence against a school to be reported immediately to law enforcement.”)

To the extent someone fucked up, it’s the cops who allegedly caused the 13-year old to be “interrogated, strip-searched and spent the night in a jail cell.”

soulofmischief · 2h ago
Police are a volatile, uncontrollable solution for anything other than white collar crime. If you introduce police into a situation where your life ot belongings aren't in immediate danger, you should be prepared for them to make someone miserable or hurt someone and you're culpable if they do.
1718627440 · 4m ago
Police exists to deescalate conflicts and prevent harm. If they don't you should file a disciplinary complaint.
fn-mote · 1h ago
> the law requires “any threat of mass violence against a school to be reported immediately to law enforcement.”

The article specifies this is a Tennessee law. The school is in Kansas.

Agreed that the cops screwed up, but the school is also responsible.

Welcome to global privacy trends in 2025.

kayodelycaon · 2h ago
Strip searching a 13-year-old girl and locking her in a jail cell for a joke? Wow. And they did all this without telling her parents.
bryanrasmussen · 2h ago
I remember when in high school in Utah we went on a field trip to a halfway house, for some reason - it was during the time when scared straight was real popular so of course every kid should be scared straight whether they have juvenile offenses or not. It was some sort of law class I think.

Anyway they took us to look at the inmates in the halfway house who were behind bars and then they could come out to the bars and as their part of the whole scared straight exercise they would of course yell stuff at us, which was mainly about how they wanted to have sex with the approximately 16 year old girls on the trip.

Gosh, Utah sure is a morally upstanding place.

yard2010 · 1h ago
This is nefarious.
DonHopkins · 1h ago
>... how they wanted to have sex with the approximately 16 year old girls on the trip.

Just like the President of the United States and his good buddy pedophile sex offender and his groomer/pimp who he wishes well and just transferred to a luxury minimum security prison and is about to pardon. Maybe he should be locked up behind bars too, despite bragging to Howard Stern that he draws the line at 12-year-olds.

https://www.yahoo.com/news/trump-told-howard-stern-2006-2239...

https://www.buzzfeednews.com/article/kendalltaggart/teen-bea...

https://www.politico.com/news/2020/08/04/trump-well-wishes-g...

https://en.wikipedia.org/wiki/Donald_Trump_sexual_misconduct...

7thaccount · 2h ago
Absolutely horrifying
sofixa · 1h ago
A joke which could, out of context, be interpreted to be a threat of violence:

> on Thursday we kill all the Mexico’s

Especially considering the frequency of violence in American schools, can't really blame the school for jumping to conclusions.

giantg2 · 1h ago
"can't really blame the school for jumping to conclusions"

Of course you can blame the school. They were too lazy to look at context and determine if the threat was real and credible. They took the determination of a complex tool as an unquestionable fact. The system supplies the fact that the user's account made the comment. All other facts need to be made by investigation. This statement provides a reasonable suspicion to investigate, but should not exhibit probable cause for an arrest as it requires a threat be credible, incite panick, etc per the specific terrorist threat law. This requires investigation and thought.

sofixa · 1h ago
> Of course you can blame the school. They were too lazy to look at context and determine if the threat was real and credible

Do you think whoever is doing this at the school is a qualified professional, e.g. a child/teen psychiatrist that knows the kid in question well enough, to be able to determine if the threat was real and credible?

giantg2 · 1h ago
You don't have to be an expert to read the context. In normal terroristic threat cases the people involved are not psychologists. The police aren't using any child psychologists either. The problem here is the brainless reliance on what a system spits outs. The world would have ended by now if we relied on automated systems without using common sense (see multiple ICBM radar false positives during the cold war).
jacquesm · 1h ago
Kids saying shit is now equivalent to 'normal terrorist[ic] threat cases'? Or did I miss something. If you can't evaluate the data, don't collect the data.
giantg2 · 1h ago
"Kids saying shit is now equivalent to 'normal terrorist[ic] threat cases'?"

It's terroristic threats. That's the law most of the school shooting threats would get charged under. The real problem is that most states have automatic reporting laws, which means you have to report anything that sounds like a threat even if it isn't. This is the main difference between regular cases and school cases - you end up with a lot of junk being reported and potentially causing more harm than it was intended to prevent.

jacquesm · 9m ago
A 13 year old girl is not able to make a credible generalized terrorist threat against a large swath of the population, and besides, the first thing you should do when a kid starts making dumb statements like that is check in with them and their parents, not to call the police. Automatic reporting laws that lead to 13 year old girls being strip-searched are not something anybody should want.

There is an easy way to stop 99.9% of all school shootings, and it isn't 'automatic reporting laws'.

astura · 1h ago
It's a threat of violence in context as well.
malwrar · 2h ago
This will condition children to think this sort of surveillance is normal, and when they’re adults the ones who think it kept them safe from mass shootings will try and advocate using our existing mass-surveillance powers to proactively monitor everyone like this. Please, we need to stop terrifying children with this lazy oppression, this is not worth the damage to society we’re causing by conditioning kids this way.
password321 · 2h ago
Half the point of school has always been to condition them to behave and think a certain way for adulthood.
astura · 1h ago
This sort of surveillance is normal, though. Most platforms are monitored for threats of violence. Are you thinking the outcome would have been much different if she threatened to kill Mexicans on Teams at work instead?
malwrar · 9m ago
The child in OP sounds to me like she thought she was making a bad-taste joke in a private form, and was shocked when it promptly led to an entanglement with an unfeeling system who was looking over her shoulder. I’ve never threatened to kill anyone in my work DMs, but I’ve definitely written stuff that I wouldn’t post in public threads. I think we all to some degree use these “private” systems this way until it burns us, only then do we adjust. Privacy is much more a feeling than a technical reality.

In that sense, it isn’t “normal”, it’s just “something that’s happening in theory but eh maybe it only affects scary people or whatever idk”. I feel like this tolerance we’re developing for outside forces invading “private” spaces, nominally for these loose justifications of harm reduction, will be what _actually does_ make it normal.

Once it’s truly normal, and people think it’s what keeps them safe from mass shootings or whatever, it will be too late to get rid of it. I think fear and normalcy will motivate its spread to places beyond school chat platforms and Snapchat.

SeanDav · 1h ago
What the article seems to have glossed over is this:

>>> A court ordered eight weeks of house arrest, a psychological evaluation and 20 days at an alternative school for the girl.

If one assumes that the court did take into consideration context and age, it appears to largely validate the follow up decision once flagged. (I don't agree with lack of parental contact, to be clear.)

kayodelycaon · 1h ago
I’ve been at small courts. I wouldn’t assume the court took any of that into consideration.

If the police were willing to make an example out of someone, the judge they work with is likely to do that as well.

evaXhill · 1h ago
Not that surprising that things have gotten to this point, just a few days ago schools in Florida were testing a new drone defense system against shootings. Between see-through backpacks, armed teachers, metal detectors, and other things you’d think it would be more easier to severely restrict firearm access to under 21 year olds and make the parents criminally liable if they are found to have facilitated access in any way in the wake of a shooting. But yeah i guess dipping into online conversations and immediately notifying both school officials and law enforcement is a good solution (/j)
IshKebab · 2h ago
Even ignoring the whole idea being awful, their response to someone who they think might want to enact a school shooting is to strip search and imprison them. What effect do they think that is going to have?

Not the sharpest bullets in the barrel...

giantg2 · 2h ago
The problem are the lazy idiots using the software. This is the reason we need capable humans in the loop for anything important.

Here we have a bad joke. The system flags it. The school sends it to the police. The police detain and interrogate the kid. Everyone is treating the determination of a complex automated system as their own determination. We also have every actor treating this as a credible threat. For this to be credible, you have to have the means to accomplish it. They gave a timeline. You know you have time to investigate before making an arrest. Problem is, nobody cares.

yard2010 · 1h ago
If we have to rely on competent humans, we are doomed.
giantg2 · 1h ago
Any system will have failures. The goal is to eliminate some of those failures through multiple checks. The automated system can help by bringing up possible issues that couldn't be monitored by humans. But then it should be humans reviewing these alerts for false positives. It should be much like using a security scanning tool. Findings will come up and you need someone to disposition them. Some will need no actions false positives or issues under your risk tolerance, but then others need appropriate responses. It seemes in the cases like the example, you have a intern running the scan tool and turning every finding into a high vulnerability because they don't know any better than to blindly trust the tool.
clwncr · 2h ago
School officials make the dubious claim that "the technology has detected dozens of imminent threats of suicide or violence."

Oh, really? Do they have data that shows a significant reduction in violence since surveillance started, or is this just reframing false positives (that can result in arrest, eight weeks of house arrest, a psychological evaluation and 20 days at an alternative school) as a net benefit. My money is on the latter.

kotaKat · 1h ago
Guaran-fucking-teed the bullied kids are still getting bullied just as bad and the system is still failing those kids while the school can put an outward appearance to the rest of the community that nothing is wrong and everything is perfectly fine inside the walls.

The victims will still be victims, they’ll just be punished by the system even harder for being a victim.

ludicrousdispla · 1h ago
Hopefully none of the students type German into the chat.
scotty79 · 1h ago
> 13-year-old girl

> When a friend asked what she was planning for Thursday, she wrote: “on Thursday we kill all the Mexico’s.”

> Taken to jail, the teen was interrogated and strip-searched, and her parents weren’t allowed to talk to her until the next day,

> She didn’t know why her parents weren’t there.

> A court ordered eight weeks of house arrest, a psychological evaluation and 20 days at an alternative school for the girl.

Wow, 2025 is wild. Police and court should have psychological evaluation instead and maybe some time off without pay to cool off. Protecting children, no matter how many of them they have to traumatize and incarcerate.

DonHopkins · 1h ago
If only it weren't so easy for a 13-year-old to get their hands on an automatic weapon, then maybe we could take jokes like "on Thursday we kill all the Mexico’s" more casually.

As well as worrying about how to prevent kids from getting their privacy and freedom of speech violated, maybe we could put some more effort into preventing them from getting shot up in schools so often. Or at LEAST as much effort as we used to with assault weapon bans, which worked.

Until then, thoughts and prayers for their lost privacy and freedom of speech, too.

internalfx · 2h ago
It's 1984 in Lawrence, KS...
cramcgrab · 2h ago
And with the recent cell phone ban you have no proof, no video or pictures.
soulofmischief · 1h ago
When I was in high school, the police illegally body searched my entire school as a racist flex after tackling, beating and tasing a student over a pack of cigarettes one foot outside the door of my math class. A student responded by filming them and handing it over to a local news station, who ran the story and footage.

To retaliate, the next day administrators had metal detector wands waiting for us right off the busses, took every single cellphone they found and locked them up at the school board office for the rest of the year.

That school was absolute hell, a battleground between students and teachers. I am not exaggerating at all when I say that being spotted outside your classroom was an immediate expulsion, with not even enough time between classes to pee or use your locker (two minutes, we had to run). As part of the escalation, the fire alarm began being pulled at least twice a day. Any student who had even a moment alone with one would pull it immediately. Absolute chaos and a direct result of power collapse due to a racist, authoritarian school board that only knew how to wield institutional violence.

For this reason, my kid will always have a phone in order to protect themselves from administrative abuse. I will fight for that tooth and nail. I had over 40 write-ups in just elementary school for refusing entertain abuse from authoritarian staff, and I'll be a failure if my kid doesn't walk the same path.

tyleo · 2h ago
Did you even read the article? For the example at the top, all the stuff these kids did was well documented. I don’t understand how having a phone helps here except to add even more distraction in school and another avenue for surveillance.

The kids used a school communication program to say something racist. Schools should monitor school communication platforms.

The only thing I disagree with is the level of punishment (sending a kid to jail for a night).

7thaccount · 2h ago
Yes. I think some are pointing out that this crazy level of surveillance is at Orwellian levels. Why are kids even messaging each other on these platforms? I know books and paper is old school, but it does the job well. If they have to use some kind of internal system, then a racist joke could've led to a detention and parent/teacher talk. It didn't need to auto-flag law enforcement and traumatize the kid for life. Involuntary commitment at school sounds as unAmerican as it gets.
internalfx · 2h ago
AND strip searching them, AND not allowing communication with their parents...
tyleo · 2h ago
I don’t understand your point. It seems like you are saying, “this punishment is so severe we shouldn’t be correcting the behavior at all.”