These connected companions could disrupt everything from make-believe to bedtime stories. No wonder some lawmakers want them banned.
See full article...
See full article...
The new Wild West of AI kids’ toys
- Thread starter JournalBot
- Start date
Show me the front and back of Daddy's credit cards and then you can make a wish for anything you want.
Upvote
49
(49
/
0)
That's an odd take. The approach to them you seem to have is "Let's use my child to experiment on and see if it's suitable". I think most folks would rather that it be extensively safety tested prior to introducing them to the spawn of our loins.
But WRT AI, you also seem to presume that if the AI will stays within its boundaries for a short time, it will always continue to. So far, that's not been the case. Especially for the interactive kinds of AI out there. LLM's are just too unreliable to not jump the track.
Going in the other direction, if you impose too strict of a guardrail, the toy quickly becomes uninteresting, like the pull-string talking dolls of my youth. Toy makers should use some imagination to create engaging toys and not leave it up to very unpredictable and unreliable LLM's to do it for them.
Upvote
13
(13
/
0)
So these Toys are:
- A Subscription Service?
- Linked to your Home Intranet?
- Collecting data?
- Real Time Generate un-pre-screen-able content?? (I.E. not like video game ESRB or MPA film ratings?)
- Are Live Alpha/Beta Testing LLM-AI product/service on not legally able to consent children?
- Made of LLM-AI that scrapped the all the rando of the World Wide Web?
- Actively linked to Sociopath Controlled Businesses, Political Institutions/Nation States, and/or (coming soon to a darker reality near you) Religions? (LLM-AI Gumby anyone?)
- Spying? (Your child ask Toy "X" about "Y". We would like to ask you, Mr./Misses/ Child Parent/Guardian about "Y")
- Engaging in Advertising? Propaganda? Proselytizing? (No, it's "Teaching". It's "Education")
- A scam that will brick in X months when the business case is no longer monitarily profitable to maintain the service infrastructure?
Upvote
49
(51
/
-2)
Anyone else getting reminded of Squee's Tickle Me Hellmo?
https://www.newgrounds.com/portal/view/206278
https://www.newgrounds.com/portal/view/206278
Upvote
0
(0
/
0)
Upvote
28
(29
/
-1)
This sounds as bad as you can imagine with what basically is a furby with upgraded security risks.
Kids might be exposed to all sorts of AI hallucinated stuff, but those things do come with a microphone and an internet connection, so all sound information is going to be stored somewhere. And all that data is a goldmine to be scraped for useful information.
Kids might be exposed to all sorts of AI hallucinated stuff, but those things do come with a microphone and an internet connection, so all sound information is going to be stored somewhere. And all that data is a goldmine to be scraped for useful information.
Upvote
13
(13
/
0)
Your child will be left behind and will never be an AI native if you don't get them a plastic puppy with an always-on microphone that tells them to eat lead paint chips.
Upvote
14
(14
/
0)
For a lot of people who just ride the hype and use the next new thing because they assume everyone else is doing it, this scenario might pique enough self-interest to have them stop and think about consequences.
You're not going to be able to give off the outward appearance of the photogenic ideal of a family if your kids are maladjusted and exhibiting addicted behaviours at a level that was inconceivable in your own childhood. Wouldn't that make you a bad parent?
These are the same potential disadvantages, just from another cause, that we might attribute to absentee parents, or to families with low socioeconomic status. Why would you want that for your children? They're going to have to be schooled either with the delinquents or with the seriously mentally ill, and you're going to spend their entire childhood fighting uphill battles against the system, or paying for the highest level of private schooling because of it. Why would you want that for yourself?
Upvote
7
(7
/
0)
Connecting an LLM to a children's toy is one of the most perverted ideas I've come across yet. No amount of master prompt tweaking is going to make that safe for a child to interact with. I also worry about the potential of priming a generation of young people to fall to AI psychosis.
Upvote
21
(21
/
0)
Those other risks were around with toys before LLMs came around. Barbies and Robosapiens and other toys with cameras, microphones, and online-enabled features were security nightmares more than a decade ago.
Upvote
12
(12
/
0)
Two thoughts - seems like any model that was trained on terabytes of adult content cannot be used in any toy, no way to add enough controls and rules to make it safe. They need to train new models from scratch using only age appropriate content. Any products doing that?
And second the toys really need to be multimodal, trained with age appropriate vision data. Kids are just learning language, so much of the context is visual, what they are holding, where they are standing, etc. Its like talking to a toddler on a phone, how well do those conversations go?
And second the toys really need to be multimodal, trained with age appropriate vision data. Kids are just learning language, so much of the context is visual, what they are holding, where they are standing, etc. Its like talking to a toddler on a phone, how well do those conversations go?
Upvote
7
(9
/
-2)
Any model meeting those requirements will still occasionally tell kids they can eat delicious rocks because they're still LLMs.
Upvote
17
(17
/
0)
OTOH, what I really really want to try is to get two of these things talking to each other to see how quickly their conversation devolves into unintelligible gibberish through model collapse. Much more amusing then inflicting them on young kids.
Upvote
15
(15
/
0)
Thanks, I just looked up lawn darts and found a very "interesting" X-Ray of someone who got hit. If you ask me, that looks like more of "prove knowledge" territory than complete ban. For example, have them only available if you show a shooting/archery range membership or other training certificate.
(The key is relative safety: if everything had to be 100% politically cushy for kids, I don't know how you'd have adult cooks or construction workers since cutting tools and heat are so "scary".)
For the AI toys, I'd currently support a ban if asked to vote because the AI orgs have not demonstrated ability to show people how to safely operate them. We don't know if there is a way to use these without harming a kid's neuroplastic developing mind, or even adult minds for that matter.
Maybe we could have them in the license territory as well: I'd love to see someone with the AI toy license on their wall, acknowledging that they went through the paperwork and mandatory course to get their kid's data to be scraped all day long by a mega-corp, lol.
Upvote
-3
(0
/
-3)
I dunno about that. I had one and took it to bed with me. It started talking in the middle of the night, I think there was a thunderstorm at the time. Scared the absolute shit out of me.
That was the end of Teddy Ruxpin.
Upvote
24
(24
/
0)
The US has UL for electrical safety, and Europe has agencies testing to ENs, like TüV. If you understand UL or CE that's enough.
But there seems to be no equivalent for software safety. And it is needed.
Upvote
22
(22
/
0)
I'm surprised any reputable company would want to go in on this given that there's still no consistent way to build ironclad guard rails. It's not gonna be a great look when the teddybear starts spouting slurs and currently it's very hard to actually stop that.
In terms of stuff you can do with AI, using it for like, fictional interactions is probably one of its better use cases but definitely not for children's toys with the way the tech is right now.
In terms of stuff you can do with AI, using it for like, fictional interactions is probably one of its better use cases but definitely not for children's toys with the way the tech is right now.
Upvote
9
(9
/
0)
I'm a fan of banning them, and the reason given is in the article:
The LLMs that are used to power these toys are designed for those who are 13+ years old.
Upvote
10
(10
/
0)
The only way a guardrail will work is this:
1. The toy is limited to "AI Play" for a limited amount of time a day (30 minutes? I don't know).
2. The training performed on the LLM is explicitly designed for children in mind. That means certain naughty words that I can't write here won't be in the training data (so it can't be said), the toy is explicit that it's a toy and make believe, and there's no easy way to violate the guardrails.
Upvote
-14
(1
/
-15)
It'll cost billions of dollars to train new models using only age appropriate content, with responses that are age appropriate (that means, child psychologists, or those who are training to be child psychologists and related jobs, need to review the output and confirm it's child safe).
Upvote
18
(18
/
0)
I don't think I understand the logic here. Some praise glorious 'AI' as being worth its risks because it does things faster; but as a toy what's the point? This 'play' is something that children do more or less for its own sake; and which is of interest to us because it turns out to be crucial to certain human development processes.
Is someone seriously expecting these things to turn their kid into an agentic 10x toddler like the ads aimed at linkedin addicts promise; or is this purely one of those situations where easy novelty is being shoved onto the market because it's easy and, through some sort of sleight of hand, it's on customers to express concerns rather than vendors to demonstrate adequacy?
The ways 'AI' systems are bad match neatly with the cynical incentives of half-assing stuff that doesn't matter as fast as possible; but that seems like less of an endorsement when you are dealing with a situation that you should really just avoid, it'll be considerably cheaper and lower effort, unless you are just thrilled by how totally worth doing it is for some reason.
Is this basically entirely supply driven; or is there an actual customer segment crying out for a faster, worse, way of parenting?
Is someone seriously expecting these things to turn their kid into an agentic 10x toddler like the ads aimed at linkedin addicts promise; or is this purely one of those situations where easy novelty is being shoved onto the market because it's easy and, through some sort of sleight of hand, it's on customers to express concerns rather than vendors to demonstrate adequacy?
The ways 'AI' systems are bad match neatly with the cynical incentives of half-assing stuff that doesn't matter as fast as possible; but that seems like less of an endorsement when you are dealing with a situation that you should really just avoid, it'll be considerably cheaper and lower effort, unless you are just thrilled by how totally worth doing it is for some reason.
Is this basically entirely supply driven; or is there an actual customer segment crying out for a faster, worse, way of parenting?
Upvote
3
(7
/
-4)
AbstractHaderach
Smack-Fu Master, in training
Right on the nose. That film was made precisely because of children being exploited by consumerism. Ah, the best horror always has a positive social message, doesn't it? Texas Chainsaw Massacre - eat more vegetables. Babadook - yes people do get mentally strained to the limit. Evil dead - hm.
Upvote
5
(5
/
0)
you guys are looking at this all wrong and seeing the horrors....
how about a real Ted?
how about a real Ted?
Upvote
-11
(3
/
-14)
They saw The Beast and found it to be an optimistic tale of social progress.
Upvote
2
(2
/
0)
Or their parents had been using it to listen to Stephen King's Cujo and forgot to swap the tape back out.
Upvote
6
(6
/
0)
Resistance
Wise, Aged Ars Veteran
I have an idea for something far better than an AI toy to accompany a child for a pretend tea party:
A human.
Upvote
15
(15
/
0)
There’s a big difference between saying “my kid shouldn’t have this” and “no kid should have this” and I feel like folks have forgotten that recently.
Upvote
-12
(3
/
-15)
Resistance
Wise, Aged Ars Veteran
So as a thought exercise I decided to try and come up with a way to make a product I would not be uncomfortable giving to a cherished child.
My first idea is to limit the outputs, have a few hundred predefined outputs and have the LLM choose one of them based on the input.
The problem is we would have to exclude anything that could be used to answer a yes or no question otherwise an LLM might output a dangerous answer. And already I have hit a roadblock rendering the toys interactivity completely non functional.
Gee, I thought about that for all of a minute and concluded that this is a horrible idea, there is no excuse for the people developing and selling these toys.
My first idea is to limit the outputs, have a few hundred predefined outputs and have the LLM choose one of them based on the input.
The problem is we would have to exclude anything that could be used to answer a yes or no question otherwise an LLM might output a dangerous answer. And already I have hit a roadblock rendering the toys interactivity completely non functional.
Gee, I thought about that for all of a minute and concluded that this is a horrible idea, there is no excuse for the people developing and selling these toys.
Upvote
6
(6
/
0)
It looks like Conan predicted the future of kid's toys with Wikibear:
View: https://www.youtube.com/watch?v=9cKiAH95V_8
View: https://www.youtube.com/watch?v=9cKiAH95V_8
Upvote
4
(4
/
0)
Honestly, I think at this point I am okay with some Chinese Communist Party talking points getting passed along.
Upvote
-5
(2
/
-7)
chateauarusi
Ars Praetorian
Words are not adequate to convey the escalating horror and despair I felt as I read through this article.
One of the great unexamined ironies of the modern age is that the more knowledge and exposure one has in one's professional life to the real-world design and implementation of today's technologies, the more likely one becomes to lean Luddite in one's personal life. Casual hobbyist tech types may be enthusiastic about AI and unconcerned about things as described in this article — but if you do tech for a living, you instinctively know these factories need to be burned to the ground.
One of the great unexamined ironies of the modern age is that the more knowledge and exposure one has in one's professional life to the real-world design and implementation of today's technologies, the more likely one becomes to lean Luddite in one's personal life. Casual hobbyist tech types may be enthusiastic about AI and unconcerned about things as described in this article — but if you do tech for a living, you instinctively know these factories need to be burned to the ground.
Upvote
23
(23
/
0)