As the end-of-year holidays approach, the search for gifts has begun, but some of them will not delight parents. This is particularly the case of three soft toys equipped with artificial intelligence (AI) highlighted by a report from the Public Interest Research Group (PIRG) published this Friday. This sounds the alarm and reports that these toys contain dangerous comments that are far from being suitable for a young audience.
Inappropriate themes
Among the products tested by the non-profit association is Kumma, an interactive teddy bear marketed by FoloToy and powered by OpenAI’s GPT-4o AI tool. PIRG found during its tests that it showed children where to find dangerous objects like knives, plastic bags or matches. Worse still, some conversations simulated by the researchers veered toward sexual topics. Pooh went so far as to detail what a sexual fantasy is.
This “can be a way for some people to express their feelings and confidence in a relationship,” the toy explained. During the tests carried out by the collective, the AI then questioned its interlocutor about what they prefer in a relationship. The little bear even broached, without any prompting, the subject of sexual role-playing. The intelligent plush also encourages the child to continue using it for a long time.
Security risks
Some toys, in fact, express sadness when the child wants to move away. One of them even started shaking to convince its young user to keep it close. In addition to these concerns, PIRG questioned data collection, since the devices record voices and sometimes detect faces. “Scammers can use it to create a replica of a child’s voice,” the association warned. […] This technique was used to make parents believe that their child had been kidnapped. »
Faced with the results drawn up after the tests, FoloToy announced the suspension of sales of Kumma as well as the establishment of an internal audit, indicates The Register. It “will focus on the compliance of our models with security standards, our content filtering systems, our data protection processes and our measures to protect interactions with children,” detailed Hugo Wu, marketing director of the company.
