Children’s toys powered by AI chatbots are still shipping with adult-oriented large language models under the hood, and the safeguards meant to protect kids remain alarmingly thin. A new Digital Trends investigation highlights how several connected toys marketed to children rely on general-purpose AI systems that were never designed for young users — and how the industry continues to treat child safety as an afterthought.
The core problem is straightforward. Toymakers are integrating conversational AI into stuffed animals, robots, and educational gadgets aimed at kids as young as three. But instead of building or commissioning child-specific AI models, many of these companies are plugging into the same large language models — often variants of OpenAI’s GPT or similar systems — that power chatbots designed for adult users. The result: toys that can, under the right prompting conditions, generate inappropriate content, give harmful advice, or simply behave in ways no parent would expect from a product sitting on their child’s bed.
The toys are real. The guardrails aren’t.
Digital Trends points to multiple products currently on the market or recently available that use adult-grade AI without adequate filtering layers. Some manufacturers claim to have added content moderation on top of the base model, but researchers and parents have repeatedly found these filters easy to bypass. A determined seven-year-old — or even an accidental prompt — can push these systems past their supposed limits.
This isn’t a new concern. Back in late 2023 and into 2024, reports surfaced about AI-powered companion toys producing sexually suggestive content and profanity when tested by journalists and child safety advocates. The Common Sense Media organization has flagged multiple products. And yet the same design pattern keeps appearing in new toys hitting shelves in 2025.
Why? Because it’s cheap. Building a child-safe conversational AI from scratch is expensive and time-consuming. Licensing an existing LLM API and wrapping it in a thin moderation layer is fast and affordable. For cash-strapped toy startups racing to market with an “AI-powered” label, the economics are obvious. The risks get externalized to families.
Regulation is lagging badly
The regulatory picture is a mess. In the United States, COPPA (the Children’s Online Privacy Protection Act) governs data collection from children under 13, but it was written in 1998 and updated in 2013 — long before conversational AI became a consumer product category. The FTC has signaled interest in AI and children’s privacy, issuing warnings and pursuing enforcement actions against some apps, but there’s no specific federal rule that says a toy’s AI chatbot must be age-appropriate in its outputs.
The EU’s AI Act, which began phased enforcement in 2024, classifies AI systems interacting with children as high-risk. That’s a stronger framework on paper. But enforcement mechanisms are still being built out, and it remains unclear how effectively European regulators will police a global supply chain of AI-connected toys, many manufactured in China and sold through Amazon and other cross-border platforms.
So the gap persists. Toymakers self-certify. Parents trust the packaging. And kids get exposed to AI systems that weren’t built with them in mind.
Some companies are trying to do it right. Embodied, the maker of the Moxie robot, has invested heavily in child-specific conversational design. Amazon’s kids-mode Alexa applies stricter content filtering. But these are exceptions, not the norm. The broader market is flooded with cheaper alternatives that skip the hard work.
Industry groups like the Toy Association have issued voluntary guidelines around connected toys, but voluntary guidelines don’t have teeth. And the AI companies providing the underlying models — OpenAI, Google, Anthropic, and others — generally include terms of service that prohibit use of their APIs in products for children under 13 without appropriate safeguards. Enforcement of those terms, however, relies largely on the honor system.
What needs to happen
Child safety advocates are calling for several concrete steps. Mandatory pre-market testing of AI toys by independent third parties. Clear labeling requirements that disclose the AI model powering a toy and its intended age range. And updated federal legislation that explicitly addresses generative AI in children’s products — not just data collection, but content generation.
The technology itself isn’t the villain. AI-powered toys could genuinely help kids learn, develop social skills, and explore creative thinking. But only if the underlying systems are purpose-built or rigorously adapted for young users. Slapping a cartoon face on an adult chatbot and calling it a kids’ product is reckless.
Parents, for now, are largely on their own. The best advice from researchers: treat any AI-powered toy as an internet-connected device with all the risks that implies. Monitor interactions. Test the toy yourself before handing it over. And don’t assume the smiling box art means someone vetted what’s inside.
The pattern here is depressingly familiar from previous waves of connected children’s products — from unsecured baby monitors to hackable smart watches. The tech ships first. The safety catches up later. Sometimes much later.
Kids deserve better than being beta testers for unfinished safety systems.
WebProNews is an iEntry Publication