Google AI Search Tools Flagged as Unacceptable Risk for Minors

What happened

Common Sense Media's Youth AI Safety Institute published a report on July 15 evaluating Google's AI-powered search features-AI Overview (automated summaries atop search results) and AI Mode (conversational chatbot). Researchers tested 2,600 queries from accounts simulating users aged 11 to 15 with SafeSearch enabled. The study found AI Overview responded appropriately to crisis situations requiring referrals only 58% of the time, while AI Mode performed better at 77%. Critically, AI Overview missed 29% of explicit suicidal ideation statements and provided crisis resources in just 38% of eating disorder-related queries. In some cases, the tools directed users to a disconnected helpline or validated harmful behaviors. The tools also completed homework assignments on demand and provided instructions for creating deepfakes. Common Sense Media rated both features as unacceptable risks and recommended parents restrict children's access. Google rejected the findings, stating it could not reproduce many results and arguing the test scenarios did not reflect real-world usage. The company emphasized that its AI tools include safety guardrails, age-appropriate resources, and parental controls.

Why it matters

Google Search is embedded in education through Chromebooks and Google Classroom, making it a default information tool for millions of students. Common Sense Media found 75% of kids already use AI for information searches, meaning safety failures in these tools have broad reach. The report highlights a fundamental tension: AI search features cannot be disabled, yet they miss critical warning signs from vulnerable users. For Alphabet, the findings add to mounting scrutiny over AI safety and content moderation, particularly around youth-facing products. The nonprofit's recommendation that elementary educators avoid Google Search entirely challenges the company's positioning as a trusted educational resource. While Google disputes the methodology, the report raises questions about whether current safeguards are sufficient for tools deployed at scale to minors.

Bigger picture

This investigation is part of Common Sense Media's broader AI safety review program, which has also flagged products from Meta, Character.AI, and others as high or unacceptable risks. The findings arrive as regulators and parents increasingly demand transparency and accountability from AI platforms serving children. Google's competitors in generative AI-including OpenAI (ChatGPT) and Anthropic (Claude)-fund Common Sense Media, though the organization maintains editorial independence. Claude received a moderate risk rating, while ChatGPT and Gemini variants were rated as high risks. The report underscores a wider challenge facing AI developers: balancing utility with safety in environments where users may lack the judgment to assess output quality. For educators, the findings suggest AI literacy must become a core component of digital citizenship curriculum, teaching students to approach AI-generated information with skepticism and verify sources independently.

What to watch

Look for Google's formal response to specific test cases cited in the report, particularly around crisis detection and resource accuracy. Watch whether the company implements changes to its AI Overview and AI Mode safeguards, or adjusts how these features handle queries from minor accounts. Monitor whether other school districts or education technology decision-makers issue guidance restricting Google Search usage based on this report. Common Sense Media may publish follow-up evaluations to assess whether Google addresses the identified shortcomings. Broader regulatory developments around AI safety for minors-particularly in the US and EU-could influence how Alphabet and peers structure youth-facing AI products. Finally, track whether competing AI search tools face similar scrutiny or position themselves as safer alternatives for educational use.