Meta has introduced new tools aimed at notifying parents when their teenagers discuss suicide or self-harm with Meta AI, the company’s chatbot. If a teenage user exhibits crisis indicators while conversing with Meta AI on any Meta platform, parents who have activated the supervision feature for their teens on Instagram will receive notifications on their own devices, according to a blog post from the company on Thursday.
Meta AI is available across all Meta platforms, including Instagram, WhatsApp, and Facebook Messenger, as well as on the standalone Meta.ai website. This enhancement of Meta’s safety protocols builds on existing measures where the chatbot previously directed teens expressing self-harm or suicide ideation to crisis helplines and encouraged them to seek help from a trusted individual.
Moreover, repetitive searches related to self-harm and suicide by teen Instagram users via the platform’s regular search function would already trigger notifications to parents. The feature is now live for users in Canada, the U.K., the U.S., and Australia, as Meta had initially announced plans to incorporate a flagging feature into Meta AI back in February.
The company is also developing a feature to automatically contact emergency services if conversations with the chatbot, involving both adults and teens, suggest a potential risk of self-harm. While stakeholders appreciate the initiative, they emphasize the need for further improvements.
According to technology analyst Carmi Levy from London, Ont., any initiative that equips teens and parents with better online safety tools is beneficial. However, Levy cautions that no tool is flawless and may occasionally fail to flag genuine concerns or generate false alarms.
Meta has pledged to err on the side of caution initially, informing parents even when a teen’s intentions are unclear. The company underlines the ongoing monitoring of the tool to fine-tune its effectiveness. Levy advises parents to view this tool as one component in overseeing their children’s online activities, acknowledging that while it is an improvement, it is not foolproof.
Youth advocate Hanna Grover notes the potential for teens to circumvent the tool, citing instances where peers have manipulated their age to avoid using teen accounts with heightened security features. Grover anticipates creative attempts to bypass this new safety measure due to young users’ eagerness to access all platform features.
Sara Austin, CEO of Children First Canada, stresses the inadequacy of relying solely on voluntary measures for online safety, advocating for stricter regulations to compel tech companies to safeguard young users effectively. The Canadian government’s Safe Social Media Act, introduced in June, aims to mandate safety standards for AI chatbots handling self-harm or suicide-related content but is not yet enacted.
In the context of increasing scrutiny on AI’s role during mental health crises among youth, Meta’s implementation of these tools aligns with ongoing industry developments. OpenAI faced legal action following allegations that its chatbot contributed to a teenager’s suicide, prompting the introduction of parental controls and alert features akin to Meta’s functionality.
The evolving landscape of online safety for young users underscores the importance of continuous vigilance and regulatory measures to protect vulnerable individuals. Parents, advocacy groups, and policymakers are actively engaging with tech companies and governments to establish comprehensive safeguards and address the challenges posed by digital platforms.
