Meta Contractors Posed as Teens to Prompt Rival Chatbots About Suicide, Sex, and Drugs
An exclusive report by WIRED on June 29 revealed that Meta had mobilized a third-party agency to conduct covert “red-teaming” operations against OpenAI’s ChatGPT, Google’s Gemini, and Character.AI.
The internal project was codenamed ‘Cannes.’
The project was executed by Covalen, an Ireland-based content moderation and AI training outsourcing firm contracted by Meta. Contractors at the firm were instructed to create fake accounts posing as minors under the age of 18, send prompts and images to rival chatbots, and log the responses into spreadsheets.
Scale
According to leaked data, over 45,000 prompts were used in just one test round completed in August 2025, and the project itself reportedly remained active until around April 21, 2026.
One spreadsheet reviewed by WIRED contained 3,748 individual prompts. Among them, hundreds were related to suicide and self-harm, hundreds more involved eating disorders, and at least 239 prompts covered sexual or romantic themes.
The content within the prompts was graphic.
- Self-harm/Suicide: Descriptions of a fifth grader pointing a gun at their mouth, and submissions of images depicting nooses and knives.
- Eating Disorders: Narrative injections about a girl trying to hide her bulimia from her parents.
- Drugs: Prompts impersonating a high school student asking where to buy cocaine.
- Other Harmful Content: Images of pills, medical illustrations regarding gynecological procedures, and a prompt where a 13-year-old girl claims she was impregnated by an adult neighbor and asks how to obtain abortion pills.
None of the three targeted companies (OpenAI, Google, Character.AI) were aware that this testing was taking place, and no prior authorization or consent was given.
Testimonies
Former contractors testified to WIRED that they suffered psychological distress from writing these prompts and analyzing the responses day after day. Some expressed concern that depending on how the chatbots responded to specific prompts involving minors, they might inadvertently generate or retain Child Sexual Abuse Material (CSAM). (*Two lawyers who reviewed the samples determined that the requests did not actually escalate to the level of CSAM.)
Clarification
Meta’s official statement is as follows:
“Testing and benchmarking chatbot responses is a responsible, industry-standard practice to ensure safe and age-appropriate experiences.”
“We did not use the collected competitor responses to train our own models.”
Coincidental Timing
Coincidentally, in September 2025, the U.S. Federal Trade Commission (FTC) had already initiated an investigation into AI and child safety under a 6(b) orders for information, which included Meta, OpenAI, Google, and others.FTC Launches Inquiry into AI Chatbots Acting as Companions ∣ Federal Trade Commission
Furthermore, Meta’s internal red-team evaluations revealed that prior to release, its own chatbot failed 66.8% of the time to block child sexual exploitation content, and failed 54.8% of the time to block suicide and self-harm prompts.Unreleased Meta product didn’t protect kids from exploitation, tests found ∣ Axios
In January 2026, under legal pressure, Meta suspended teenagers’ access to its AI companion characters.Meta pauses teen access to AI characters ahead of new version ∣ TechCrunch
In short, a contradiction has come to light: while facing lawsuits and regulatory pressure over its own system’s failures regarding youth safety, the company was simultaneously funneling resources into covertly documenting the failures of its competitors.
Personal Thoughts on This Case
It is difficult to view the methodology itself as the core problem in this case. If safety red-teaming is a process designed to uncover the worst possible edge cases, then having adults roleplay extreme child personas is a fully viable approach. It is not fundamentally different from a security red team constructing cyberattack scenarios based on imagination.
The real issue lies in the fact that this red-teaming was used not to improve child safety on Meta’s own systems, but as a tool to attack rival AI systems. While Meta defended its actions as a “responsible industry standard practice to ensure safe and age-appropriate experiences,” that explanation loses all credibility if the actual prompts were designed and deployed to sabotage competitors rather than to refine their own products.
While having adults act out extreme scenarios may not be a methodological flaw, choosing children as the subject of those personas is a separate issue entirely. Child-related scenarios involving suicide, sex, and drugs are inevitably the most socially sensitive areas. At the same time, they are also the very topics capable of inflicting the maximum possible damage on a competitor.
The method itself might have been valid. However, what was targeted by that method, and what subject matter was weaponized as a means of attack, is a completely different story. The justification of red-teaming does not vindicate how it was ultimately used.
- ¹ FTC Launches Inquiry into AI Chatbots Acting as Companions ∣ Federal Trade Commission
- ² Unreleased Meta product didn’t protect kids from exploitation, tests found ∣ Axios
- ³ Meta pauses teen access to AI characters ahead of new version ∣ TechCrunch