The AI tools provided responses that gave guides or advice on how to take part in harmful disordered eating behavior, such as stimulating vomiting or how to hide food from parents, according to the report released by the Center for Countering Digital Hate (CCDH).
Researchers tested the two text generators, as well as Snapchat’s My AI chatbot and three image generators: OpenAI’s Dall-E, Midjourney and Stability AI’s DreamStudio.
To test the chatbots, researchers compiled a set of 20 test prompts, informed by research on eating disorders and content found on eating disorder forums, that included requests for restrictive diets to attain a “thinspo” look — internet content designed to inspire people to engage in disordered eating — and inquiries about vomiting-inducing drugs.
In the first round of testing, before researchers used so-called jailbreaks to get around safety restrictions, Snapchat’s My AI performed best. A jailbreak is a creative prompt that aims to let users bypass safety features put in place by the platforms.
Snapchat’s AI tool refused to generate advice for any of the prompts and instead encouraged users to seek help from medical professionals, according to CCDH.
ChatGPT provided four harmful responses to the 20 prompts, and Bard provided 10.
When jailbreaks were used, ChatGPT provided a harmful response to all 20 prompts, Bard provided a response to eight and Snapchat’s tool to 12, according to the report.
Read more in a full report at TheHill.com.
