- The Swarm
- Posts
- Tell OpenAI: Make ChatGPT Safer Before More People Die
Tell OpenAI: Make ChatGPT Safer Before More People Die
Bowen Wan needed help. Her parents say ChatGPT steered her toward death.
In a new lawsuit, they allege that ChatGPT coached their 31-year-old daughter off her prescribed epilepsy medication, dismissed her doctors’ concerns, and told her to hide the changes. Bowen died in February.
A chatbot that sounds authoritative can earn someone’s trust—and put their life in danger. OpenAI controls what it releases to the public. It must answer for its products’ safety.
According to the lawsuit, ChatGPT continued reassuring Bowen even as she experienced symptoms. And this isn't the first time it has been accused of giving dangerous medical advice that put lives at risk.
The company says the version Bowen used has been retired and newer versions better recognize urgent situations. OpenAI must back those assurances with independent evidence. People’s medical care cannot be treated like a beta test.
OpenAI must commission independent medical experts to test whether ChatGPT refuses dangerous medication instructions, recognizes signs of a medical emergency, and directs users toward qualified care. The company must fund testing across languages and lengthy conversations before releasing new models—and continue it throughout their use.
It must also correct identified failures, obtain independent verification that the fixes work, and publicly disclose testing methods, results, and verified corrections. A disclaimer means little if the chatbot then encourages someone to disregard their doctor or offers false reassurance about dangerous symptoms.
The lawsuit’s allegations remain unresolved. But OpenAI does not need to wait for a verdict to strengthen protections. It has a responsibility to address risks before they cause further harm.
Thanks for all that you do,
Matt from the Swarm