Researchers warn about ChatGPT's new health service

"In more than half of the cases where a patient should have been sent to the hospital immediately, the system instead advised them to stay home or get a regular doctor's appointment."

"The service performed better in clear emergency situations, such as strokes or severe allergic reactions, but had trouble handling more complex or ambiguous symptoms."

"The researchers also point to shortcomings in how the system handled suicide risk, where warning functions sometimes disappeared depending on what additional information was added to the scenario."

A Nature Medicine study evaluated ChatGPT Health's safety by testing it against 60 realistic patient scenarios ranging from mild to acute conditions. Three physicians pre-assessed required care levels and compared them to the AI's recommendations. Results showed the system failed to recommend immediate hospitalization in more than half of cases where it was medically necessary. While ChatGPT Health performed adequately with clear emergencies like strokes or severe allergic reactions, it struggled significantly with complex or ambiguous symptoms. The study also identified critical gaps in suicide risk assessment, where warning functions inconsistently appeared based on scenario details.

Read at Computerworld