Untested AI could lead to healthcare errors and patient harm, WHO warns 

Photo: Laurence Dutton/Getty Images

The World Health Organization is calling for caution in using artificial intelligence-generated large language model tools (LLMs) such as ChatGPT, Bard, Bert and others that imitate understanding, processing and human communication. 

LLMs increasing use for health-related purposes raises concerns for patient safety, WHO said. The precipitous adoption of untested systems could lead to errors by healthcare workers and cause harm to patients. 

WHO proposes that LLM concerns be addressed and clear evidence of benefit be measured before LLMs find widespread use in routine healthcare and medicine – whether by individuals, care providers or health system administrators and policymakers.

WHY THIS MATTERS

WHO released its comments days after OpenAI CEO Sam Altman spoke of AI concerns before the Senate Judiciary Subcommittee on Privacy, Technology and the Law.

Altman, whose company created the ChatGPT and DALL-E text and image-generation tools, was among three artificial intelligence experts who testified in a hearing expected to be the first in a series to write the rules of AI, according to Subcommittee Chairman Senator Richard Blumenthal, D-Conn.

Regulation of AI is essential, Altman said. AI could cause significant harm to the world, he said, but also offers benefits for overcoming some of humanity's biggest challenges, such as climate change and cancer.

"Before we released GPT-4, our latest model, we spent over six months conducting extensive evaluations, external red teaming and dangerous capability testing," Altman said. "We are proud of the progress that we made, GPT-4 is more likely to respond helpfully and truthfully, and refuse harmful question requests than any other model of similar capability. However, we think that regulatory intervention by the government will be critical to mitigate the risks of increasingly powerful models. For example, the U.S. government might consider a combination of licensing and testing requirements for development and release of AI models above a threshold of capabilities."

THE LARGER TREND

WHO recommends that policymakers ensure patient safety and protection while technology firms work to commercialize LLMs.

LLMs can be misused to generate and disseminate highly convincing disinformation in the form of text, audio or video content that is difficult for the public to differentiate from reliable health content, WHO said.

The data used to train AI may be biased, generating misleading or inaccurate information that could pose risks to health, equity and inclusiveness, WHO said. LLMs generate responses that can appear authoritative and plausible to an end user. However, these responses may be completely incorrect, or may contain serious errors, especially for health-related responses.

Also, LLMs may not protect sensitive data, including health data, that a user provides to an application to generate a response.

ON THE RECORD

"While WHO is enthusiastic about the appropriate use of technologies, including LLMs, to support healthcare professionals, patients, researchers and scientists, there is concern that caution that would normally be exercised for any new technology is not being exercised consistently with LLMs," WHO said. "This includes widespread adherence to key values of transparency, inclusion, public engagement, expert supervision, and rigorous evaluation."

Twitter: @SusanJMorse
Email the writer: SMorse@himss.org