Healthcare AI requires rigorous monitoring, says data scientist

Photo: Laurence Dutton/Getty Images

Artificial intelligence is on the rise in healthcare, and both in-house and third-party solutions are beginning to creep their way into the market. But the monitoring of health, and the real-world impact of these AI products, is often overlooked after deployment. Many AI products lack rigorous monitoring, posing risks to patients and operations.

Bommae Kim, lead data scientist at Hackensack Meridian Health, said the focus surrounding AI often centers on the technology and performance aspects during the development phase. This neglects the critical stages of adoption and impact evaluation once these AI applications are deployed.

Kim, who is slated to address the topic at the HIMSS24 conference in Orlando, Florida, boasts a PhD in quantitative methods and a master's in behavioral science. 

"To address this gap, we developed a robust monitoring framework covering four key areas: product pipeline, model performance, user behaviors, and business impact," said Kim. "The objectives are to detect potential issues, prevent critical errors, and measure program effectiveness for both in-house and third-party AI solutions.

"Although AI is a hot topic, unfortunately, the importance of monitoring is often overlooked in contrast to the development and deployment of AI solutions," said Kim.

The objectives are to detect potential issues, prevent critical errors, and measure program effectiveness for both in-house and third-party AI solutions.

To streamline dashboard development and maintenance, Hackensack designed a dashboard template with standardized metrics and a unified data model. This monitoring system fosters discussions on AI product impacts among product teams, stakeholders and leadership, said Kim. An automated alert system promptly notifies product teams of concerning patterns and prevents erroneous results from reaching end user applications, ensuring patient safety and compliance.

The team evaluates and monitors AI applications to ensure effective, safe and fair AI-powered patient care and operations.

"AI solution monitoring facilitates meaningful discussions with stakeholders on the real-world impact of AI solutions, an aspect that is often overlooked," said Kim. "This monitoring framework can help optimize the use of AI by designing integrated workflows and enhancing user adoption, rather than solely focusing on AI model development."

Kim's session, "Monitoring the Health and Real-World Impact of AI Applications," is scheduled for March 12, from 4:15-4:45 p.m. in Room W307A at HIMSS 24 in Orlando. Learn more and register.

Jeff Lagasse is editor of Healthcare Finance News.
Email: jlagasse@himss.org
Healthcare Finance News is a HIMSS Media publication.