Skip to main content

HIMSS AI Forum: Implementing agentic AI in healthcare requires caution

Agentic AI offers the potential to streamline clinical workflows, but healthcare organizations must carefully assess its risks to ensure safe and effective implementation.
By Jessica Hagen , Executive Editor
Agentic AI panel at HIMSS AI Forum 2025
Photo courtesy of Anthony Vecchione

NEW YORK – At the HIMSS AI Forum on Wednesday, during the talk "The Future of Healthcare with Agentic AI," panelists discussed how healthcare is using agentic AI, which is artificial intelligence designed to act autonomously to make decisions, take actions and adjust to achieve a specific goal with minimal human intervention. 

"They can be standalone agents that actually act autonomously, but they are now part of a workflow that spans multiple departments, or could be deployed into multiple departments in an organization," Lyle McMillin, AVP of product management at Hyland, said during the panel. 

"What we're seeing and what organizations need to prepare for is that the type of data you need to start working with is going to expand. More than half the data in a healthcare organization is unstructured. You've probably spent a lot of time learning how to get your structured data into the format you need. Now you've got to start thinking through what do I do with the unstructured data."

Dr. Lukasz Kowalczyk, gastroenterologist, CEO and founder of Soothien HealthTech Advisory, and moderator of the panel, highlighted the need for healthcare organizations also to understand how agentic AI fits into workflow and where to start when implementing the technology. 

Jason Smith, venture fellow at Matter, said the first thing an organization should ask itself is, "Why move from AI to agentic AI?" 

Smith emphasized the newness of the technology, suggesting that organizations must understand the desired outcome and implement it slowly. 

"Find the workflows where you can do something that does not have a lot of variance in the workflow," Smith told the audience. 

"Maybe it's summarization; maybe it's pre-op documentation summarization. It is things where you can start to teach [the AI] a little bit, and you and your organization can start to understand that one piece of an outcome. You can start to understand the basics of LLMs." 

Dr. Jonah Feldman, medical director of clinical transformation and informatics at NYU Langone Health System, emphasized that no matter what, care providers need to be especially cautious when implementing agentic AI into diagnostic workflows. 

There is a weight that comes with being a physician who makes decisions for patients, so care needs to be taken when working with AI, Feldman said.

"I envision physicians partnering with AI, but we want to keep some weight," Feldman told the audience. "Just because you have another, maybe, mind or another intelligence helping you with the tasks that you're working on, we should not take off the weight." 

Kowalczyk agreed with Feldman, conveying that physicians took a Hippocratic oath, and the whole way clinicians view the world is through risk and mitigating very low probability events. 

"We bear that risk for health systems and have that distributed liability risk, and I think that's a really key point – if you now centralize that decision-making process, what company or department is going to take that [liability] on themselves? I think those are really interesting questions that we need to wade through," Kowalczyk said. 

McMillin emphasized that health systems using agentic AI need to consider that they are giving those agents the authority to make decisions on their behalf and take action on those decisions.  

"There are things you should be doing to make sure that you are, as much as possible, understanding how they make the decision as best you can," McMillin said. 

The panelists outlined how partnering with the vendors offering agentic AI is key to understanding the complete complexity of the offering and how it can fit into the clinical workflow effectively. 

"This technology is moving so fast. Everyone is figuring this out. No one has this figured out, and so I think that [partnership] is the key underlying current theme," Kowalczyk said.