If clinical AI isn’t an “everything machine,” what should it be instead?

Every Thursday, Roy Ziegelstein, MD, MACP, still has to take out his own trash.

He’s thought about handing the job to ChatGPT, but the LLM can’t seem to get a grip on the rolling bins.

“One day, there will be AI robots designed to do all that and more,” he said, “and I will gladly give up that particular task to them. But right now, it would be ridiculous to expect ChatGPT to handle that for me. We all know it’s not designed for it.

“So why are we doing the equivalent with AI in the clinical world? We want our AI to do ambient listening, chart summarization or clinical decision support … but we don’t always fully understand and accept the limitations of what these tools are intended to do or what they can actually provide.”

The task for technologists, health system leaders, policymakers and educators is to collaboratively identify and define the boundaries of what roles AI can and should play in the clinical decision-making process and which tasks are better left to humans with the hands, heads and hearts to deliver meaningful, person-centered care.

New tools don’t replace good clinicians. They make better ones.

As a practicing cardiologist for more than 30 years, a professor of medicine at Johns Hopkins University School of Medicine and the editor-in-chief and chief medical officer of DynaMed, Ziegelstein has seen his fair share of innovative digital tools enter the clinical environment.

Most of them have come with their own hype cycles, and their own fears that adding a new layer of technology between the provider and the patient will result in less human care delivered by less intrinsically skilled clinicians.

“When point-of-care ultrasound came on the scene, there was great concern that people would stop being able to use their stethoscope,” he recalled. “In fact, I think point-of-care ultrasound can actually make me a better diagnostician by obtaining information that I can’t obtain with my stethoscope, and potentially vice versa. It makes me a better history taker, a better interviewer, a better diagnostician in general.

“It provides information right at the point of care, literally in seconds, that I cannot obtain by talking to and examining my patient alone. But it would be ludicrous to think, for example, that it could pass a multiple-choice quiz that would test medical knowledge. So while it outperforms even an expert cardiologist at certain tasks, it’s woefully inadequate for things it was not designed to do. AI, and in particular AI clinical decision support, can be thought of similarly.”

Matching the tool to the clinical question

The key for clinicians is knowing when and where to apply new technologies, including AI clinical decision support, and when to use other forms of clinical investigation and judgment, Ziegelstein explained.

“I use point-of-care ultrasound all the time,” he said. “I use AI clinical decision support all the time. But I don’t use point-of-care ultrasound every time I see a patient. And I don’t use AI clinical decision support every time I see a patient. It depends on the clinical question that I’m trying to answer.”

AI should be viewed as one way to facilitate fact-finding and provide support for making choices about patient care, not a substitute for established, evidence-based practices that include authentic connections with patients.

“CT scanning, for example, is not a substitute for examining the abdomen of a patient. You still have to interact with them physically. You still have to take a history,” he said. “Those aren’t optional parts of delivering care.

“At the same time, if there’s a clinician out there who thinks that a physical exam and a history are all they need, and that diagnostic imaging isn’t helpful at all, that’s a big problem. If your clinician is like that, you should run as fast as you can.”

AI tools are no different, when they’re integrated into the care process according to the role they’re specifically designed to support.

“Medical educators need to emphasize the importance of getting to know patients as people, how to take a history from a patient, and how to do a careful physical examination,” he stressed. “We need to teach people how to use books and journals and other traditional resources, but also how to bring AI into that equation so it can do the things it’s really very good at doing.”

Leveraging AI while preserving the human elements of care

The hidden danger of treating AI like an “everything machine” is that the definition of “everything” can creep away from purely scientific clinical decision support and into the interpersonal realm of the patient-provider relationship, Ziegelstein cautioned.

“I have no doubt that AI can be better than even an expert clinician at certain tasks, but not at what I would call the gray areas of medicine, which actually dominate the picture,” he said. “Most of the issues that clinicians deal with are not black and white. They’re gray. We’re increasingly asking AI to handle some of these things because of time and resource pressures. For example, some folks are exploring how to bring AI into empathetic, compassionate conversations with patients, because there isn’t always enough bandwidth for clinicians to have those conversations right now.

“I think AI is inadequate in those areas. That’s a human job. Any proposal about the value and appropriateness of AI in the clinical environment needs to think carefully around what it’s designed to do, and what it’s not designed to do, when those situations arise.”

However, those discussions need to be careful not to position AI as the enemy of human-driven, patient-centered care. Just like clinical decision-making itself, the issue isn’t quite so clear-cut.

“It’s a failure on the part of educators to be all-or-nothing about the value of new technologies,” Ziegelstein said, counting himself among them after many years as an educator. “We’ve communicated this view to learners that AI is the enemy because it can alter the way we generate, analyze and apply information. But by that view, so is the point-of-care ultrasound and the CT scanner, and we know that’s not true.

“What I would love to see is AI do what AI is intended to do: free up time for clinicians to be active listeners with their patients, really get to uncover and understand things that AI can’t do, and then use the information that AI is phenomenally good at obtaining to make the best possible decisions about patient care.”

Jennifer Bresnick is a journalist and freelance content creator with a decade of experience in the health IT industry.  Her work has focused on leveraging innovative technology tools to create value, improve health equity, and achieve the promises of the learning health system.  She can be reached at [email protected].