AI helpful in measuring benefits of therapy

Behavioral healthcare involves many different treatment components, some of which seem hard to quantify in the same way as physical clinical care. One-to-one or group therapy sessions involve an intricate combination of spoken and unspoken communication, producing masses of qualitative data filled with deep semantic complexity.  

While researchers and developers are getting very good at using AI and machine learning to comb through clinical data to identify patterns and analyze the effectiveness of care decisions, the unstructured world of behavioral healthcare data is still largely unexplored.   

That means that despite having access to a large number of standardized interventions, behavioral healthcare clinicians may have fewer tools at their disposal to understand the impact of their work with patients and further develop evidence-based medicine to improve outcomes. 

Changing this equation requires AI algorithms that can accurately capture and analyze the nuances in conversations between patients and behavioral healthcare providers to better understand how evidence-based interventions are being applied in practice, says Shiri Sadeh-Sharvit, PhD, Chief Clinical Officer at Eleos Health.   

“Even the best-trained clinicians don’t always follow all the guidelines recommendations to the letter, because behavioral healthcare can be unpredictable and requires flexibility and intuition,” she explained to Digital Health Insights. “Patients may have new issues crop up during the week that need to be addressed, or they might need to repeat concepts to reinforce their skill set.”   

“Combined with the stresses of being a behavioral healthcare provider in a time of massive shortages and burnout, it’s easy to see why there might be gaps between theory and the real-world practice. Our job is to understand when and why those differences occur so we can support clinicians with the tools they need to get into closer alignment.” 

To start the process, Sadeh-Sharvit and researchers from Palo Alto University and Stanford Medical Center conducted a study using AI to identify the use of homework assignments in behavioral health sessions.   

Homework, or action plans, are often used in cognitive behavioral therapy and other types of therapy to provide structure for patients and extend the impact of the session into their daily lives. 

“There are clear associations between homework assignments completed and desirable outcomes for patients, but much of the literature is based on self-reports, which can be less than ideally reliable,” Sadeh-Sharvit said. 

“We wanted to expand our understanding of the extent to which homework is used, the type of homework assigned, and the frequency of follow-up on that assignment during the next appointment. The challenge is that audio data is high-volume and unstructured, making it difficult to extract meaning from it.” 

Behavioral healthcare data is also highly sensitive and subject to specific privacy laws that can vary by state. Working with this type of information requires patient consent and reliable deidentification, which the team accomplished by stripping out potential identifiers of individuals or detailed contextual clues and only using these blinded snippets of data in their analysis. 

To train the algorithm involved in the study, the Eleos Health team extracted 2.83 million of these “micro dialogues” from close to 35,000 individual behavioral health sessions. For this use case, experts then narrowed down the dataset to 4000 sessions in which homework was indicated. 

An analysis of 100 random sessions found that homework was assigned in 61 sessions. In 21 of the cases, more than one homework assignment was provided. The assignments addressed key therapeutic topics such as practicing skills learned during sessions, taking positive actions, journaling, and learning specific new skills. 

When human therapists reviewed the results of the AI model, they agreed in 90% of the cases on whether or not homework had been assigned during the session. 

“This is just a first step in using AI to support clinicians and track the progress of individuals undergoing behavioral healthcare,” said Sadeh-Sharvit. “In the future, we would like to use this type of information to provide reminders to clinicians that they assigned homework during the last session so they can follow up, or even prompt them to give a new homework assignment or other evidence-based intervention relevant to the patient’s needs.”  

“AI can never really understand the full extent and meaning of the patient-provider relationship in the behavioral health context, or why a provider ended up deviating from a given process.  But it can help clinicians be more aware and planful so that they are making those decisions in an intentional way to reduce stress on themselves while helping patients achieve their goals.”  

Jennifer Bresnick is a journalist and freelance content creator with a decade of experience in the health IT industry.  Her work has focused on leveraging innovative technology tools to create value, improve health equity, and achieve the promises of the learning health system.  She can be reached at jennifer@inklesscreative.com.