How Researchers are Using Artificial Intelligence to Help Patients Understand Their Health
Digital avatar uses AI technologies to interact with patients
Have you ever talked to Siri or Alexa, only to find the virtual assistant didn’t interpret what you said exactly right? It’s a frustration for many people, especially non-native English speakers.
Now imagine a low-income Latina breast cancer survivor who did not finish high school. She’s afraid to ask her doctor all the questions she has about her health. That’s where Francisco Iacobelli’s intelligent tutoring system comes in.
Iacobelli is a visiting researcher at the Northwestern Center for Advancing Safety of Machine Intelligence (CASMI) and an associate professor of computer science at Northeastern Illinois University. His research team is designing a digital avatar that uses artificial intelligence (AI) technologies to interact with low-literacy minority individuals and to help them understand their medical conditions.
On May 10, Iacobelli gave a presentation entitled, “An Intelligent Tutoring System for the Minority: An Intentional Choice for Equitable AI.” The talk focused on his CASMI-funded research project and emphasized a tailored approach to address the needs of a particular community.

U.S. government data shows 30 million Americans have below basic-level literacy skills. Of that group, roughly 12 million are Hispanics and another 6 million are Black. For reference, someone with basic skills can read cooking instructions or street signs.
Iacobelli demonstrated how an avatar named Dr. Maria Gonzalez can answer questions with minimal interruptions to the human, a key objective. The language model is being trained to recognize the pauses in natural conversations.
“This woman and this population that I was working with, they pause to finish their ideas and to formulate their ideas,” Iacobelli said. “Our speech recognition system needs to be able to understand that some pauses are yielding the floor, and some pauses are not.”
An Inclusive Approach to Research
Supporting research projects like these is critical to CASMI’s mission of building intelligent systems that are safe and effective, said Kristian Hammond, Bill and Cathy Osborn professor of computer science and director of CASMI.

The tutoring system is specifically designed for low-literacy Latina breast cancer survivors, a group Iacobelli is interviewing. His diverse research team includes Hispanic students and non-traditional students, such as single parents and older adults. He believes a team like that is fundamental to bridge the gap between researchers and these marginalized communities to identify unique challenges in them. Iacobelli is also consulting with community leaders and physicians for this project.
“They're going to give me a lot of good criticism,” Iacobelli said.
One community leader told Iacobelli that the avatar shouldn’t have a robotic voice because people would not engage with that. Human voices are now being recorded for the avatar to use.
Iacobelli’s research team includes high schoolers, undergraduate students, graduate students, and post-doctoral scholars. Northwestern third-year computer science major Kevin Rueda joined the research project in January. He built a configuration tool that helps people communicate with the intelligent tutoring system.

Northwestern third-year computer science major Raquel Saldivar began working as a research assistant in October 2022. She was involved in implementing an interface that controls the avatar. She said she is passionate about helping minority populations.

Iacobelli stressed the importance of being involved in the communities where you work. He is a member of the Medical Organization for Latino Advancement and the Chicago Cancer Health Equity Collaborative.
“To break structures of power and to build trust, you have to be involved in the community,” Iacobelli said.
How the Tutoring System Works
The tutoring system is designed so that people with proficient literacy skills (like doctors, community leaders, and researchers) can control the avatar. Computer scientists or web programmers can make other modifications so that the tutoring system can better interpret what the human is saying and use vocabulary that is easier for people to understand.
To train the model, Iacobelli accounted for instances in which Latina breast cancer survivors used the wrong word. For example, all the women Iacobelli worked with incorrectly believed that after breast surgery, they could develop lymphoma – cancer of the lymph nodes – when in fact the increased risk was for lymphedema, swelling caused by a blockage in the lymphatic system.
“I don't want the system not to give them the information because it's so picky about it being called the right word,” Iacobelli said. “It should understand the alternative.”
Breast cancer surgeons who have seen the digital avatar love it, Iacobelli said. But for him, it’s equally important for everyone who is impacted by these systems to feel the same way.