Exploring the Reliability of Machine Learning Models in Healthcare

Karina Zadorozhny’s experience doing research started at age 14 when she worked full-time in the laboratory of Lumir Krejci, associate professor of biology at Masaryk University in Brno. There, she studied mechanisms of DNA damage repair. During her three years collaborating with Krejci, Zadorozhny contributed to published studies in journals such as Molecular Cell and Nature.

“I was very interested in molecular biology, but because there are no universities in my hometown, I had to move to a different city in the Czech Republic,” said Zadorozhny, a senior-year student earning a combined bachelor of arts degree in mathematics and master of science degree in computer science. “The fact the university was quite far away ended up being a benefit because I could fully focus on doing research. It was a very rewarding experience.”

Since then, Zadorozhny’s passion for research has both increased and evolved. She recently presented work at the intersection of machine learning and healthcare during the 6th International Workshop on Health Intelligence, part of the Thirty-Sixth Association for the Advancement of Artificial Intelligence Conference on Artificial Intelligence in Vancouver, British Columbia, on February 28, and March 1, 2022.

Applying AI in medicine

Zadorozhny’s presentation, titled “Out-of-Distribution Detection for Medical Applications: Guidelines for Practical Evaluation,” is based on research she conducted during an internship at Pacmed, a startup company in Amsterdam building clinical decision-making support systems based on machine learning for hospitals, general practitioners, and medical education centers.

Zadorozhny described approaches to testing the reliability of machine learning models by obtaining a separate uncertainty score for training data versus only calculating one prediction per sample.

“Due to data shifts after a model is deployed, the assumption that machine learning models are going to see input data similar to the training data can be easily violated,” Zadorozhny said. “This leaves no guarantees on model performance for new inputs.”

Using medical tabular data, the team investigated methods for uncertainty quantification in patient data and techniques for flagging out-of-distribution samples, or those not well represented in the model training dataset. 



“We described a general framework to help with the deployment of safe and reliable AI in a medical environment,” Zadorozhny said. “More specifically, we want machine learning models to detect and admit that they simply don’t know the right answer instead of giving an incorrect guess.”

The research has significant implications for the application of AI and machine learning in healthcare settings.

“When doctors use machine learning-based tools, such as those assessing whether a patient is ready to be discharged from a hospital, they don’t have a lot of guarantees that these tools are reliable,” Zadorozhny said. “We hope that our work will help establish more safety checks on machine learning models. That is, if a doctor uses a predictive tool on data of a patient with characteristics that are very dissimilar from other patients, the tool should itself alert the doctor and indicate that this patient is out of the tool's intended scope.”

Coauthored by Giovanni Cinà, a research lead at Pacmed, and Patrick Thoral and Paul Elbers, medical specialists at Amsterdam University Medical Centers, the paper will be published in Springer’s Studies in Computational Intelligence book series.

Shifting focus

Zadorozhny’s research journey has included many stops.

She was a member of a research team at Northwestern’s Institute for Augmented Intelligence in Medicine led by Lee Cooper, associate professor of pathology and preventive medicine at Northwestern University Feinberg School of Medicine and (by courtesy) professor of electrical and computer engineering at Northwestern Engineering.

She also completed internships in various areas of molecular biology at the University of Cologne and at the Buck Institute for Research on Aging as a SENS Research Foundation Summer Scholar, where she researched diabetic kidney disease. 

When Zadorozhny joined Northwestern, however, she pivoted to computer science. 

“I want to understand how machine learning algorithms and computational analysis can be applied to biology,” Zadorozhny said. “Coming in with no computational background, I couldn’t appreciate more how supportive and welcoming the CS department has been. The CS faculty are great and many professors, especially Professor Khuller, really invest their time in the students and support them as much as possible.”