An early warning system that predicts which patients are at risk of deteriorating during their hospital stay has been linked to fewer unexpected deaths, a new study finds.
The study, published Monday in the Canadian Medical Association Journalshows a 26% reduction in non-palliative deaths among patients in the general internal medicine unit at St. Michael’s Hospital when the artificial intelligence tool was used.
“We found that there is a lot of hype and excitement around artificial intelligence in medicine. We also found that the deployment of these tools in real clinical settings was not as great,” said lead author Dr. Amol Verma, a general internal medicine specialist and scientist at the Toronto hospital.
“This is an early example of deploying a tool that has been rigorously tested and evaluated and shows promise for improving patient care,” Verma, who is also a professor of research and teaching of AI in medicine at the University of Toronto, said in an interview.
The technology, called CHARTwatch, continuously analyzed more than 100 different pieces of information about each patient in the unit, Verma said.
When the AI tool predicts that a patient is deteriorating, it sends an alert to doctors and nurses, prompting them to intervene quickly.
The machine learning tool brings together information that is already routinely collected in a patient’s electronic medical record.
This includes information such as age and medical history, as well as measurements such as vital signs, blood pressure, heart rate and lab test results.
“It puts all of this information together to make a prediction about the patient’s risk of becoming more seriously ill in the future, and then it updates its prediction model every hour based on how all of these things change over time,” Verma explained.
If, after examining the patient, the clinician agreed with the AI’s prediction, necessary action was taken. This could include transferring the patient to an intensive care unit, giving them antibiotics for serious infections such as sepsis, or monitoring them more frequently.
If a patient’s death was inevitable, he or she would receive end-of-life care sooner than he or she otherwise would, which would ease his or her suffering, Verma said.
It’s important to note that AI isn’t telling the clinician, ‘Prescribe this drug, intervene with this test or this treatment.’ That’s up to the nurses and doctors who are providing care, he added. It’s a signal that says, ‘Hey, pay attention to this patient.’
These early warning signals are important in busy hospitals where each nurse or doctor cares for many patients who undergo multiple lab tests, imaging scans and other procedures that can change their prognosis, Verma said.
“It’s just not possible for humans to keep an eye on 20 or 30 patients at the same time, all the time,” he said.
Muhammad Mamdani, co-senior author of the study, added that AI can process large amounts of patient data and combining that data with a human clinician’s judgment can lead to better care.
Doctors and nurses should still err on the side of caution when using the tool, said Mamdani, who is vice-president of data science and advanced analytics at Unity Health Toronto, which includes St. Michael’s Hospital.
“What we tell our clinicians is if you think this patient is going to die, but the AI says, ‘No, he’s fine,’ don’t believe the AI. Trust your gut,” he said. “But if the AI says, ‘This patient is going to die,’ and you don’t think so, don’t trust your gut, trust the AI.”
The study focused on deaths of non-palliative patients in the general internal medicine unit between 1er November 2020 and the 1er June 2022 when the AI tool was used and compared them to an earlier period – from 1er November 2016 to 1er June 2020 – when the technology had not been used.
The researchers found a non-palliative death rate of 2.1% when AI was not used, compared to 1.6% when it was.
To reduce the possibility that the results were attributed to the different time periods, the researchers used the hospital’s cardiology, pulmonology, and nephrology units—which did not have the AI tool—for comparison. None of these units showed a difference in nonpalliative deaths between the two time periods.
The researchers controlled for potential confounders such as age. Additionally, because the study period coincided with the COVID-19 pandemic, which could be another variable influencing the results, the researchers excluded data from COVID patients.
In total, the study included 13,649 patient admissions to the general internal medicine unit and 8,470 patient admissions to the comparison cardiology, pulmonology, and nephrology units.
Dr. Verma said that while the results are promising, they should be interpreted with caution and that a randomized controlled trial is needed for AI research to be considered as robust as a drug study.
Ross Mitchell, a professor in the University of Alberta’s faculty of medicine who was not involved in the study, called the research “very encouraging.”
“This specific technology, CHARTwatch, needs to be looked at in a broader sense,” said Mitchell, who is Alberta Health Services’ chair in AI in health. “It needs to be deployed in more hospitals across Canada, so that more than one hospital is involved.”