Quality Improvement Surveillance by Machine Learning
J. D. Perkins1, E. Clemens2, M. Granich3, J. Yeung3, J. Reyes1
1Surgery, University of Washington, Seattle, WA, 2Pharmacy, University of Washington, Seattle, WA, 3Center of Clinical Excellence, University of Washington, Seattle, WA
Meeting: 2021 American Transplant Congress
Abstract number: 1297
Keywords: Outcome, Post-operative complications, Rejection, Screening
Topic: Clinical Science » Organ Inclusive » Machine Learning, Artificial Intelligence and Social Media in Transplantation
Session Information
Session Name: Machine Learning, Artificial Intelligence and Social Media in Transplantation
Session Type: Poster Abstract
Session Date & Time: None. Available on demand.
Location: Virtual
*Purpose: The purpose of this quality study was to compare our newly developed natural language processing (NLP) machine learning surveillance algorithm in detecting the rate of rejection episodes in our liver transplant recipients to our previous electronic medical record (EMR) structured data collection system.
*Methods: In 292 consecutive liver transplant recipients from 2/9/2017 to 2/4/2020 with a follow-up to 6/1/2020, 27 (10.3%) recipient were confirmed to have a rejection episode (biopsy and treatment) within 3 months of transplantation by two transplant professionals. A phrase-based NLP surveillance algorithm evaluated the transplant notes of the healthcare providers for the first three months following transplantation to label recipients with a rejection episode. The recipients labeled with a rejection episode in our EMR as entered by transplant providers during the care of these recipients were recorded.
*Results: The EMR recorded events labeled 19 (70.4%) rejection episodes correctly, missed 8(29.6%) rejection episodes, and mislabeled 12 rejection episodes that were not rejection episodes. The NLP surveillance algorithm labeled 21(77.8%) rejection episodes correctly, missed 6 (22.2%), and mislabeled no rejection episode that were not rejection episodes. The kappa coefficient agreement statistics to the confirmed list for the NLP algorithm of 0.86 (CI 0.75 to 0.97) was much better than the EMR recorded events of 0.61 (CI 0.46 to 0.77).
*Conclusions: Transplant programs follow many quality measures to improve their care of their patients. Following these measures have required cost and time of chart abstractors or busy clinicians directly entering these events into structure fields in the EMR. We have shown that the evolving machine learning method of phrase-based NLP is better labeling transplant recipients as having rejection events compared to recorded events in our EMR. The improved accuracy of NLP for surveillance of quality measures makes this a valuable technique for quality improvement surveillance.
To cite this abstract in AMA style:
Perkins JD, Clemens E, Granich M, Yeung J, Reyes J. Quality Improvement Surveillance by Machine Learning [abstract]. Am J Transplant. 2021; 21 (suppl 3). https://atcmeetingabstracts.com/abstract/quality-improvement-surveillance-by-machine-learning/. Accessed May 12, 2025.« Back to 2021 American Transplant Congress