Abstract: SA-PO030
Validation of AKI Prediction Model as Clinical Decision Supporting System
Session Information
- AKI: Clinical, Outcomes, and Trials - Management
October 26, 2024 | Location: Exhibit Hall, Convention Center
Abstract Time: 10:00 AM - 12:00 PM
Category: Acute Kidney Injury
- 102 AKI: Clinical, Outcomes, and Trials
Authors
- You, Eun Mi, Seoul National University Bundang Hospital, Seongnam, Korea (the Republic of)
- Yi, Jinyeong, Seoul National University Graduate School of Convergence Science and Technology, Gwanak-gu, Seoul, Korea (the Republic of)
- Han, Sangyub, Seoul National University College of Medicine, Jongno-gu, Seoul, Korea (the Republic of)
- Kim, Sejoong, Seoul National University Bundang Hospital, Seongnam, Korea (the Republic of)
- Yun, Giae, Seoul National University College of Medicine Department of Internal Medicine, Seoul, Korea (the Republic of)
Background
Acute kidney injury (AKI) is a critical clinical syndrome requiring immediate intervention. Our previous research developed a ‘PRIME solution’ AI model for AKI. This study aims to evaluate the usefulness of this AI model in improving the predictive capabilities of AKI prediction.
Methods
We utilized convolutional neural networks with a residual block to predict AKI in hospitalized patients. The training set comprised data from 183,221 patients at Seoul National University Hospital (2013-2017). At Seoul National University Bundang Hospital (2020-2021), we randomly selected 74 patients from departments with high AKI rates, including 15% AKI cases. We assessed the impact of an AI model on clinical decisions by comparing evaluations with and without AI assistance.
Results
Accuracy was highest for physicians, with students and the AI model were similar (physician: 0.797, student: 0.574, AI: 0.568). AI assistance improved recall and F1 scores for all almost individuals (recall: 52.4% to 71.4%, F1: 37.7% to 46.1%). In the AKI predicted group, recall increased while F1 decreased for physicians (recall: 36.4% to 60%, F1: 43.2% to 33.3%) and students (recall: 54.5% to 80%, F1: 44.4% to 36.9%). For the non-AKI predicted group, both saw significant gains in recall and F1 with AI (physicians: recall 16.7% to 87.5%, F1: 18.2% to 66.7%; students: recall 44.4% to 75%, F1: 21.1% to 40%). Review times decreased for all with AI (median: 69.0 to 52.0 seconds, p=0.032), especially in the non-AKI predicted group (68.5 to 46.0 seconds, p<0.001; AKI predicted group 71.0 to 57 seconds, p<0.001).
Conclusion
AI notably improved physician performance, especially in the non-AKI predicted group with significant time savings and higher F1 scores. However, its impact was less marked in the AKI-predicted group. Alongside enhancing AI, studies on its application and target groups are essential.