Interpretable Machine Learning Model Based on SOFA Score for ICU Sepsis Mortality Prediction with Multicenter Validation
Keywords:
Sepsis patients, ICU, machine learning, mortality prediction, multicenter validation, interpretable models.Abstract
The Sequential Organ Failure Assessment (SOFA) score is a widely employed scoring system in clinical practice for predicting mortality in patients with sepsis. The integration of machine learning techniques into clinical scoring systems has enhanced predictive performance; however, many of these models function as "black boxes," offering limited interpretability regarding the contribution of individual clinical variables to the final prediction.
This study aims to develop an interpretable machine learning model based on the SOFA score, leveraging its most relevant variables, to predict mortality in Intensive Care Unit (ICU) patients with sepsis using a multicenter validation.
The model was trained on data from 15,100 ICU patients in the MIMIC-IV v3.0 dataset and externally validated on 8,201 patients from the eICU v2.0 dataset. The application of an Odds Ratio analysis enabled the identification of the SOFA components demonstrating the strongest association with mortality. This approach facilitated the reduction of variables while enhancing the performance of the model.
The interpretability of the model was further addressed by employing SHapley Additive exPlanations (SHAP) values to elucidate the contribution of each variable to the model's predictions. The resulting model demonstrated superior predictive accuracy in comparison to the conventional SOFA score, while exhibiting enhanced efficiency and transparency.
This interpretable machine learning model, which is based on a SOFA variant, has the potential to support the earlier and more precise intervention required in the clinical management of sepsis ICU patients.
Downloads
References
bibitem{Singer2016}
M. Singer textit{et al.}, ``The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3),'' textit{JAMA}, vol. 315, no. 8, pp. 801--810, Feb. 2016. DOI: href{https://doi.org/10.1001/jama.2016.0287}{10.1001/jama.2016.0287}
bibitem{Moreno2023}
R. Moreno textit{et al.}, ``The Sequential Organ Failure Assessment (SOFA) Score: has the time come for an update?,'' textit{Critical Care}, vol. 27, no. 1, p. 15, Jan. 2023. DOI: href{https://doi.org/10.1186/s13054-022-04290-9}{10.1186/s13054-022-04290-9}
bibitem{Khan2022}
A. M. Khan and S. M. Aslam, ``Comparison of qSOFA Score, SIRS Criteria, and SOFA Score as predictors of mortality in patients with sepsis,'' textit{Ghana Medical Journal}, vol. 56, no. 3, pp. 191--197, Sep. 2022. DOI: href{https://doi.org/10.4314/gmj.v56i3.9}{10.4314/gmj.v56i3.9}
bibitem{Raith2017}
E. P. Raith textit{et al.}, ``Prognostic Accuracy of the SOFA Score, SIRS Criteria, and qSOFA Score for In-Hospital Mortality Among Adults With Suspected Infection Admitted to the Intensive Care Unit,'' textit{JAMA}, vol. 317, no. 3, pp. 290--300, Jan. 2017. DOI: href{https://doi.org/10.1001/jama.2016.20328}{10.1001/jama.2016.20328}
bibitem{Rudd2020}
K. E. Rudd textit{et al.}, ``Global, regional, and national sepsis incidence and mortality, 1990–2017: analysis for the Global Burden of Disease Study,'' textit{Lancet}, vol. 395, no. 10219, pp. 200--211, Jan. 2020. DOI: href{https://doi.org/10.1016/S0140-6736(19)32989-7}{10.1016/S0140-6736(19)32989-7}
bibitem{Pan2023}
X. Pan, J. Xie, L. Zhang textit{et al.}, ``Evaluate prognostic accuracy of SOFA component score for mortality among adults with sepsis by machine learning method,'' textit{BMC Infect Dis}, vol. 23, p. 76, 2023. DOI: href{https://doi.org/10.1186/s12879-023-08045-x}{10.1186/s12879-023-08045-x}
bibitem{Abdullah2021}
T. A. A. Abdullah, M. S. M. Zahid, and W. Ali, ``A Review of Interpretable ML in Healthcare: Taxonomy, Applications, Challenges, and Future Directions,'' textit{Symmetry}, vol. 13, no. 12, article 2439, 2021. DOI: href{https://doi.org/10.3390/sym13122439}{10.3390/sym13122439}
bibitem{Linardatos2020}
P. Linardatos, V. Papastefanopoulos, and S. Kotsiantis, ``Explainable AI: A Review of Machine Learning Interpretability Methods,'' textit{Entropy}, vol. 23, no. 1, p. 18, Dec. 2020. DOI: href{https://doi.org/10.3390/e23010018}{10.3390/e23010018}
bibitem{stsiva}
C. Santos, M. A. Bravo, and C. A. Fajardo,
``SOFA+: Applying Machine Learning to Enhance the Prognostic Power of the SOFA Score for Predicting ICU Sepsis Mortality,''
in textit{Proc. 2025 XXV Symp. Image, Signal Processing, and Artificial Vision (STSIVA)}, Bucaramanga, Colombia, 2025, pp. 1--5. DOI: href{https://doi.org/10.1109/STSIVA66383.2025.11156628}{10.1109/STSIVA66383.2025.11156628.}
bibitem{Johnson2016}
A. E. W. Johnson textit{et al.}, ``Machine Learning and Decision Support in Critical Care,'' textit{Proc. IEEE}, vol. 104, no. 2, pp. 444--466, 2016. DOI: href{https://doi.org/10.1109/JPROC.2015.2501978}{10.1109/JPROC.2015.2501978}
bibitem{Vincent1996}
J.-L. Vincent textit{et al.}, ``The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure. On behalf of the Working Group on Sepsis-Related Problems of the European Society of Intensive Care Medicine,'' textit{Intensive Care Medicine}, vol. 22, no. 7, pp. 707--710, 1996. DOI: href{https://doi.org/10.1007/BF01709751}{10.1007/BF01709751}
bibitem{Yang2025}
J. Yang, L. Li, and S. Fu, ``Association analysis of sepsis progression to sepsis-induced coagulopathy: a study based on the MIMIC-IV database,'' textit{BMC Infectious Diseases}, vol. 25, no. 1, p. 573, 2025. DOI: href{https://doi.org/10.1186/s12879-025-10972-w}{10.1186/s12879-025-10972-w}
bibitem{Ki2025}
M. S. Ki textit{et al.}, ``Association Between Plasma Granzyme B Levels, Organ Failure, and 28-Day Mortality Prediction in Patients with Sepsis,'' textit{Journal of Clinical Medicine}, vol. 14, no. 5, p. 1461, 2025. DOI: href{https://doi.org/10.3390/jcm14051461}{10.3390/jcm14051461}
bibitem{Sacha2022}
G. L. Sacha textit{et al.}, ``Association of Catecholamine Dose, Lactate, and Shock Duration at Vasopressin Initiation With Mortality in Patients With Septic Shock,'' textit{Critical Care Medicine}, vol. 50, no. 4, pp. 614--623, 2022. DOI: href{https://doi.org/10.1097/CCM.0000000000005317}{10.1097/CCM.0000000000005317}
bibitem{Saragih2021}
I. D. Saragih, I. S. Saragih, S. O. Batubara, and C.-J. Lin, ``Dementia as a mortality predictor among older adults with COVID-19: A systematic review and meta-analysis of observational study,'' textit{Geriatric Nursing}, vol. 42, no. 5, pp. 1230--1239, 2021. DOI: href{https://doi.org/10.1016/j.gerinurse.2021.03.007}{10.1016/j.gerinurse.2021.03.007}
bibitem{Estiri2021}
H. Estiri, Z. H. Strasser, J. G. Klann, P. Naseri, K. B. Wagholikar, and S. N. Murphy,
"Predicting COVID-19 mortality with electronic medical records,"
textit{NPJ Digital Medicine}, vol. 4, no. 1, p. 15, 2021, DOI: href{https://doi.org/10.1038/s41746-021-00383-x}{10.1038/s41746-021-00383-x.}
bibitem{Charoenngam2021}
N. Charoenngam, A. Shirvani, N. Reddy, D. M. Vodopivec, C. M. Apovian, and M. F. Holick,
"Association of vitamin D status with hospital morbidity and mortality in adult hospitalized patients with COVID-19,"
textit{Endocrine Practice}, vol. 27, no. 4, pp. 271--278, 2021. DOI: href{https://doi.org/10.1016/j.eprac.2021.02.013}{10.1016/j.eprac.2021.02.013}
bibitem{Ball2016}
I. M. Ball, S. M. Bagshaw, K. E. A. Burns, D. J. Cook, A. G. Day, P. M. Dodek, D. J. Kutsogiannis, S. Mehta, J. G. Muscedere, H. T. Stelfox, A. F. Turgeon, G. A. Wells, and I. G. Stiell,
"A clinical prediction tool for hospital mortality in critically ill elderly patients,"
textit{Journal of Critical Care}, vol. 35, pp. 206--212, 2016. DOI: href{https://doi.org/10.1016/j.jcrc.2016.05.026}{10.1016/j.jcrc.2016.05.026.}
bibitem{Luo2022}
X. Q. Luo, P. Yan, S. B. Duan, Y. X. Kang, Y. H. Deng, Q. Liu, T. Wu, and X. Wu,
"Development and validation of machine learning models for real-time mortality prediction in critically ill patients with sepsis-associated acute kidney injury,"
textit{Frontiers in Medicine}, vol. 9, 2022.
DOI: href{https://doi.org/10.3389/fmed.2022.853102}{10.3389/fmed.2022.853102.}
bibitem{Shi2025}
S. Shi, L. Zhang, S. Zhang, J. Shi, D. Hong, S. Wu, X. Pan, and W. Lin,
"Developing a rapid screening tool for high-risk ICU patients of sepsis: integrating electronic medical records with machine learning methods for mortality prediction in hospitalized patients—model establishment, internal and external validation, and visualization,"
textit{Journal of Translational Medicine}, vol. 23, no. 1, 2025.
DOI: href{https://doi.org/10.1186/s12967-025-06102-4}{10.1186/s12967-025-06102-4.}
bibitem{Zhang2024}
G. Zhang, F. Shao, W. Yuan, J. Wu, X. Qi, J. Gao, R. Shao, Z. Tang, and T. Wang,
``Predicting sepsis in-hospital mortality with machine learning: a multi-center study using clinical and inflammatory biomarkers,''
emph{European Journal of Medical Research}, vol. 29, no. 1, 2024, DOI: href{https://doi.org/10.1186/s40001-024-01756-0}{10.1186/s40001-024-01756-0.}
bibitem{Castillo2025}
J. A. Castillo, C. Santos, and C. A. Fajardo,
``Multicenter validation of a machine learning algorithm for mortality prediction at decision-critical timestamps in ICU sepsis patients,''
in emph{Proceedings of the 2025 8th International Conference on Robot Systems and Applications (ICRSA 2025)}, Wuhan, China, Sept. 19--21, 2025, pp. 1--5. Article accepted.
bibitem{Rahman2024}
M. S. Rahman, K. R. Islam, J. Prithula, J. Kumar, M. Mahmud, M. F. Alam, M. B. I. Reaz, A. Alqahtani, and M. E. H. Chowdhury,
``Machine learning-based prognostic model for 30-day mortality prediction in Sepsis-3,''
emph{BMC Medical Informatics and Decision Making}, vol. 24, no. 1, 2024, DOI: href{https://doi.org/10.1186/s12911-024-02655-4}{10.1186/s12911-024-02655-4.}
bibitem{Shumilov2025}
A. Shumilov, Y. Zhu, N. Ashrafi, A. Abdollahi, G. Placencia, K. Alaei, and M. Pishgar,
``Data-Driven Machine Learning Approaches for Predicting In-Hospital Sepsis Mortality,''
emph{Lecture Notes in Networks and Systems}, vol. 1424 LNNS, pp. 393--409, 2025, DOI:href{https://doi.org/10.1007/978-3-031-92605-1_24}{10.1007/978-3-031-92605-1_24.}
bibitem{Hu2022}
C. Hu, L. Li, W. Huang, T. Wu, Q. Xu, J. Liu, and B. Hu,
``Interpretable Machine Learning for Early Prediction of Prognosis in Sepsis: A Discovery and Validation Study,''
emph{Infectious Diseases and Therapy}, vol. 11, no. 3, pp. 1117--1132, 2022, DOI: href{https://doi.org/10.1007/s40121-022-00628-6}{10.1007/s40121-022-00628-6.}
bibitem{Li2023}
S. Li, R. Dou, X. Song, K. Y. Lui, J. Xu, Z. Guo, X. Hu, X. Guan, and C. Cai,
``Developing an interpretable machine learning model to predict in-hospital mortality in sepsis patients: A retrospective temporal validation study,''
emph{Journal of Clinical Medicine}, vol. 12, no. 3, 2023, Art. no. 915. DOI: href{https://doi.org/10.3390/jcm12030915}{10.3390/jcm12030915.}
bibitem{Zhuang2023}
J. Zhuang, H. Huang, S. Jiang, J. Liang, Y. Liu, and X. Yu,
``A generalizable and interpretable model for mortality risk stratification of sepsis patients in intensive care unit,''
emph{BMC Medical Informatics and Decision Making}, vol. 23, no. 1, 2023, Art. no. 2279. DOI: href{https://doi.org/10.1186/s12911-023-02279-0}{ 10.1186/s12911-023-02279-0.}
bibitem{Shen2024}
L. Shen, J. Wu, J. Lan, C. Chen, Y. Wang, and Z. Li,
``Interpretable machine learning-based prediction of 28-day mortality in ICU patients with sepsis: a multicenter retrospective study,''
emph{Frontiers in Cellular and Infection Microbiology}, vol. 14, 2024, Art. no. 1500326. DOI: href{https://doi.org/10.3389/fcimb.2024.1500326}{10.3389/fcimb.2024.1500326.}
bibitem{Mondrejevski2024}
L. Mondrejevski, F. Rugolon, I. Miliou, and P. Papapetrou,
``MASICU: A multimodal attention-based classifier for sepsis mortality prediction in the ICU,''
in emph{Proc. IEEE Symp. Computer-Based Medical Systems (CBMS)}, 2024, pp. 326--331. DOI: href{10.1109/CBMS61543.2024.00061.}{10.1109/CBMS61543.2024.00061.}
bibitem{Zhu2025}
C. Q. Zhu, M. Tian, L. Semenova, J. Liu, J. Xu, J. Scarpa, and C. Rudin,
``Fast and interpretable mortality risk scores for critical care patients,''
emph{J. Amer. Med. Informatics Assoc.}, vol. 32, no. 4, pp. 736--747, 2025. DOI: href{10.1093/jamia/ocae318.}{10.1093/jamia/ocae318.}
bibitem{Lundberg2017}
S. M. Lundberg and S.-I. Lee,
``A unified approach to interpreting model predictions,''
in emph{Advances in Neural Information Processing Systems 30 (NeurIPS 2017)},
I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, Eds.
Curran Associates, Inc., 2017, pp. 4765--4774. DOI: href{https://doi.org/10.48550/arXiv.1705.07874}{10.48550/arXiv.1705.07874.}
bibitem{Ribeiro2016}
M. T. Ribeiro, S. Singh, and C. Guestrin,
``"Why should I trust you?": Explaining the predictions of any classifier,''
in emph{Proc. 22nd ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining (KDD)},
San Francisco, CA, USA, 2016, pp. 1135--1144. DOI: href{https://doi.org/10.1145/2939672.2939778}{10.1145/2939672.2939778.}
bibitem{Santos2025}
C. Santos, J. X. Ramos Garzón, S. D. Pertuz, and C. A. Fajardo,
``Significant clinical factors for mortality prediction in ICU sepsis patients: A machine learning approach,''
emph{Smart Health}, vol. 38, p. 100613, 2025. DOI: href{https://doi.org/10.1016/j.smhl.2025.100613}{10.1016/j.smhl.2025.100613}.
bibitem{MIMIC_database}
A. E. W. Johnson, L. Bulgarelli, L. Shen, A. Gayles, A. Shammout, S. Horng, T. J. Pollard, S. Hao, B. Moody, B. Gow, L.-w. H. Lehman, L. A. Celi, and R. G. Mark,
``MIMIC-IV, a freely accessible electronic health record dataset,''
emph{Scientific Data}, vol. 10, no. 1, p. 1, 2023. DOI: href{https://doi.org/10.1038/s41597-022-01899-x}{10.1038/s41597-022-01899-x.}
bibitem{eICU_database}
T. J. Pollard, A. E. W. Johnson, J. D. Raffa, L. A. Celi, R. G. Mark, and O. Badawi,
``The eICU collaborative research database, a freely available multi-center database for critical care research,''
emph{Scientific Data}, vol. 5, no. 1, p. 180178, 2018. DOI: href{https://doi.org/10.1038/sdata.2018.178}{10.1038/sdata.2018.178.}
bibitem{sepsis3}
M. Singer, C. S. Deutschman, C. W. Seymour, M. Shankar-Hari, D. Annane, M. Bauer, R. Bellomo, G. R. Bernard, J.-D. Chiche, C. M. Coopersmith, R. S. Hotchkiss, M. M. Levy, J. C. Marshall, G. S. Martin, S. M. Opal, G. D. Rubenfeld, T. van der Poll, J.-L. Vincent, and D. C. Angus,
``The third international consensus definitions for sepsis and septic shock (Sepsis-3),''
emph{JAMA}, vol. 315, no. 8, pp. 801--810, Feb. 2016. DOI: href{https://doi.org/10.1001/jama.2016.0287}{10.1001/jama.2016.0287}.
bibitem{miceforest_45}
The miceforest Development Team, emph{miceforest: Fast, Memory Efficient Imputation with LightGBM}, versión 6.0.3, 2024. [Online]. Available: url{https://pypi.org/project/miceforest/}
bibitem{Porat2025}
A.~Porat, B.~S.~Bhutta, and S.~Kesler, ``Urosepsis,'' in emph{StatPearls [Internet]}, Treasure Island (FL): StatPearls Publishing, 2025. [Updated: Aug. 17, 2023]. [Online]. Available: url{https://www.ncbi.nlm.nih.gov/books/NBK482344/}.
PMID: 29493969. Bookshelf ID: NBK482344.
bibitem{Avni2015vasopressors}
T.~Avni, A.~Lador, S.~Lev, L.~Leibovici, M.~Paul, and A.~Grossman, ``Vasopressors for the treatment of septic shock: Systematic review and meta‑analysis,'' emph{PLOS ONE}, vol.~10, no.~8, p.~e0129305, 2015, doi: href{https://doi.org/10.1371/journal.pone.0129305}{10.1371/journal.pone.0129305}.
