Accurately predicting mortality in Intensive Care Units (ICUs) is a vital component of clinical decision support. However, traditional feature-attribution methods rarely provide the clinical narrative necessary for practical bedside use. A new feasibility study, utilizing the eICU Demo dataset, investigates whether Large Language Models (LLMs) can bridge this gap by providing more interpretable explanations.
Study Methodology
Researchers analyzed a dataset comprising 2,353 ICU stays with an 8.1% mortality rate. An XGBoost model was initially employed to predict mortality, achieving an AUROC of 0.855. The study then compared two distinct approaches for generating clinical explanations: a standalone LLM and a pre-specified four-step agentic pipeline. This pipeline was designed to separate data interpretation, guideline checking, and the final narrative construction.
Key Findings and Safety Implications
The comparison, conducted on a stratified 38-case subset, revealed critical performance variances:
- Outcome Leakage: The standalone LLM produced one explanation with explicit outcome leakage, whereas the agentic pipeline produced none, suggesting higher structural integrity.
- Guideline Grounding: The agentic pipeline showed significantly higher grounding in clinical guidelines (0.762 vs. 0.143).
- Value Specificity: The multi-step approach provided better patient-specific detail (0.236 vs. 0.143) and slightly higher overall plausibility.
- SHAP Alignment: Interestingly, the standalone LLM demonstrated higher alignment with SHAP feature attribution (mean Jaccard 0.171 vs. 0.077).
The clinical findings suggest that while agentic decomposition improves safety-relevant grounding and specificity, it is not a complete solution. The researchers conclude that these pipelines should be paired with attribution-based checks before being deployed for high-stakes risk explanations in a medical setting.
