Download PDF
Thorac Cardiovasc Surg 2018; 66(08): 651-660
DOI: 10.1055/s-0037-1608897
Original Cardiovascular
Georg Thieme Verlag KG Stuttgart · New York

Validation of Three Postoperative Risk Prediction Models for Intensive Care Unit Mortality after Cardiac Surgery

Samuel Henry Howitt
1   Institute of Cardiovascular Sciences, University of Manchester, Manchester, England, United Kingdom of Great Britain and Northern Ireland
2   Department of Cardiothoracic Anaesthesia and Critical Care, University Hospital of South Manchester NHS Foundation Trust, Manchester, United Kingdom of Great Britain and Northern Ireland
,
Camila Caiado
3   Department of Statistics, Durham University, Durham, United Kingdom of Great Britain and Northern Ireland
,
Charles McCollum
1   Institute of Cardiovascular Sciences, University of Manchester, Manchester, England, United Kingdom of Great Britain and Northern Ireland
,
Michael Goldstein
3   Department of Statistics, Durham University, Durham, United Kingdom of Great Britain and Northern Ireland
,
Ignacio Malagon
2   Department of Cardiothoracic Anaesthesia and Critical Care, University Hospital of South Manchester NHS Foundation Trust, Manchester, United Kingdom of Great Britain and Northern Ireland
,
Rajamiyer Venkateswaran
4   Department of Cardiothoracic Surgery, University Hospital of South Manchester NHS Foundation Trust, Manchester, England, United Kingdom of Great Britain and Northern Ireland
,
Stuart William Grant
1   Institute of Cardiovascular Sciences, University of Manchester, Manchester, England, United Kingdom of Great Britain and Northern Ireland
› Author Affiliations Funding Statement Work performed producing this manuscript was funded by the British Heart Foundation (grant number PG/16/80/32411).
Further Information

Publication History

17 August 2017

19 October 2017

Publication Date:
09 January 2018 (online)

    Permissions and Reprints All articles of this category

Abstract

Background Several cardiac surgery risk prediction models based on postoperative data have been developed. However, unlike preoperative cardiac surgery risk prediction models, postoperative models are rarely externally validated or utilized by clinicians. The objective of this study was to externally validate three postoperative risk prediction models for intensive care unit (ICU) mortality after cardiac surgery.

Methods The logistic Cardiac Surgery Scores (logCASUS), Rapid Clinical Evaluation (RACE), and Sequential Organ Failure Assessment (SOFA) scores were calculated over the first 7 postoperative days for consecutive adult cardiac surgery patients between January 2013 and May 2015. Model discrimination was assessed using receiver operating characteristic curve analyses. Calibration was assessed using the Hosmer–Lemeshow (HL) test, calibration plots, and observed to expected ratios. Recalibration of the models was performed.

Results A total of 2255 patients were included with an ICU mortality rate of 1.8%. Discrimination for all three models on each postoperative day was good with areas under the receiver operating characteristic curve of >0.8. Generally, RACE and logCASUS had better discrimination than SOFA. Calibration of the RACE score was better than logCASUS, but ratios of observed to expected mortality for both were generally <0.65. Locally recalibrated SOFA, logCASUS and RACE models all performed well.

Conclusion All three models demonstrated good discrimination for the first 7 days after cardiac surgery. After recalibration, logCASUS and RACE scores appear to be most useful for daily risk prediction after cardiac surgery. If appropriately calibrated, postoperative cardiac surgery risk prediction models have the potential to be useful tools after cardiac surgery.

Keywords

postoperative care - statistics - outcomes

Letter to this article:

Developing a Risk Prediction Model for Intensive Care Unit Mortality after Cardiac SurgeryThorac Cardiovasc Surg 2018; 66(08): e1-e2
DOI: 10.1055/s-0038-1639336

Letter to this article:

Reply to “Developing a Risk Prediction Model for Intensive Care Unit Mortality after Cardiac Surgery”Thorac Cardiovasc Surg 2018; 66(08): e3-e4
DOI: 10.1055/s-0038-1639342

 

  • References

  • 1 Nashef SA, Roques F, Michel P, Gauducheau E, Lemeshow S, Salamon R. European system for cardiac operative risk evaluation (EuroSCORE). Eur J Cardiothorac Surg 1999; 16 (01) 9-13
    CrossrefPubMedGoogle Scholar
  • 2 Roques F, Michel P, Goldstone AR, Nashef SA. The logistic EuroSCORE. Eur Heart J 2003; 24 (09) 881-882
    CrossrefPubMedGoogle Scholar
  • 3 Nashef SA, Roques F, Sharples LD. , et al. EuroSCORE II. Eur J Cardiothorac Surg 2012; 41 (04) 734-744 , discussion 744–745
    CrossrefPubMedGoogle Scholar
  • 4 Hekmat K, Kroener A, Stuetzer H. , et al. Daily assessment of organ dysfunction and survival in intensive care unit cardiac surgical patients. Ann Thorac Surg 2005; 79 (05) 1555-1562
    CrossrefPubMedGoogle Scholar
  • 5 Howitt SH, Grant SW, Riding DM, Malagon I, McCollum CN. Risk models that use postoperative patient monitoring data to predict outcomes in adult cardiac surgery: a systematic review. J Cardiothorac Vasc Anesth 2017; 31 (05) 1865-1877
    CrossrefPubMedGoogle Scholar
  • 6 Hekmat K, Doerr F, Kroener A. , et al. Prediction of mortality in intensive care unit cardiac surgical patients. Eur J Cardiothorac Surg 2010; 38 (01) 104-109
    CrossrefPubMedGoogle Scholar
  • 7 Doerr F, Badreldin AM, Heldwein MB. , et al. A comparative study of four intensive care outcome prediction models in cardiac surgery patients. J Cardiothorac Surg 2011; 6: 21
    CrossrefPubMedGoogle Scholar
  • 8 Ceriani R, Mazzoni M, Bortone F. , et al. Application of the sequential organ failure assessment score to cardiac surgical patients. Chest 2003; 123 (04) 1229-1239
    CrossrefPubMedGoogle Scholar
  • 9 Pätilä T, Kukkonen S, Vento A, Pettilä V, Suojaranta-Ylinen R. Relation of the Sequential Organ Failure Assessment score to morbidity and mortality after cardiac surgery. Ann Thorac Surg 2006; 82 (06) 2072-2078
    CrossrefPubMedGoogle Scholar
  • 10 Gomes RV, Tura B, Mendonça Filho HT. , et al. A first postoperative day predictive score of mortality for cardiac surgery. Ann Thorac Cardiovasc Surg 2007; 13 (03) 159-164
    PubMedGoogle Scholar
  • 11 Badreldin A, Elsobky S, Lehmann T, Brehm BB, Doenst T, Hekmat K. Daily-Mean-SOFA, a new derivative to increase accuracy of mortality prediction in cardiac surgical intensive care units. Thorac Cardiovasc Surg 2012; 60 (01) 43-50
    Article in Thieme ConnectPubMedGoogle Scholar
  • 12 Doerr F, Badreldin AM, Can F, Bayer O, Wahlers T, Hekmat K. SAPS 3 is not superior to SAPS 2 in cardiac surgery patients. Scand Cardiovasc J 2014; 48 (02) 111-119
    CrossrefPubMedGoogle Scholar
  • 13 Ariyaratnam P, Loubani M, Biddulph J. , et al. Validation of the intensive care national audit and research centre scoring system in a UK adult cardiac surgery population. J Cardiothorac Vasc Anesth 2015; 29 (03) 565-569
    CrossrefPubMedGoogle Scholar
  • 14 Heldwein MB, Badreldin AM, Doerr F. , et al. Logistic Organ Dysfunction Score (LODS): a reliable postoperative risk management score also in cardiac surgical patients?. J Cardiothorac Surg 2011; 6: 110
    CrossrefPubMedGoogle Scholar
  • 15 Vincent JL, Moreno R, Takala J. , et al. The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure. On behalf of the Working Group on Sepsis-Related Problems of the European Society of Intensive Care Medicine. Intensive Care Med 1996; 22 (07) 707-710
    CrossrefPubMedGoogle Scholar
  • 16 Badreldin AM, Doerr F, Ismail MM. , et al. Comparison between Sequential Organ Failure Assessment score (SOFA) and Cardiac Surgery Score (CASUS) for mortality prediction after cardiac surgery. Thorac Cardiovasc Surg 2012; 60 (01) 35-42
    Article in Thieme ConnectPubMedGoogle Scholar
  • 17 Exarchopoulos T, Charitidou E, Dedeilias P, Charitos C, Routsi C. Scoring Systems for Outcome Prediction in a Cardiac Surgical Intensive Care Unit: A Comparative Study. Am J Crit Care 2015; 24 (04) 327-334 , quiz 335
    CrossrefPubMedGoogle Scholar
  • 18 Doerr F, Badreldin AM, Bender EM. , et al. Outcome prediction in cardiac surgery: the first logistic scoring model for cardiac surgical intensive care patients. Minerva Anestesiol 2012; 78 (08) 879-886
    PubMedGoogle Scholar
  • 19 Badreldin AM, Doerr F, Bender EM. , et al. Rapid clinical evaluation: an early warning cardiac surgical scoring system for hand-held digital devices. Eur J Cardiothorac Surg 2013; 44 (06) 992-997 , discussion 997–998
    CrossrefPubMedGoogle Scholar
  • 20 R Core Team. R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing; 2015
    Google Scholar
  • 21 DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 1988; 44 (03) 837-845
    CrossrefPubMedGoogle Scholar
  • 22 Hosmer DW, Lemeshow S. Applied Logistic Regression. 2nd ed. NY, USA: Wiley; 2000: 147-155
    Google Scholar
  • 23 Bridgewater B, Grant SW, Hickey GL. , et al. National Adult Cardiac Surgery Audit Report. National Institute for Cardiovascular Outcomes Research; 2012 2012
    Google Scholar
  • 24 Hickey GL, Grant SW, Murphy GJ. , et al. Dynamic trends in cardiac surgery: why the logistic EuroSCORE is no longer suitable for contemporary cardiac surgery and implications for future risk models. Eur J Cardiothorac Surg 2013; 43 (06) 1146-1152
    CrossrefPubMedGoogle Scholar
  • 25 Grant SW, Grayson AD, Jackson M. , et al. Does the choice of risk-adjustment model influence the outcome of surgeon-specific mortality analysis? A retrospective analysis of 14,637 patients under 31 surgeons. Heart 2008; 94 (08) 1044-1049
    CrossrefPubMedGoogle Scholar
  • 26 Ferreira FL, Bota DP, Bross A, Mélot C, Vincent JL. Serial evaluation of the SOFA score to predict outcome in critically ill patients. JAMA 2001; 286 (14) 1754-1758
    CrossrefPubMedGoogle Scholar
  • 27 Grant SW, Hickey GL, Dimarakis I. , et al. How does EuroSCORE II perform in UK cardiac surgery; an analysis of 23 740 patients from the Society for Cardiothoracic Surgery in Great Britain and Ireland National Database. Heart 2012; 98 (21) 1568-1572
    PubMedGoogle Scholar
  • 28 Grant SW, Hickey GL, Cosgriff R. , et al. Creating transparency in UK adult cardiac surgery data. Heart 2013; 99 (15) 1067-1068
    PubMedGoogle Scholar