SEPRES: Intensive Care Unit Clinical Data Integration System to Predict Sepsis

Abstract

Background The lack of information interoperability between different devices and systems in the intensive care unit (ICU) hinders further utilization of data, especially for early warning of specific diseases in the ICU.

Objectives We aimed to establish a data integration system. Based on this system, the sepsis prediction module was added to compose the Sepsis PREdiction System (SEPRES), where real-time early warning of sepsis can be implemented at the bedside in the ICU.

Methods Data are collected from bedside devices through the integration hub and uploaded to the integration system through the local area network. The data integration system was designed to integrate vital signs data, laboratory data, ventilator data, demographic data, pharmacy data, nursing data, etc. from multiple medical devices and systems. It integrates, standardizes, and stores information, making the real-time inference of the early warning module possible. The built-in sepsis early warning module can detect the onset of sepsis within 5 hours preceding at most.

Results Our data integration system has already been deployed in Ruijin Hospital, confirming the feasibility of our system.

Conclusion We highlight that SEPRES has the potential to improve ICU management by helping medical practitioners identify at-sepsis-risk patients and prepare for timely diagnosis and intervention.

Keywords

sepsis - information system - intensive care units

Background and Significance

Sepsis is a syndrome of physiologic, pathologic, and biochemical abnormalities induced by infection.[1] It is a global medical problem associated with unacceptably high mortality, long-term morbidity, and a major cost burden on health care resources.[2] [3] Early detection and timely administration of appropriate antibiotics are probably the most important factors in improving the prognosis of septic patients.[4] However, non-specific symptoms in septic patients lead to delayed diagnosis and delayed intervention.[5]

Machine learning, including regression models, survival models, decision trees, and neural networks, has become a promising tool for predicting sepsis based on electronic medical records, laboratory data, and biomedical signals.[6] [7] [8] [9] [10] [11] [12] In 2016, Singer et al proposed a new definition (Sepsis-3) of sepsis, which defined sepsis as life-threatening organ dysfunction caused by a dysregulated host response to infection.[1] According to this, many recent papers defined sepsis by Sequential Organ Failure Assessment (SOFA) and infection instead of SIRS.[13] [14] [15] [16] [17] [18] [19]

Most studies on sepsis prediction used historical medical data,[20] such as the Medical Information Mart for Intensive Care (MIMIC-III).[21] However, the raw data needed for clinical model inference, such as bedside data, laboratory data, demographic data, and doctor's orders, usually come from different devices. Moreover, the information cannot interact directly due to differences in the data transfer protocols between devices. Data displayed on discrete devices can divide medical practitioners' attention and hinder further data utilization. On the other hand, there are sepsis alert systems based on traditional scoring or screening processes that have been successfully deployed,[22] [23] [24] but the electronic health record (EHR) or single data source they used may constrain the performance of machine learning models targeting prediction.

Efforts have been made to integrate bedside medical devices. Smielewski et al developed ICM+ software that allowed easy configuration and real-time trending of complex parameters derived from multiple bedside monitoring devices.[25] Meyer et al implemented a system for the operating room that integrates data from surgical and anesthesia devices, information systems, and a location tracking system.[26] Goldstein et al developed a real time, physiologic data acquisition system in the pediatric intensive care unit (ICU).[27] Gjermundrod et al implemented the Intensive Care Window which can retrieve and integrate data from different patient monitoring devices in ICU.[28] Sun et al proposed an integrated system Integrated System for Multimodal Data Acquisition and Analysis (INSMA), which supports multimodal data acquisition, parsing, real-time data analysis, and visualization in the ICU.[29] In Shimabukuro et al, Burdick et al, and McCoy and Das,[30] [31] [32] the authors combined information systems with sepsis prediction, with data autonomously obtained from EHR, and demonstrated the effectiveness of machine learning algorithms in predicting sepsis clinically. However, these data integration systems or prediction systems integrate a more limited number of devices and data types to present the complete perspective of a doctor.

Objectives

We aim to integrate various devices and systems at the bedside in the ICU to obtain high-density raw data. These data were processed and fed into the sepsis prediction module to help achieve real-time prediction of sepsis. In this study, we developed a data integration system that integrates IntelliVue Information Center, Ventilators, Philips ICCA system, Laboratory Information System (LIS), and Hospital Information System, and established a real-time early warning system for sepsis in the ICU, named SEpsis PREdiction System (SEPRES).

Methods

SEPRES comprises a data integration system with a sepsis early warning module. The data integration system completes the collection, storage, processing, and display of medical data. The sepsis early warning module comprises a sepsis prediction model and an interpretative tool. The sepsis prediction model is an ensemble model of gradient boosting machine and multilayer perceptron (MLP) that can output the risk value of sepsis onset within 5 hours preceding at most. The interpretative tool supplies information about how the model works by attributing importance value to each input feature. We focused on the construction of the data integration system in this work. The construction of the sepsis prediction model in SEPRES will be briefly described and more information can be found in Chen et al.[33]

System Framework

As shown in [Fig. 1], the system comprises a physical server with the PostgreSQL database to store the sepsis warning data and the web server to deploy the portal for user access. The web release system of the sepsis early warning system applies Brower/Server architecture. The whole architecture can be divided into the following parts.

• Device side: The medical device integration hub transmits the device data to the data integration system through the local area network.

• Data management side: Heterogeneous data are integrated into the data integration system. The interface data, service data, and model predictions are stored and managed by the Structured Query Language (SQL) server, while the parts needed for the sepsis early warning module are sent to the PostgreSQL database. The Message Queuing Telemetry Transport server sends real-time data from the data integration system to the browser.

• Data server side: The web server uses the AJAX interface to respond to the browser's request and calls the sepsis early warning module. Data fetching, data cleaning, feature extraction, standardization, and other preprocessing are implemented in turn. Model inference is then executed, and the predictions are stored in the PostgreSQL database. The data server side provides business support for the browser-side interface, including some related services (real-time calculation of the SOFA score, determination of suspected infection, data statistics, data charts, and historical data query).

• Application side: The user's request is passed to the web server in this layer, and the processing results are displayed in the system. The JavaScript program is used for dynamic HTML page development, and the AJAX interface is used for data interaction with the web server. Spring MVC is used to build full-featured MVC modules for web applications, combined with NODEJS to provide an elegant and highly maintainable method for creating templates. Users can use the system anytime and anywhere with a browser in various ways, such as on PCs and mobile terminals.

Fig. 1 System deployment framework.

The data collected by SEPRES is shown in [Table 1]. These data can be output in HL-7 V2 format. Also, our system supports SQL queries based on datetime, and the results are converted to MS Excel for output. Considering data availability and importance, we extracted 63 variables for predicting sepsis in Ruijin Hospital, as shown in [Supplementary Appendix A] (available in the online version).

Table 1 The types of data collected from the devices and systems

Source device/system	Data type	Output medium	Format
IntelliVue Information Center	Vital signs data	Network	HL-7 V2
PB 840 Ventilator Maquet Servo-i Ventilator Maquet Servo-s Ventilator	Ventilator data	RS-232	According to the device output format
Philips ICCA	Pharmacy data, GCS, and urine output	Network	Web services
Laboratory Information System	Laboratory data	Network	Web services
Hospital Information System	Admission, discharge, and hospitalization data	Network	Web services

Abbreviations: GCS, Glasgow Coma Scale; ICCA, International Congress and Convention Association.

The data integration system queries the devices and systems at regular intervals or receives data sent by the system at regular intervals. The detailed modes and frequencies are shown in [Table 2].

Table 2 The types of data collected from the devices and systems

Source device/system	Update time	Description
IntelliVue Information Center	Per minute	The IIC sends data every minute and the integration system receives
PB 840 Ventilator Maquet Servo-i Ventilator Maquet Servo-s Ventilator	Per minute	The medical device integration hub queries data to the devices every minute and uploads it to the integration system
Philips ICCA	Recorded by nurses at regular intervals or when executing medical orders	The integration system regularly queries the ICCA database for recently updated nurse records and medical orders
Laboratory Information System	Recorded when reports are complete	The integration system regularly queries the LIS for completed inspection reports
Hospital Information System	Recorded when the patient is admitted to or leaves the ward	The integrated system regularly queries the HIS for the current status of patients in the ward

Abbreviation: ICCA, International Congress and Convention Association; LIS, Laboratory Information System.

Medical Device Integration Hub

We developed a medical device integration hub that can acquire and transmit data from different brands of medical devices. The medical device integration hub consists of customized device connection lines, a hub, and an integrated data receiver. The identification module containing encoding is inserted into each medical device, enabling the hub to identify the type of online device and collect data automatically according to the communication protocol. The integrated data receiver receives and translates the raw data and uploads them to the integration server through the local area network. The medical device integration hub has the following functions:

• Device online services: detecting device connections and starting a data reading program corresponding to the device.

• Decoding: parsing raw data into structured data for further processing.

• Storage: storing parsed data into native memory.

• Remote settings: supporting remote system setup and sending system status.

• Uploading: uploading the received data to the specified database.

Most medical devices communicate through the network or RS-232 port, and the device data are transmitted to the data integration system through the HL-7 V2 interface, web services, or other protocols.

Study Design for Sepsis Prediction Model

For reasons of completeness, we briefly describe the implementation of the sepsis prediction model. A more detailed review can be found in Chen et al.[33] Three data sources were used in this study, including MIMIC-III database (version 1.4), the private Historical Database of Ruijin Hospital (HDRJH), and the real-time operational data of the ICU at Ruijin Hospital. We defined sepsis according to the criteria of Sepsis-3. After the data extraction step, we filtered out 6,891 patients from the MIMIC-III from 2001 to 2012, and 453 patients from the HDRJH from 2011 to 2019 to be included in the study. As a validation of the model in the real world, the real-time data at Ruijin Hospital was collected from 67 consecutive patients from the SEPRES running between February 2021 and June 2021. During the run, all patients over the age of 14 were included. Patient characteristics from MIMIC-III, HDRJH, and Ruijin real-world data were summarized in [Table 3].

Table 3 Variable statistics on admission

Variable	MIMIC-III (n = 6,891)		HDRJH (n = 453)		RJ real world (n = 67)
	Number (%)	Missing	Number (%)	Missing	Number (%)	Missing
Gender, male	3,995 (57.97)	0	258 (56.95)	0	43 (65.15)	1
Ventilation	2,166 (31.43)	0	172 (37.97)	0	33 (50)	1
	Mean (SD)	Missing	Mean (SD)	Missing	Mean (SD)	Missing
SOFA	2.67 (2.37)	0	3.13 (2.53)	0	4.64 (3.37)	0
Age (y)	61.32 (17.15)	0	61.76 (18.66)	4	63.36 (17.13)	1
Weight (kg)	83.18 (24.38)	299	63.54 (15.38)	147	63.54 (14.29)	3
Heart rate (bpm)	84.25 (17.3)	4	88.18 (18.65)	0	88.83 (23.33)	2
Systolic blood pressure (mm Hg)	123.85 (22.5)	4	132.15 (20.97)	2	130.75 (27.63)	2
Diastolic blood pressure (mm Hg)	63.75 (15.23)	4	75.04 (14.46)	2	75.92 (18.54)	2
Mean arterial pressure (mm Hg)	81.35 (16.47)	4	88.34 (14.67)	2	93.18 (17.91)	2
Respiratory rate (insp/min)	18.93 (5.37)	6	19.42 (4.76)	0	19.74 (5.34)	2
Temperature (°C)	36.85 (0.72)	8	37.29 (0.62)	0	37.03 (0.79)	2
SpO2 (%)	97.09 (3.04)	4	99.12 (1.91)	0	98.44 (4.31)	19
pH	7.39 (0.07)	3,006	7.4 (0.05)	1	7.4 (0.07)	2
PaO2 (mm Hg)	142.84 (77.85)	3,486	108.68 (42.29)	53	139.73 (65.91)	2
SaO2 (%)	96.41 (4.14)	5,084	96.6 (5.55)	1	99.04 (1.27)	2
AaDO2 (mm Hg)	461.85 (121.42)	6,203	110.02 (83.84)	12	129.86 (84.33)	2
PCO2 (mm Hg)	41.69 (9.65)	3,378	39.49 (8.74)	1	41.09 (14.31)	2
Bicarbonate (mEq/L)	24.93 (4.42)	3	24.01 (3.12)	1	23.94 (5.31)	2
Base excess	0.29 (4.77)	3,326	−0.22 (3.68)	1	−0.13 (5.52)	2
White blood cell count (1012/L)	11.32 (8.93)	1	10.75 (4.78)	0	10.32 (6)	2
Neutrophils (%)	78.81 (13.73)	3,801	80.9 (12.18)	3	85.85 (12.78)	2
Monocytes (%)	4.7 (3.4)	3,801	4.55 (4.77)	4	3.43 (2.68)	2
Lymphocytes (%)	12.97 (10.6)	3,801	12.23 (9.22)	0	8.31 (5.24)	2
Red blood cell count (1012/L)	3.61 (0.64)	2	3.55 (0.76)	0	5.1 (15.13)	2
Hemoglobin (g/dL)	10.85 (1.9)	1	10.43 (2.23)	0	9.74 (2.55)	3
Hematocrit (%)	32.06 (5.39)	0	31.31 (6.52)	0	30.97 (7.68)	3
Platelets (1012/L)	238.27 (129.13)	1	217.08 (111.56)	0	180.02 (118.32)	2
BUN (mg/dL)	24.52 (21.25)	1	6.63 (4.7)	260	9.99 (8.38)	2
Creatinine (mg/dL)	1.35 (1.64)	1	0.96 (1.04)	2	1.7 (2.32)	2
Uric acid (mg/dL)	5.68 (3.01)	6,594	3.98 (2.1)	13	4.87 (2.57)	2
LDH (IU/L)	394.09 (806.44)	4,057	250.06 (162.55)	232	505.29 (1,849.89)	2
ALP (U/L)	115.37 (124.08)	2,946	91.89 (93.57)	3	92.48 (59.21)	2
AST (U/L)	131.16 (562.35)	2,857	56.17 (152.79)	2	177.83 (851.23)	2
Bilirubin (mg/dL)	1.61 (4.07)	2,889	1.64 (2.42)	2	1.61 (1.65)	2
Bilirubin direct (mg/dL)	3.36 (4.94)	6,387	10.8 (23.92)	6	10.98 (17.33)	2
Albumin (g/dL)	3.11 (0.65)	3,637	3.09 (0.5)	1	3.03 (0.51)	2
Partial thromboplastin time (s)	36.08 (19.9)	616	35.71 (17.02)	3	32.89 (13.33)	2
Prothrombin time (s)	15.04 (5.86)	594	14.04 (4.54)	3	14.72 (3.53)	2
INR	1.38 (0.71)	594	1.18 (0.27)	3	1.26 (0.32)	2
Fibrinogen (mg/dL)	365.79 (199.95)	5,711	292.56 (131.82)	8	339.38 (120.92)	2
Lactate (mmol/L)	1.85 (1.5)	4,091	2.48 (1.92)	51	2.25 (1.64)	3
Glucose (mg/dL)	135.77 (55.62)	0	156.29 (54.24)	69	193.65 (87.6)	2
Sodium (mEq/L)	138.64 (4.42)	1	138.81 (5.8)	113	138.98 (6.7)	2
Chloride (mEq/L)	104.64 (5.64)	2	101.89 (4.98)	0	106.32 (6.35)	2
Potassium (mEq/L)	4.07 (0.6)	0	3.86 (0.45)	2	3.92 (0.63)	2
Phosphorus (mEq/L)	3.5 (1.25)	306	3.05 (1.18)	54	3.37 (1.76)	2
Magnesium (mg/dL)	2.01 (0.36)	80	2 (0.35)	173	2.05 (0.39)	16
Troponin I (ng/mL)	6.14 (9.28)	6,656	0.26 (2.49)	235	0.25 (0.68)	7
Creatine kinase (IU/L)	716.22 (3592.92)	3,314	258.64 (895.02)	232	175.38 (254.81)	1
Creatine kinase MB (ng/mL)	30.53 (73.44)	4,075	2.93 (4.45)	234	3.8 (6.12)	1

Abbreviations: ALP, alanine transaminase; AST, aspartate transaminase; BUN, blood urea, nitrogen; HDRJH, Historical Database of Ruijin Hospital; INR, international normalized ratio; LDH, lactate dehydrogenase; MIMIC, Medical Information Mart for Intensive Care; SD, standard deviation; SOFA, Sequential Organ Failure Assessment.

Sepsis Prediction Model

We used 63 extracted variables and a 5-hour time window as input to predict the occurrence of sepsis. To take advantage of the rich data in MIMIC-III to mitigate the impact of insufficient data in HDRJH, the sepsis prediction model underwent two stages of training. Meanwhile, the model underwent three stages of validation. First, the MLP and LightGBM models were trained on MIMIC-III and validated on a test set that was randomly divided at the patient level. After that, these models are fine-tuned and retrained on HDRJH and validated on a test set that was randomly divided at the patient level. We took the inference average of the two types of models as the ensemble model. Finally, as an observational study, the ensemble model was deployed in SEPRES and validated in the real world.

Results

The performance of our machine learning model on the three datasets is shown in [Table 4]. We evaluated it through a combination of accuracy, receiver operating characteristic area under the curves (AUC), sensitivity, and specificity. Our machine learning model obtained an AUC of 0.98 on MIMIC-III from 1 to 5 hours preceding. The AUCs were 0.94 to 0.94 and 0.86 to 0.9, respectively on HDRJH and Ruijin real-world data from 1 to 5 hours preceding. Our model outperformed most of the similar literature on MIMIC-III, where the details can be found in Chen et al.[33] We next focus on the implementation of the data integration system.

Table 4 The results of sepsis prediction model

	Dataset	Preceding hours	Accuracy	AUC	Sensitivity	Specificity
MLP	MIMIC	1	0.84	0.95	0.71	0.98
		2	0.84	0.95	0.71	0.97
		3	0.84	0.96	0.71	0.97
		4	0.85	0.96	0.73	0.96
		5	0.85	0.96	0.74	0.96
LightGBM	MIMIC	1	0.89	0.98	0.81	0.97
		2	0.9	0.98	0.82	0.97
		3	0.9	0.98	0.83	0.96
		4	0.91	0.98	0.85	0.97
		5	0.91	0.98	0.84	0.97
Ensemble	HDRJH	1	0.86	0.94	0.72	1
		2	0.86	0.94	0.72	1
		3	0.88	0.94	0.75	1
		4	0.82	0.94	0.64	0.99
		5	0.87	0.94	0.75	0.99
Ensemble	Real-world data	1	0.82	0.86	0.83	0.82
		2	0.84	0.88	0.87	0.78
		3	0.85	0.9	0.86	0.81
		4	0.85	0.9	0.88	0.78
		5	0.86	0.89	0.9	0.76

Abbreviations: AUC, area under the receiver operating characteristics curve; HDRJH, Historical Database of Ruijin Hospital; MIMIC, Medical Information Mart for Intensive Care; MLP, multilayer perceptron.

Model Inference

We use Python to implement these models. We apply Python.Net package to realize the interaction between .NET Framework and Python, and SQLAlchemy package to realize the interaction between Python and the database system. When executing model inference, the following steps are performed sequentially:

1. Use SQL query statements to obtain the real-time features of patients, and pass them into Python through the interaction between Python and PostgreSQL.

2. Standardize the features by calling the scaler which is the standardizing function obtained in the training set.

3. Call the trained models to get the prediction results.

4. Call the interpretive tools to get the importance of the features based on the prediction results.

5. Transmit the results to the NET Framework using Python.Net.

6. Output and store the prediction results and interpretations in a standard format.

System Deployment

[Fig. 2] shows the medical device integration hub installed at Ruijin Hospital. The hub was placed at the bedside, receiving data from multiple devices via different interfaces shown at the bottom of the figure, storing the past 72 hours of data into native memory, and transmitting data with a time delay of under 10 seconds. The interfaces distributed on two sides of the hub include two universal network interfaces, four USB interfaces for mouse, keyboard, and U disk, two HDMI for extended display, one RS-232, and eight or sixteen USB and Ethernet multiplexing interfaces for medical devices. The hub can integrate data from the monitor, ventilator, infusion pump, and dialysis machine. These processed data were then transmitted to the data integration system.

Fig. 2 The medical device integration hub installed at Ruijin Hospital.

Patient Statistics

As shown in [Table 3], the variables on admission to the ICU for the 67 patients in the RJ real-world section were collected by SEPRES. The Missing column indicates the number of patients for which the variable was not recorded. The absence of records, especially laboratory variables, was attributed mainly to the short stay in the ICU for some patients, resulting in the lack of corresponding tests.

System Operation

SEPRES provides predictions and explanations for every patient in the ICU every hour, including the risk of sepsis onset in the next 5 hours, the influence of features on the predictions calculated by SHapley additive explanation (SHAP)[34] and SOFA predictions. SEPRES helps doctors focus on patients who are more likely to develop sepsis and observe changes in physiological data and conditions more conveniently.

The PC terminal of the user interface is deployed in the ICU of Ruijin Hospital. [Fig. 3] shows an example of the page displaying all 12 patients in the ICU. Each patient takes a panel, and the title bar provides the patient's ward number, patient identification number, name, and gender information. For privacy reasons, the image has been processed and the patient identifier has been removed. Recent trends in SOFA are shown in the upper left, and the maximum and minimum values of SOFA and sepsis-onset prediction in the last 24 hours are summarized in the upper right. The lower part of the panel shows the predicted sepsis-onset risk for the last 5 hours and the next 4 hours, with the trend fitted. High and low risks are shown by red and blue bars, respectively. By double-clicking on any panel, the influence of features calculated by SHAP for two models' predictions is displayed, and the features with the highest absolute value of importance are displayed on the right side, as shown in [Fig. 4]. The title bar of this page provides information on the patient identification number, calculation time, and prediction time.

Fig. 3 An example of the user interface. The original figure has been translated and the patient identifying information has been removed.

Fig. 4 The example of SHAP values calculated for two models in a single prediction. The original figure has been translated and the patient identifying information has been removed.

Another major interface of the early warning system is shown in [Fig. 5]. In this interface, variable data for a specific patient during a certain historical period can be queried. At the top of the page is the filtering criteria, including ward number, patient identification number, query start time, and query end time. On the left side is the optional variable type. After the selection is completed, the results are displayed on the right side of the page, with numerical data and line graphs sorted by time on the top and bottom, respectively. Medical practitioners can filter variables of interest freely from any period in the past to track the patient's condition promptly.

Fig. 5 Historical data review for an individual patient. The original figure has been translated and the patient identifying information has been removed.

Discussion

Different types of monitoring equipment used in ICU often reflect various aspects of a patient's status. These devices contain a large amount of information, however, due to the different data types and transmission protocols between different types and brands of devices, the data usually cannot interact with each other, creating a silo effect and making it difficult for further data utilization. Our system identified devices through customized modules, collected data according to the corresponding communication protocols, and then integrated them into one system, enabling high-density and real-time data recording. Vital signs and ventilation data are updated in SEPRES every minute and other data are updated at regular intervals, making real-time sepsis prediction possible. Although prediction was performed at an hourly frequency in our system, we highlight that a higher frequency of prediction is also feasible.

We have optimized the display of the data based on feedback from clinicians. To reduce the false alarm rate, we increased the alert threshold of the model from 0.5 to 0.7. This operation improved the specificity of the prediction within 5 hours preceding from 0.76 to 0.89, which can be found in Chen et al[33] and its Supplementary Material. Another example is that we have adjusted the frequency of presenting data statistics to match the rhythm of clinicians' observations, such that blood pressure will be averaged four times a day, blood glucose will be counted twice in the morning and evening, and heart rate will be presented in real time.

Data such as vital signs are generally of high resolution, which can lead to a lack of storage space. To solve this problem, we divided the data storage into three levels, with devices at different levels storing data for different time periods: at the device side, the data collected in real time is stored in the integration hub, indexed by the device ID and time, stored in JSON format through NOSQL; at the central station side, the data from each integration hub is indexed by the device ID, patient ID and time, and stored in JSON format through NOSQL technology; at the server side, the data from the integration hubs are uploaded individually to the remote database host, and stored in different tables in the relational database according to different device types and data requirements. Each side can remove the obsolete data according to the time to save storage space.

With access to rich data, we can leverage the data to perform higher-level tasks. In SEPRES, we used machine learning models to predict the onset of sepsis in real time, giving risk predictions for each patient. Machine learning methods have been shown to be a promising approach for sepsis early warning.[6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [30] [31] [32] Our results also confirm the feasibility of this approach. Furthermore, this workflow applies to alerts for other diseases. By utilizing the data integration system to collect the features and data required for model construction, we can conveniently construct different models for multiple tasks, such as disseminated intravascular coagulation and acute kidney injury. In the future, we may consider incorporating additional types of data, such as prehospital data, to contribute to the prediction of sepsis and other diseases.[35]

Our model achieved high AUCs on MIMIC-III (0.98) and HDRJH (0.94). Slightly lower AUCs were obtained in the Ruijin real-world data (0.86–0.90), which may be due to potential differences in distribution between these datasets. When training the sepsis prediction model, we used a control group of non-septic patients. The division may be too pure, leading to the possibility that the prediction model may incorrectly identify patients with severe disease as septic. This is a limitation of our current model. To address this issue, we will classify patients with different severity levels and further optimize the prediction model.

Another limitation of our research is the small number of cases in our real-world study (67 cases). However, since we are concerned with the construction of SEPRES in this work, fewer observed cases are acceptable. These cases proved that SEPRES can work well in a period of time. In addition, this observational study is ongoing. We will continue our observations of the system predictions and monitor the results presented. In the future, we can also conduct a multicenter study to examine whether the system can perform well in different health care settings. An additional limitation is that SEPRES depends on the ICU to be equipped with electricity and a network, which requires the cooperation of the hospital for deployment.

In addition to performing basic ICU information system and real-time prediction functions, SEPRES can also provide a valuable source of data for future research works, including retraining current models, other prediction tasks, data analysis, causal inference, etc.

Conclusion

In conclusion, we established an ICU bedside sepsis early warning system, SEPRES to achieve the real-time prediction of ICU patients through the data integration system. This system has been installed in Ruijin Hospital. Our real-world study confirms the feasibility of our system. The system can display the patient's historical data in the user interface, to facilitate doctors to obtain the change of the patient's condition intuitively. The risk of sepsis occurrence calculated by SEPRES allows medical practitioners to focus more on specific patients, enabling early diagnosis of sepsis and more effective management of ICU patients.

Clinical Relevance Statement

SEPRES can integrate various types of information about patients in the ICU and provide early warning of sepsis. It helped medical practitioners to observe patients' disease progression more efficiently and make timely interventions.

Multiple-Choice Questions

1. How many parts can SEPRES be divided into?

   • 2

   • 3

   • 4

   • 5

   Correct Answer: The correct answer is option c. SEPRES can be divided into Device side, Data management side, Data server side, and Application side.

2. Which method was used by SEPRES to provide interpretation of sepsis early warnings?

   • Permutation Feature Importance

   • SHapley Additive exPlanations (SHAP)

   • Local Surrogate (LIME)

   • Counterfactual Explanations

   Correct Answer: The correct answer is option b. SHAP is a game theory-based approach that assigns an importance value to each feature of each prediction.

Conflict of Interest

None declared.

Acknowledgment

The authors thank the MIMIC team for their efforts in collecting and making the MIMIC data publicly available.

Human Subjects Protections

The study was performed in compliance with the World Medical Association Declaration of Helsinki on Ethical Principles for Medical Research Involving Human Subjects, and was approved by the Ruijin Hospital Ethics Committee (no.: 2020 [140]).

• References

• 1 Singer M, Deutschman CS, Seymour CW. et al. The third international consensus definitions for sepsis and septic shock (Sepsis-3). JAMA 2016; 315 (08) 801-810
• 2 Cecconi M, Evans L, Levy M, Rhodes A. Sepsis and septic shock. Lancet 2018; 392 (10141): 75-87
• 3 Angus DC, Linde-Zwirble WT, Lidicker J, Clermont G, Carcillo J, Pinsky MR. Epidemiology of severe sepsis in the United States: analysis of incidence, outcome, and associated costs of care. Crit Care Med 2001; 29 (07) 1303-1310
• 4 Marik PE, Farkas JD. The changing paradigm of sepsis: early diagnosis, early antibiotics, early pressors, and early adjuvant treatment. Crit Care Med 2018; 46 (10) 1690-1692
• 5 Filbin MR, Lynch J, Gillingham TD. et al. Presenting symptoms independently predict mortality in septic shock: Importance of a previously unmeasured confounder. Crit Care Med 2018; 46 (10) 1592-1599
• 6 Henry KE, Hager DN, Pronovost PJ, Saria S. A targeted real-time early warning score (TREWScore) for septic shock. Sci Transl Med 2015; 7 (299) 299ra122
• 7 Thiel SW, Rosini JM, Shannon W, Doherty JA, Micek ST, Kollef MH. Early prediction of septic shock in hospitalized patients. J Hosp Med 2010; 5 (01) 19-25
• 8 Bhattacharjee P, Edelson DP, Churpek MM. Identifying patients with sepsis on the hospital wards. Chest 2017; 151 (04) 898-907
• 9 Calvert JS, Price DA, Chettipally UK. et al. A computational approach to early sepsis detection. Comput Biol Med 2016; 74: 69-73
• 10 Kam HJ, Kim HY. Learning representations for the early detection of sepsis with deep neural networks. Comput Biol Med 2017; 89: 248-255
• 11 Lauritsen SM, Kalør ME, Kongsgaard EL. et al. Early detection of sepsis utilizing deep learning on electronic health record event sequences. Artif Intell Med 2020; 104: 101820
• 12 Futoma J, Hariharan S, Heller K. et al. An Improved Multi-Output Gaussian Process RNN with Real-Time Validation for Early Sepsis Detection. PMLR; 2017. :243–254
• 13 Desautels T, Calvert J, Hoffman J. et al. Prediction of sepsis in the intensive care unit with minimal electronic health record data: a machine learning approach. JMIR Med Inform 2016; 4 (03) e28
• 14 Nemati S, Holder A, Razmi F, Stanley MD, Clifford GD, Buchman TG. An interpretable machine learning model for accurate prediction of sepsis in the ICU. Crit Care Med 2018; 46 (04) 547-553
• 15 Moor M, Horn M, Rieck B, Roqueiro D, Borgwardt K. Early recognition of sepsis with gaussian process temporal convolutional networks and dynamic time warping. machine learning for healthcare conference. PMLR 2019; 106: 2-26
• 16 Barton C, Chettipally U, Zhou Y. et al. Evaluation of a machine learning algorithm for up to 48-hour advance prediction of sepsis using six vital signs. Comput Biol Med 2019; 109: 79-84
• 17 Aşuroğlu T, Oğul H. A deep learning approach for sepsis monitoring via severity score estimation. Comput Methods Programs Biomed 2021; 198: 105816
• 18 Rosnati M, Fortuin V. MGP-AttTCN: an interpretable machine learning model for the prediction of sepsis. PLoS One 2021; 16 (05) e0251248
• 19 Persson I, Östling A, Arlbrandt M, Söderberg J, Becedas D. A machine learning sepsis prediction algorithm for intended intensive care unit use (NAVOY sepsis): proof-of-concept study. JMIR Form Res 2021; 5 (09) e28000
• 20 Teng AK, Wilcox AB. A review of predictive analytics solutions for sepsis patients. Appl Clin Inform 2020; 11 (03) 387-398
• 21 Johnson AE, Pollard TJ, Shen L. et al. MIMIC-III, a freely accessible critical care database. Sci Data 2016; 3: 160035
• 22 Lloyd JK, Ahrens EA, Clark D, Dachenhaus T, Nuss KE. Automating a manual sepsis screening tool in a pediatric emergency department. Appl Clin Inform 2018; 9 (04) 803-808
• 23 Gibbs KD, Shi Y, Sanders N. et al. Evaluation of a sepsis alert in the pediatric acute care setting. Appl Clin Inform 2021; 12 (03) 469-478
• 24 Dewan M, Vidrine R, Zackoff M. et al. Design, implementation, and validation of a pediatric ICU sepsis prediction tool as clinical decision support. Appl Clin Inform 2020; 11 (02) 218-225
• 25 Smielewski P, Czosnyka M, Steiner L, Belestri M, Piechnik S, Pickard JD. ICM+: software for on-line analysis of bedside monitoring data after severe head trauma. Acta Neurochir Suppl (Wien) 2005; 95: 43-49
• 26 Meyer MA, Levine WC, Egan MT. et al. A computerized perioperative data integration and display system. Int J Comput Ass Rad 2007; 2: 191-202
• 27 Goldstein B, McNames J, McDonald BA. et al. Physiologic data acquisition system and database for the study of disease dynamics in the intensive care unit. Crit Care Med 2003; 31 (02) 433-441
• 28 Gjermundrod H, Papa M, Zeinalipour-Yazti D. et al. Intensive Care Window: A Multi-Modal Monitoring Tool for Intensive Care Research and Practice. Paper presented at: Twentieth IEEE International Symposium on Computer-based Medical Systems; June 20, 2007; Maribor, Slovenia
• 29 Sun Y, Guo F, Kaffashi F, Jacono FJ, DeGeorgia M, Loparo KA. INSMA: An integrated system for multimodal data acquisition and analysis in the intensive care unit. J Biomed Inform 2020; 106: 103434
• 30 Shimabukuro DW, Barton CW, Feldman MD, Mataraso SJ, Das R. Effect of a machine learning-based severe sepsis prediction algorithm on patient survival and hospital length of stay: a randomised clinical trial. BMJ Open Respir Res 2017; 4 (01) e000234
• 31 Burdick H, Pino E, Gabel-Comeau D. et al. Effect of a sepsis prediction algorithm on patient mortality, length of stay and readmission: a prospective multicentre clinical outcomes evaluation of real-world patient data from US hospitals. BMJ Health Care Inform 2020; 27 (01) e100109
• 32 McCoy A, Das R. Reducing patient mortality, length of stay and readmissions through machine learning-based sepsis prediction in the emergency department, intensive care unit and hospital floor units. BMJ Open Qual 2017; 6 (02) e000158
• 33 Chen Q, Li R, Lin C. et al. Transferability and interpretability of the sepsis prediction models in the intensive care unit. medRxiv 2021;
• 34 Lundberg S, Lee SI. A Unified approach to interpreting model predictions. Paper presented at: Proceedings of the 31st international conference on neural information processing systems; 2017;4768–4777
• 35 Desai MD, Tootooni MS, Bobay KL. Can prehospital data improve early identification of sepsis in emergency department? An integrative review of machine learning approaches. Appl Clin Inform 2022; 13 (01) 189-202

Publication History

Received: 20 July 2022

Accepted: 28 November 2022

Accepted Manuscript online:
30 November 2022

Article published online:
25 January 2023