Chronic Obstructive Pulmonary Diseases:Journal of the COPD Foundation

Running Head: Machine Learning-Based Models and COPD AECOPDs

Funding Support: The authors did not receive support from any organization for the submitted work.

Date of Acceptance: July 1, 2024 | Publication Online Date: July 3, 2024

Abbreviations: AE=acute exacerbation; AECOPD=acute exacerbation of chronic obstructive pulmonary disease; AI=artificial intelligence; AUC=area under the receiver operating characteristic curve; BMI=body mass index; BNP=brain natriuretic peptide; BP=blood pressure; c-index=concordance index; CAT=COPD Assessment Test; COPD=chronic obstructive pulmonary disease; EMR=electronic medical record; FEV1=forced expiratory volume in 1 second; GBM=gradient boosting machine; GOLD=Global initiative for chronic Obstructive Lung Disease; ICD-10=International Classification of Diseases, Tenth Revision; ICD-10-CM=International Classification of Diseases, Tenth Revision, Clinical Modification; ICU=intensive care unit; IgA=immunoglobin A; KNN=k-nearest neighbor; LASSO=least absolute shrinkage and selection operator; LR=logistic regression; ML=machine learning; mMRC=modified Medical Research Council; NT-proBNP=N-terminal prohormone in brain natriuretic peptide; NPV=negative predictive value; PPV=positive predictive value; RF=random forest; RFE=recursive feature elimination; ROC=receiver operating characteristic; SHAP=SHapley Additive exPlanations; SVM=support vector machine; XGB=extreme gradient boosting

Citation: Jia Q, Chen Y, Zen Q, et al. Development and validation of machine learning-based models for prediction of intensive care unit admission and in-hospital mortality in patients with acute exacerbations of chronic obstructive pulmonary disease. Chronic Obstr Pulm Dis. 2024; 11(5): 460-471. doi: http://doi.org/10.15326/jcopdf.2023.0446

Online Supplemental Material: Read Online Supplemental Material (557KB)

Introduction

As a chronic condition, chronic obstructive pulmonary disease (COPD) shows typical features of chronic airway obstruction, chronic bronchitis, and emphysema. COPD patients usually have progressively and irreversibly declining lung function.1 As suggested by the World Health Organization statistics, COPD may rank third among factors inducing death by 2030 worldwide.2 Acute exacerbations of chronic obstructive pulmonary disease (AECOPDs) refer to sudden airway functional deterioration or respiratory symptom aggravation among COPD cases,3 and is tightly associated with COPD occurrence and progression. The onset of an AECOPD is a major factor inducing hospitalization and mortality of COPD cases. As previously reported, hospitalized AECOPD recurrence usually takes place within a short period, and can exacerbate the COPD course even after treatment, eventually increasing the hospitalization and mortality rates.4,5 It is the key factor leading to declining lung function and health status. Therefore, it is important to explore the influences on AECOPD prognosis to improve treatment for COPD cases.

It is suggested that AECOPD prognosis is related to factors like traditional laboratory and clinical parameters.6 COPD progression is the key factor resulting in greater AECOPD severity and occurrence frequency.7 Factors, like >65 years in age, chronic mucus hypersecretion, obvious comorbidities, and mild airflow obstruction with forced expiratory volume in 1 second (FEV1) < 50% of predicted, are associated with higher hospital admission, readmission, and disease exacerbation risks.8-10 Pneumonia and dyspnea severity have been identified as predicting factors for early readmission and in-hospital mortality of AECOPD.11 Chronic comorbidities that are not related to lung involvement, such as diabetes mellitus, arterial hypertension, and ischemic heart disease, etc., and the Charlson index (2 or > 2 comorbidities other than COPD) are related to a poor, short-term prognosis.12 Biomarkers can also predict the prognosis of AECOPD patients. Leukocytosis in the stable phase, elevated levels of C-reactive protein, increased stable-phase fibrinogen level, and acute-phase D-dimer have been found to be involved in the early relapse of an AECOPD.6 Putcha et al found that subnormal immunoglobin A (IgA) content in serum was related to a higher acute exacerbation risk, which supported that the mild impairment of IgA levels was the contributor for COPD incidence. Besides, the decreased serum IgA was dose-dependently related to numerous exacerbations in patients whose serum IgA levels were within the lowest decile, which supported the relation of serum IgA level with exacerbation incidence.13 In addition, some radiographic features (such as elevated chest computed tomography-derived muscle and bone measures capture markers) on chest imaging examinations are suggested to be the alternative markers for comorbidities among COPD patients.14,15 Meanwhile, it is worth mentioning that inadequate antibiotic treatment can be regarded as a related factor to long-term outcomes in AECOPDs.6 To reduce the risk of a poor prognosis in AECOPD, a comprehensive multivariate analysis of prognosis is needed. Machine learning (ML) has been widely applied to disease prognosis and prediction because it can estimate unknown dependencies through the given dataset and use this to predict new output.16 The application of health care administrative data or electronic medical records (EMRs) has provided real-world data for ML, promoting the potential for ML in predicting the prognosis of diseases affected by multiple factors. Recently, ML has been applied to predicting and analyzing AECOPDs with more precision and better performance.17,18

To our knowledge, there has been no study applying ML methods to explore the multivariate impact on the severity and survival outcome of AECOPD patients. This work focused on using ML models for constructing effective prediction models to identify the severity and risk of in-hospital mortality among AECOPD patients.

Materials and Methods

Study Population

This work gained approval from the Institutional Review Board of the University of the Chinese Academy of Sciences Shenzhen Hospital. Protocols were established to ensure ethical compliance. Before collecting data, informed consents were obtained from every included patient for using the data in later health-related studies. Methods in this work were conducted strictly following relevant laws and regulations. In order to preserve and uphold the privacy and confidentiality of all patients, we carried out an extensive process to remove any sensitive or personally identifiable information before commencing with our analysis, including name, address, and contact details.

The present study began by selecting the initial study population from the COPD Pay-for-Performance Program database, encompassing those with COPD from 2012 to 2018. This database was based on patient information from the Department of Pulmonary and Critical Care Medicine, University of Chinese Academy of Sciences Shenzhen Hospital. All patients admitted for acute COPD were included in this database. The purpose of the Pay-for-Performance Program was to rationalize quality improvement spending on the care quality and health insurance costs in COPD cases. The preliminary study population comprised over 4900 patients, serving as a foundation for our research. Focusing on our goal in the present work, which aims to establish the early risk evaluation tool for AECOPD inpatients, this study narrowed our population by identifying 1954 AECOPD patients who were discharged as our intermediate study population. In order to further solve the heterogeneity of the study population, we limited the study to patients admitted with acute COPD as the primary symptom (the principal admission diagnosis was AECOPD).

Subsequently, the distinction between severe and non-severe patients was introduced by evaluating the occurrence of ICU admission within the intermediate population. Our final sample size included 322 hospital records procured for AECOPD patients aged above 18 years, chosen through the International Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM) code for COPD (J44.100, J44.101) in the primary diagnosis field.19 To achieve methodological coherence, patients with ICU admissions were categorized into severe, whereas patients who did not need ICU admission were labeled as non-severe. Based on this classification, we calculated the severe AECOPD patient proportion, and our results indicate that 36.6% (118/322) of the patient sample were classified as severe AECOPD patients. Whereas the rest, 63.4% (204/322), were nonsevere patients.

The methodology involving the analysis of how basic indicators, inflammation, and comorbidities affected frequent severe acute exacerbations (AEs) of COPD patients was used. For ensuring the impartial and robust AE risk evaluation in COPD cases, all cases were classified into a training or test cohort according to the respective admission dates before or after December 31, 2018, respectively. Clinical data of patients in the training cohort were employed for developing the prediction models, while those in the test cohort were applied to evaluating the model performance. There were 225 cases in the training cohort, which included 83 severe and 142 nonsevere ones. Meanwhile, there were 97 cases in the test cohort, including 35 severe and 62 nonsevere ones. To eliminate bias in the analysis, we excluded samples with numerous missing values. Table 1 displays the distribution of the severe and nonsevere groups among AECOPD patients. Table 1 indicates that among 322 cases enrolled in this work, a total of 181 patients had a history of smoking, of which 82 patients were categorized as belonging to the severe group. In comparison, 99 patients belonged to the nonsevere group. Conversely, 141 patients had no history of smoking, with 36 and 105 of these patients categorized into severe and nonsevere groups, respectively. Moreover, out of the total population, 191 patients were male, with 77 patients belonging to the severe group, and 114 to the nonsevere group. On the other hand, 131 patients were female, with 41 patients classified as belonging to the severe group while 90 patients were in the nonsevere group.

JCOPDF-2024-0446-Table1

Outcome

Our primary goal was developing the prediction model for identifying ICU admission within AECOPD cases that were admitted into the hospital. Meanwhile, identification of in-hospital death presence was considered a secondary outcome, which was defined as deaths resulting from adverse events that are related to emergency department visits or admission with an International Classification of Diseases, Tenth Revisions (ICD-10) code of AECOPD (J43.x–44.x, except for J430).17,20

Feature Engineering

The study extracted data from the EMR database, a data and database repository collected based on diverse EMR systems. This dataset comprised 90 features that were obtained from the clinical records of outpatients (the list of 90 features is shown in Supplementary Table 1 of the online supplement). The data were collected within 6 months preceding the patient's most recent visit before their initial admission due to AECOPD. The features included different perspectives, such as demographic data like age, gender, and body mass index (BMI) and clinical characteristics such as COPD Assessment Test (CAT) scores, postbronchodilator test results, COPD Global initiative for chronic Obstructive Lung Disease (GOLD) scores, modified Medical Research Council (mMRC) dyspnea scores, vital signs, respiratory symptoms, laboratory results, comorbidities, and medication usage. To ensure feature variability and model accuracy, we eliminated features whose prevalence was <5% out of analysis. As a result, a total of 32 features were excluded.21 The best feature subset was chosen using recursive feature elimination (RFE) for predicting the AECOPD incidence. There were altogether 38 features chosen by RFE through 10-fold cross-validation conducted in 5 replicates. Collinearity was assessed using the variance inflation factor (>2), which identified and excluded the following features: postbronchodilator FEV1/forced vital capacity ratio, hemoglobin, eosinophil-to-lymphocyte ratio, and COPD GOLD score. Additionally, expert COPD physicians were consulted to finalize the list of 34 included features (gender, age, hypertension, diabetes mellitus, chronic renal dysfunction, pulmonary heart disease, hypothyroidism, coronary heart disease, smoking status, BMI, body temperature, respiratory rate, pulse rate, diastolic blood pressure, systolic blood pressure, mechanical ventilation, hematocrit, lactate dehydrogenase, platelet count, white blood cells, neutrophil ratio, partial pressure of carbon dioxide, partial pressure of oxygen, oxygen saturation, pH, total bilirubin, D-dimer, fibrinogen, albumin, creatinine, brain natriuretic peptide [BNP], malignant tumor, sepsis, mMRC dyspnea scores).

Statistical Analysis and Machine Learning Algorithms

First, we abandoned severe data missing variables, accounting for more than 20% of the total variables data. For the variables with missed data less than 20% of the total data, a multiple imputations approach was utilized for imputing missing data.22 This present study reports categorical variables as proportions and corresponding counts, while continuous variables are indicated by medians and their interquartile ranges. To compare the categorical variables, we utilized the Chi-square test, and compared continuous variables by employing a nonparametric test.

Figure 1 displays a framework utilized in creating prediction models for ICU admission and hospital death in patients with an AECOPD, encompassing 4 primary steps: data preprocessing, feature engineering, ML model establishment, as well as model training. Seven distinct ML algorithms were utilized, including logistic regression, support vector machine (SVM), most minor absolute shrinkage and selection operator, random forest (RF), k-nearest neighbor, extreme gradient boosting (XGB), and gradient boosting machine (GBM). We employed an exhaustive grid search algorithm to be the hyperparameter tuning approach. We executed a 5-fold cross-validation for training the subset to identify the optimal hyperparameter combination. Hyperparameters resulting in the greatest area under the receiver operating characteristic curve (AUC) of validation set in every ML model were selected. We employed 4 kernel functions, namely polynomial, linear, radial, and sigmoid, to be basic functions in constructing the SVM model. The hyperparameters including gamma, cost, epsilon, and degree were adopted for tuning the SVM model for each of the kernels, as mentioned above. We obtained altogether 182,000 hyperparameter combinations for the SVM model. Moreover, we utilized ntree, mtry, and nodesize as hyperparameters for the RF model and conducted altogether 65,322 hyperparameter combinations.

JCOPDF-2024-0446-Figure1

In creating the XGB model, 163,180 hyperparameters were considered, out of which those optimum hyperparameters consisted of gamma, eta, nrounds, and maximal depth of a tree. For developing the GBM model, similar hyperparameters were explored, such as interaction.depth, shrinkage, bag.fraction, and n.minobsinnode, with the objective of identifying the hyperparameters that would provide the greatest AUC of the validation set. During the development of these ML models, one-hot encoding was employed to handle categorical data, followed by standardization of every continuous feature prior to analysis. Upon finalizing models with the training cohort, this study proceeded to assess their predictive performance by measuring AUC as well as 5 assessment metrics: sensitivity, specificity, positive/negative predicted value (PPV/NPV), accuracy, and F1 score, with respect to test set. Model discrimination was assessed using the concordance index (c-index). We utilized Youden’s index to determine the threshold that optimally classifies ICU admissions. Given that our primary goal is to predict ICU admissions for assisting patients, we prioritized the increased F1 scores and prediction accuracy while evaluating the models. The F1 score is a performance metric that takes into account both sensitivity and PPV and is scaled between 0 and 1. The F1 score can be calculated as follows, F1 = 2 * (precision x recall) / (precision + recall) . To further assess the clinical utility of our models, we performed a decision curve analysis. We also evaluated calibration, the measure of agreement between predicted levels, and real measurements of ICU admission in AECOPD patients.

Descriptive analysis was conducted with SPSS, while ML models were developed with R software (version 3.6.2). Statistically significant results were defined as those with p<0.05 (2-tailed).

Results

Demographics

The entire program was described, consisting of feature selection, prediction model establishment, as well as performance assessment (Figure 1). There were 322 cases enrolled in the present work, including 225 and 97 in training and test sets, respectively. Baseline features in patients from the severe and nonsevere groups in the training and test sets were compared (Table 1). The history of pulmonary heart disease and smoking history were significantly different in the severe group compared with the nonsevere group in the training cohort (P<0.05).

Prediction Models for Intensive Care Unit Admission

There were a total of 7 prediction models regarding different ML classifiers constructed with AUCs 0.827–0.973 in the training group and 0.648–0.828 in the testing group (Table 2). Calibration plots and ROC curves were used for visualizing the 2 cohorts (Supplementary Figure S1 in the online supplement). The prediction model on the RF classifier outperformed others with regard to the AUC of the test set of 7 ML-based models, and its AUC, C-index, accuracy, sensitivity, specificity, PPV, NPV, and F1 score were 0.973, 0.973, 92.00%, 93.98%, 90.85%, 85.71%, 96.27%, and 0.897 respectively for the training set; whereas the values for the test set were 0.828, 0.828, 77.89%, 80.00%, 64.52%, 68.84%, 94.07%, and 0.738, respectively (Supplementary Figure S2 in the online supplement, Table 2). As revealed by the calibration curve, actual observations were consistent with RF-predicted results which indicated great calibration capacity.

JCOPDF-2024-0446-Table2

Prediction Models for In-Hospital Death

Baseline features between cases with and without hospital death in the 2 datasets were compared (Table 3). Altogether 23 patients (10.2%) in the training cohort and 10 patients (10.3%) in the test cohort reported in-hospital deaths. The history of pulmonary heart disease, history of bronchiectasis, and smoking history were significantly different between the living group and the deceased group in the training cohort (P<0.05).

JCOPDF-2024-0446-Table3

Altogether 7 prediction models regarding different ML classifiers for in-hospital mortality were constructed, and AUCs were 0.957–0.993 and 0.547–0.705 for training and test sets, respectively (Table 4). Calibration plots and ROC curves were adopted for visualization in the 2 cohorts (Supplementary Figure S3 in the online supplement). The prediction model on the RF classifier outperformed others, with regard to the AUC of the test set of 7 ML-based models, and its AUC, C-index, accuracy, sensitivity, specificity, PPV, NPV, and F1 score were 0.982, 0.982, 94.67%, 91.30%, 95.05%, 67.74%, 98.97%, and 0.778, respectively, for the training set. Those values for the test set were 0.705, 0.705, 64.95%, 60.00%, 65.52%, 16.67%, 93.44%, and 0.261, respectively (Supplementary Figure S3 in the online supplement, Table 4). As revealed by the calibration curve, actual observations were consistent with RF-predicted results which indicated great calibration capacity. For comparison, the performance of ICU admission in predicting in-hospital mortality was evaluated, and the AUC, C-index, accuracy, sensitivity, specificity, PPV, NPV, and F1 score were 0.969, 0.969, 96.00%, 86.96%, 97.03%, 76.92%, 98.49%, and 0.816 respectively for the training set; whereas the values for the test set were 0.546, 0.546, 65.98%, 30.00%, 70.11%, 10.34%, 89.71%, and 0.154, respectively (Supplementary Figure S4 in the online supplement, Table 4).

JCOPDF-2024-0446-Table4

We generated an ML signature based on the RF-based model, which was combined with ICU admission to develop an integrated nomogram model for predicting in-hospital death (Figure 2A). The AUC, C-index, accuracy, sensitivity, specificity, PPV, NPV, and F1 score of this integrated model were 0.992, 0.992, 96.00%, 95.65%, 96.04%, 73.33%, 99.49%, and 0.830, respectively for the training set; while the values for the test set were 0.754, 0.754, 71.13%, 60.00%, 72.41%, 20.00%, 94.03%, and 0.300, respectively (Table 4). This integrated model exhibited excellent classification performance by ROC curves as well as precision-recall plots (Figure 2B and 2C) and had uniform calibration ability (Figure 2D) and high clinical benefit (Figure 2E).

JCOPDF-2024-0446-Figure2

Discussion

The present investigation aimed to construct and validate an intelligible ML-supported risk evaluation tool to anticipate the likelihood of ICU admission and in-hospital mortality of AECOPD patients. According to our results, ML models exhibited superb discrimination performance in forecasting ICU admission, since the AUC was >0.80. These findings indicate that ML has significant potential to be implemented clinically as an estimator of ICU admission and in-hospital death risks in AECOPD patients. Of those ML models used, the RF method demonstrated the greatest prediction ability, as a result, it was used for creating an explicable ML-based exacerbation risk assessment approach.

In this work, the GBM model was most accurate in predicting severe AECOPDs (ICU admissions because of an AECOPD and in-hospital mortality), and its AUC value reached 0.83. Like the Hussain et al model,23 the ML-based model constructed in this study could precisely forecast severe AECOPDs (ICU admission because of an AECOPD and in-hospital mortality) without considering the risk factor of exacerbation history of the patient. Nonetheless, it may be a challenge to compare our findings to those of Hussain et al, because their study did not provide an AECOPD definition or specific study population. Consequently, using ML-based models, in particular GBM models, is the precise and potential way to predict severe AECOPDs (ICU admission because of an AECOPD and in-hospital mortality) with no consideration of the exclusive risk factor of exacerbation history. Such ML-based models may be potentially utilized as the clinical decision-making approaches, which can identify high-risk patients for AECOPDs that probably gain benefits from specialist referral and treatment adjustment. Moreover, the GBM model did not use exacerbation history as one of its features, but it attained high accuracy comparable to previous GBM models where exacerbation history is used as a feature. Consequently, our prediction model appears to be suitable for assessing the risk of patients with no prior exacerbations, including those diagnosed with COPD for the first time, or those with incomplete medical records.

In the outpatient context of COPD care, the primary objectives are to prevent AEs and mitigate unwanted outcomes. Despite being a dependable predictor of future exacerbations, a history of AECOPDs is insufficient as a definitive basis for identifying trustworthy clinical features that can inform treatment decisions and prevention strategies for AECOPDs.24 Additionally, the discrimination performance of the prediction model that relies only on AECOPD history is lower than that of the ML-based model.18 To take an example, Tavakoli et al18 leveraged ML for developing the model that could identify high-risk patients for AECOPD-related hospitalization. According to their results, the GBM model outperformed other prediction models relying only on AECOPD history as a feature. Specifically, the AUC of the GBM model was 0.82 compared to the AUC of 0.68 for the model exclusively considering AECOPD history.21,25 Assessing the risk of an initial event of AECOPD based solely on a patient’s history of the condition may be insufficient, as some medical records may not contain prior exacerbation information. To overcome these limitations, Hussain et al developed a prediction model using the GBM approach, which excluded any consideration of a patient’s AECOPD history. Remarkably, this model23 performed well in discrimination, as evidenced by the AUC value of 0.96. In this study, we formulated a framework that leveraged ML-based modeling to predict AECOPDs. We included different pertinent clinical features with real-world data for interpreting local population features.

Recently, one systematic review examined the existing AECOPD prediction models, which involved 27 models established using traditional statistical techniques, accounting for various patient data, symptoms, and lung function, together with COPD-related risk factors. These models demonstrated variable levels of performance, and AUC values were 0.58–0.78. In contrast to conventional statistical approaches designed for verifying certain hypotheses, ML provides an alternative approach to AECOPD prediction modeling that highlights performance optimization. Moreover, ML is constructed on the basis of a minimal number of assumptions regarding the data-generating system, thus, potentially improving model accuracy over traditional statistical methods.26 When assessing the risk of AECOPD, Wang et al27 carried out a comparative analysis of conventional logistic regression with ML algorithms, like RF, SVM, k-nearest neighbors, logistic regression, and naive Bayes algorithms. As a result, ML-based models were more accurate than traditional statistical approaches.27 Likewise, as suggested by Tavakoli et al,18 the GBM model was more accurate in predicting AECOPDs than logistic regression, RF, as well as neural network models.18 This work verified the above results and supported that the RF model showed higher discrimination performance in AECOPD (ICU admission because of an AECOPD and in-hospital mortality) prediction. Consequently, ML-based models, in particular RF models, perform well in AECOPD (ICU admission because of an AECOPD and in-hospital mortality) prediction.

It is an important step to select the best features to enhance the ML model performance. Therefore, Hussain et al23 and Tavakoli et al18 constructed the GBM models to incorporate related patient features, such as demographic data, vital signs, symptoms, laboratory data, questionnaire responses, hospitalizations, medication dispensation records, and outpatient services. These models performed well, and AUCs were >0.80, indicating their excellent prediction performance. In this study, we added different clinical parameters in developing the RF model, such as demographic data, symptoms, vital signs, comorbidities, prescribed medications, CAT scores, and laboratory data. Such features were comprehensive relative to those utilized in previous models and had comparable performance. Likewise, a previous model used the RF model according to hundreds of single nucleotide polymorphisms for predicting asthma exacerbation.28 The integration of genomic data in ML models can more accurately predict AECOPDs.

AECOPD is heterogeneous and complicated, suggesting that it involves different non-linearly and dynamically interacting components. Such interactions cannot be observed in every case or in one specific case all the time.29 Such dynamic heterogeneity and complexity suggest that it is important to adopt the precision medicine method for optimizing AECOPD evaluation, management, and outcomes.30-32 As ML models have been more and more incorporated into precision medicine, they shed more light on the relevant mechanisms and trajectories of chronic disorders, including AECOPDs.33 In many studies, using ML models in predicting AECOPDs can achieve favorable results. However, artificial intelligence (AI)[BH2] has a black-box nature, which hinders its clinical application. When there is no interpretable AI model, clinicians have few data to convey to their patients, which may lead to reduced patient contentment and trust.34 SHapley Additive exPlanations (SHAP) [BH3] accounts for the game-theoretic technology put forward by Lundberg and Lee,35 which focused on elucidating the contributions of features to output changes in ML models. In addition, SHAP values can offer the locally precise and uniform attribute values for every feature incorporated into this prediction model, which reflects the importance. By visualizing data using SHAP, users can more readily comprehend intricate black-box integration models. SHAP methods have been used recently in diverse clinical contexts, such as coronary artery calcification or venous thrombosis among osteoarthritis patients.36,37 Additionally, local explanation results may be presented as feature changes during prediction, from basic values to model outputs, thereby facilitating to visually present the estimated results for clinicians.

Our predictive model is based on 34 characteristic variables that include clinical history, vital signs, and auxiliary test information during hospitalization. Long-term COPD is often accompanied by pulmonary heart disease, and in general, the occurrence of pulmonary heart disease indicates the insufficient compensatory capacity for cardiac function. Therefore, once patients with pulmonary heart disease develop an AECOPD, the risk of admission to the ICU is often significantly increased. One study reported that the N-terminal prohormone in BNP (NT-proBNP) can serve as the biomarker to diagnose left ventricular systolic dysfunction among AECOPD patients.38 Our study found that BNP was also a significant predictor for ICU admission due to an AECOPD and in-hospital mortality during hospitalization, which was supported by the results of previous studies. In addition, oxygen saturation and the use of mechanical ventilation are essential markers in patients with an AECOPD. Intervention with mechanical ventilation often reflected severe respiratory failure in patients with an AECOPD. For such patients, it frequently predicted a poor prognosis during hospitalization. Therefore, our study emphasizes the critical effect of heart function and lung function on prognosis prediction of AECOPD patients. Our results have significant clinical value for front-line clinicians in the assessment of AECOPD patients.

Certain limitations in the present work warrant consideration. First, the data used were derived from a single health care system, and thus, the generalizability of our findings to those who receive care in additional health care institutions may be limited. Therefore, for optimizing the prediction accuracy, multicenter external validation should be conducted. Additionally, documentation habits and accuracy are notable sources of residual confounding, and this may introduce some bias to our results. Secondly, our model was developed using exclusively structured data, and further exploration should be performed for including multidimensional data, such as environmental factors, unstructured data (like images), patient activities, habits, or other relevant factors for improving prediction model accuracy. Third, it is essential to note that our study utilized standard ML techniques exclusively to construct a prediction model. Recently, employing deep learning techniques is found to be beneficial in medical modeling. Therefore, future research endeavors should establish the deep learning model to predict a first-time AECOPD. Fourth, seasonal alteration of AECOPD prevalence has been the widely recognized phenomenon, with fast temperature change being a contributing factor, as evidenced by 2 Taiwanese studies.39,40 AECOPD patients may exhibit heightened sensitivity to temperature changes in comparison to the general healthy population, with short-time exposures to these temperature changes being responsible for exacerbations. Nonetheless, this present work cannot obtain real-time data regarding seasonal temperature changes, and this is the notable limitation in the research. Finally, as many variables were incorporated for analysis and our sample size was insufficient, there may be an overfitting of the ML model. Therefore, while explaining the accuracy of our prediction model, it should also be noted that our model might be associated with certain bias.

Conclusion

The findings of our investigation indicate that the RF-based model effectively assessed the probability of ICU admission and in-hospital mortality in patients suffering from an AECOPD. Furthermore, the utilization of ML-based models allowed for clear and precise explanations of personalized risk predictions that could help clinicians comprehend the significance of critical model features as well as decision-making process. Such approaches may prove to be instrumental in optimizing individualized therapeutic strategies for AECOPD patients by incorporating prognostic risks into clinical decision-making. Ultimately, further implementation of ML methods in clinical practice has the potential to improve patient outcomes significantly through tailored and informed treatments.

Acknowledgements

Author contributions: QJ, YC, QZ, SC, SL, TW, and XY made substantial contributions to conception and design and revised the manuscript critically for important intellectual content. TW revised the manuscript and gave final approval for the version to be published. All authors read and approved the final manuscript.

Data sharing statement: The datasets used and analyzed during the current study are available from the corresponding author upon reasonable request.

Declaration of Interests

The authors have no competing interests to declare that are relevant to the content of this article.

Online Supplement

Click here to view the Online Supplement.

1. Vogelmeier CF, Criner GJ, Martinez FJ, et al. Global strategy for the diagnosis, management, and prevention of chronic obstructive lung disease 2017 report. GOLD executive summary. Am J Respir Crit Care Med. 2017;195(5):557-582. https://doi.org/10.1164/rccm.201701-0218PP

2. GBD Chronic Respiratory Disease Collaborators. Prevalence and attributable health burden of chronic respiratory diseases, 1990-2017: a systematic analysis for the Global Burden of Disease Study 2017. Lancet Respir Med. 2020;8(6):585-596. https://doi.org/10.1016/S2213-2600(20)30105-3

3. Macintyre N, Huang YC. Acute exacerbations and respiratory failure in chronic obstructive pulmonary disease. Proc Am Thorac Soc. 2008;5(4):530-535. https://doi.org/10.1513/pats.200707-088ET

4. Antoniu SA, Carone M. Hospitalizations for chronic obstructive pulmonary disease exacerbations and their impact on disease and subsequent morbidity and mortality. Expert Rev Pharmacoecon Outcomes Res. 2013;13(2):187-189. https://doi.org/10.1586/erp.13.9

5. Kim S, Emerman CL, Cydulka RK, et al. Prospective multicenter study of relapse following emergency department treatment of COPD exacerbation. Chest. 2004;125(2):473-481. https://doi.org/10.1378/chest.125.2.473

6. Mantero M, Rogliani P, Di Pasquale M, et al. Acute exacerbations of COPD: risk factors for failure and relapse. Int J Chron Obstruct Pulmon Dis. 2017;12:2687-2693. https://doi.org/10.2147/COPD.S145253

7. Hurst JR, Vestbo J, Anzueto A, et al. Susceptibility to exacerbation in chronic obstructive pulmonary disease. N Engl J Med. 2010;363(12):1128-1138. https://doi.org/10.1056/NEJMoa0909883

8. Miravitlles M, Guerrero T, Mayordomo C, Sánchez-Agudo L, Nicolau F, Segú JL. Factors associated with increased risk of exacerbation and hospital admission in a cohort of ambulatory COPD patients: a multiple logistic regression analysis. Respiration. 2000;67(5):495-501. https://doi.org/10.1159/000067462

9. Anzueto A, Miravitlles M, Ewig S, Legnani D, Heldner S, Stauch K. Identifying patients at risk of late recovery (≥ 8 days) from acute exacerbation of chronic bronchitis and COPD. Respi. Med. 2012;106(9):1258-1267. https://doi.org/10.1016/j.rmed.2012.06.002

10. Cao Z, Ong K C, Eng P, Tan WC, Ng TP. Frequent hospital readmissions for acute exacerbation of COPD and their associated factors. Respirology. 2006;11(2):188-195. https://doi.org/10.1111/j.1440-1843.2006.00819.x

11. Steer J, Norman EM, Afolabi OA, Gibson GJ, Bourke SC. Dyspnoea severity and pneumonia as predictors of in-hospital mortality and early readmission in acute exacerbations of COPD. Thorax. 2012;67(2): 117-121. https://doi.org/10.1136/thoraxjnl-2011-200332

12. Almagro P, Cabrera FJ, Diez J, et al. Comorbidities and short-term prognosis in patients hospitalized for acute exacerbation of COPD. the EPOC en Servicios de medicina interna (ESMI) study. Chest. 2012;142(5):1126-1133. https://doi.org/10.1378/chest.11-2413

13. Putcha N, Paul GG, Azar A, et al. Lower serum IgA is associated with COPD exacerbation risk in SPIROMICS. PLoS One. 2018;13(4):e0194924. https://doi.org/10.1371/journal.pone.0194924

14. Singhvi D, Bon J. CT imaging and comorbidities in COPD. Beyond lung cancer screening. Chest. 2021;159(1):147-153. https://doi.org/10.1016/j.chest.2020.08.2053

15. Wilson AC, Bon JM, Mason S, et al. Increased chest CT derived bone and muscle measures capture markers of improved morbidity and mortality in COPD. Respir Res. 2022;23:311. https://doi.org/10.1186/s12931-022-02237-w

16. Jiang F, Jiang Y, Zhi H, et al. Artificial intelligence in healthcare: past, present and future. Stroke Vasc Neurol. 2017;2(4):230-243. https://doi.org/10.1136/svn-2017-000101

17. Peng J, Chen C, Zhou M, Xie X, Zhou Y, Luo CH. A machine-learning approach to forecast aggravation risk in patients with acute exacerbation of chronic obstructive pulmonary disease with clinical indicators. Sci Rep. 2020;10: 3118. https://doi.org/10.1038/s41598-020-60042-1

18. Tavakoli H, Chen W, Sin DD, FitzGerald JM, Sadatsafavi M. Predicting severe chronic obstructive pulmonary disease exacerbations. Developing a population surveillance approach with administrative data. Ann Am Thorac Soc. 2020;17(9):1069-1076. https://doi.org/10.1513/AnnalsATS.202001-070OC

19. Tsai CL, Sobrino JA, Camargo CA, Jr. National study of emergency department visits for acute exacerbation of chronic obstructive pulmonary disease, 1993-2005. Acad Emerg Med. 2008;15(12):1275-1283. https://doi.org/10.1111/j.1553-2712.2008.00284.x

20. Lee J, Jung HM, Kim SK, et al. Factors associated with chronic obstructive pulmonary disease exacerbation, based on big data analysis. Sci Rep. 2019;9:6679. https://doi.org/10.1038/s41598-019-43167-w

21. Ogundimu EO, Altman DG, Collins GS. Adequate sample size for developing prediction models is not simply related to events per variable. J Clin Epidemiol. 2016;76:175-182. https://doi.org/10.1016/j.jclinepi.2016.02.031

22. Pedersen AB, Mikkelsen EM, Cronin-Fenton D, et al. Missing data and multiple imputation in clinical epidemiological research. Clin Epidemiol. 2017;9:157-166. https://doi.org/10.2147/CLEP.S129785

23. Hussain A, Choi HE, Kim HJ, Satyabrata A, Saqlain M, Kim HC. Forecast the exacerbation in patients of chronic obstructive pulmonary disease with clinical indicators using machine learning techniques. Diagnostics (Basel). 2021;11(5):829. https://doi.org/10.3390/diagnostics11050829

24. Sadatsafavi M, McCormack J, Petkau J, Lynd LD, Lee TY, Sin DD. Should the number of acute exacerbations in the previous year be used to guide treatments in COPD? Eur Respir J. 2021;57(2):2002122. https://doi.org/10.1183/13993003.02122-2020

25. Jiang L, Gershon AS. Using health administrative data to predict chronic obstructive pulmonary disease exacerbations. Ann Am Thorac Soc. 2020;17(9):1056-1057. https://doi.org/10.1513/AnnalsATS.202006-704ED

26. Bzdok D, Altman N, Krzywinski M. Statistics versus machine learning. Nat Methods. 2018;15:233-234. https://doi.org/10.1038/nmeth.4642

27. Wang C, Chen X, Du L, Zhan Q, Yang T, Fang Z. Comparison of machine learning algorithms for the identification of acute exacerbations in chronic obstructive pulmonary disease. Comput Methods Programs Biomed. 2020;188:105267. https://doi.org/10.1016/j.cmpb.2019.105267

28. Xu M, Tantisira KG, Wu A, et al. Genome Wide Association Study to predict severe asthma exacerbations in children using random forests classifiers. BMC Med Genet. 2011;12:90. https://doi.org/10.1186/1471-2350-12-90

29. Agustí A. The path to personalised medicine in COPD. Thorax. 2014;69(9):857-864. https://doi.org/10.1136/thoraxjnl-2014-205507

30. Agustí A, Macnee W. The COPD control panel: towards personalised medicine in COPD. Thorax. 2013;68(7):687-690. https://doi.org/10.1136/thoraxjnl-2012-202772

31. Agustí A, Antó JM, Auffray C, et al. Personalized respiratory medicine: exploring the horizon, addressing the issues. Summary of a BRN-AJRCCM workshop held in Barcelona on June 12, 2014. Am J Respir Crit Care Med. 2015;191(4):391-401. https://doi.org/10.1164/rccm.201410-1935PP

32. Agustí A, Bel E, Thomas M, et al. Treatable traits: toward precision medicine of chronic airway diseases. Eur Respir J. 2016;47(2):410-419. https://doi.org/10.1183/13993003.01359-2015

33. Subramanian M, Wojtusciszyn A, Favre L, et al. Precision medicine in the era of artificial intelligence: implications in chronic disease management. J Transl Med. 2020;18:472. https://doi.org/10.1186/s12967-020-02658-5

34. Kundu S. AI in medicine must be explainable. Nat Med. 2021;27:1328. https://doi.org/10.1038/s41591-021-01461-z

35. Lundberg S, Lee SI. A unified approach to interpreting model predictions. ArXiv. 2017;1705.07874v2. https://doi.org/10.48550/arXiv.1705.07874

36. Oh TR, Song SH, Choi HS, et al. Predictive model for high coronary artery calcium score in young patients with non-dialysis chronic kidney disease. J Pers Med. 2021;11(12):1372. https://doi.org/10.3390/jpm11121372

37. Lu C, Song J, Li H, et al. Predicting venous thrombosis in osteoarthritis using a machine learning algorithm: a population-based cohort study. J Pers Med. 2022;12(1):114. https://doi.org/10.3390/jpm12010114

38. Andrijevic I, Milutinov S, Lozanov Crvenkovic Z, et al. N-terminal prohormone of brain natriuretic peptide (NT-proBNP) as a diagnostic biomarker of left ventricular systolic dysfunction in patients with acute exacerbation of chronic obstructive pulmonary disease (AECOPD). Lung. 2018;196:583-590. https://doi.org/10.1007/s00408-018-0137-3

39. Tseng CM, Chen YT, Ou SM, et al. The effect of cold temperature on increased exacerbation of chronic obstructive pulmonary disease: a nationwide study. PloS One. 2013;8(3):e57066. https://doi.org/10.1371/journal.pone.0057066

40. Liang WM, Liu WP, Kuo HW. Diurnal temperature range and emergency room admissions for chronic obstructive pulmonary disease in Taiwan. Int J Biometeorol. 2009;53:17-23. https://doi.org/10.1007/s00484-008-0187-y

Images

  • Development and Validation of Machine Learning-Based Models for Prediction of Intensive Care Unit Admission and In-Hospital Mortality in Patients With Acute Exacerbations of Chronic Obstructive Pulmonary Disease
  • Development and Validation of Machine Learning-Based Models for Prediction of Intensive Care Unit Admission and In-Hospital Mortality in Patients With Acute Exacerbations of Chronic Obstructive Pulmonary Disease
  • Development and Validation of Machine Learning-Based Models for Prediction of Intensive Care Unit Admission and In-Hospital Mortality in Patients With Acute Exacerbations of Chronic Obstructive Pulmonary Disease
  • Development and Validation of Machine Learning-Based Models for Prediction of Intensive Care Unit Admission and In-Hospital Mortality in Patients With Acute Exacerbations of Chronic Obstructive Pulmonary Disease
  • Development and Validation of Machine Learning-Based Models for Prediction of Intensive Care Unit Admission and In-Hospital Mortality in Patients With Acute Exacerbations of Chronic Obstructive Pulmonary Disease
  • Development and Validation of Machine Learning-Based Models for Prediction of Intensive Care Unit Admission and In-Hospital Mortality in Patients With Acute Exacerbations of Chronic Obstructive Pulmonary Disease

Share This Article

E-mail this article to a friend