Content overview : According to the statistics of the International Diabetes Federation (IDF), the number of diabetic patients in China will account for 26% of the world in 2021
. Diabetic patients with long-term blood sugar out of control have a very high risk of complications such as coronary heart disease. Recently, researchers at Macheng People's Hospital in Hubei Province analyzed and compared various models, and used the machine learning model with the best performance to predict the mortality rate of 26.83% within one year in Chinese elderly patients with coronary heart disease complicated with diabetes or impaired glucose tolerance
.Key words : coronary heart disease survival prognosis gradient machine
This article was first published by HyperAI on the WeChat public platform~
According to the 2017 diabetes survey, there are as many as 78.13 million elderly people with diabetes in my country. Combined with a number of large-scale population studies, it has been found that there is a high degree of "co-morbidity" relationship between abnormal glucose metabolism and cardiovascular disease, that is, diabetic patients are often accompanied by complications such as coronary heart disease, and the latter has become a major cause of death in diabetic patients—approximately 75% of diabetic patients died of coronary heart disease. However, there are few relevant studies on the survival risk factors of patients with coronary heart disease complicated with diabetes mellitus or impaired glucose tolerance .
|Remarks : Impaired Glucose Tolerance (IGT) is a state of abnormal glucose metabolism that transitions from normal blood sugar to diabetes. It belongs to pre-diabetes and may further develop into diabetes (diabetes mellitus, DM).
In order to break through this situation, researchers from Macheng People's Hospital in Hubei Province, China, pioneered the comparison of logistic regression model (LR) and three machine learning models, and successfully predicted the risk of Chinese elderly patients with coronary heart disease complicated with diabetes mellitus or impaired glucose tolerance. Mortality within one year helps the medical community to identify patients at short-term risk of death in a timely manner, so as to give early warning and treatment.
The research has been published in the journal Cardiovascular Diabetology, titled "Machine learning-based models to predict one-year mortality among Chinese older patients with coronary artery disease combined with impaired glucose tolerance or diabetes mellitus" .
Figure 1: The research results have been published in "Cardiovascular Diabetology"
Paper address:
https://cardiab.biomedcentral.com/articles/10.1186/s12933-023-01854-z
experiment procedure
Dataset: Collect data from 451 elderly patients with coronary heart disease in 301 hospitals
This study analyzed 974 elderly patients with CHD admitted to the Department of Geriatric Cardiology, General Hospital of the Chinese People's Liberation Army between October 2007 and July 2011. Among them, the researchers further screened according to two conditions , namely:
- over the age of 60;
- Have impaired glucose tolerance (IGT) or diabetes mellitus (DM).
The final generated data set contained 451 patients, which were randomly divided into training set (n = 308) and test set (n = 143) according to the ratio of 7:3 . The training set is used to train and optimize the logistic regression model and 3 machine learning models, and the test set is used to test the prediction performance of the model. The data set screening process is as follows:
Figure 2: Flow chart outlining patient enrollment and study design
Model development: select 4 major models for horizontal comparison
In this study, the researchers developed a logistic regression model and three machine learning models, namely the gradient boosting machine model (GBM), the random forest model (RF) and the decision tree model (DT) to establish a predictive model , and according to the distribution Several indicators such as Brier Score, AUC (Area Under the Curve), calibration curve (calibration curve) and decision curve (decision curve) are used to evaluate the prediction effect.
Brier Score : A way to measure the difference between the probability predicted by the algorithm and the real result. Its value ranges from 0 to 1, with higher scores indicating worse predictions and less calibration.
Figure 3: Brill fraction calculation formula
AUC : refers to the area under the curve. In statistics and machine learning, AUC is commonly used to evaluate the performance of binary classification models. Its value ranges from 0 to 1, and the closer the value is to 1, the better the performance of the model; the closer the value is to 0.5, the weaker the predictive ability of the model.
Feature screening and parameter tuning for 3 machine learning models
At the same time, the researchers conducted feature screening and parameter tuning on the developed machine learning model . First, they used the LASSO (least absolute shrinkage and selection operator) algorithm combined with 10-fold cross-validation to screen out 7 features that were significantly correlated with one-year mortality as input to the model. These 7 features were hemoglobin, HDL-C, Albumin, serum creatinine, NT-proBNP, CHF, and statins. They then used 5-fold cross-validation and bootstrap to find the best combination of parameters to obtain the best area under the curve (AUC) through a randomized hyperparameter search.
Figure 4: Hyperparameter tuning process
A : Least absolute shrinkage and selection operator (LASSO) coefficient curve for all variables
B : Optimal parameter combination
C : Correlation coefficient between clinical features
From Figure 4, all correlation coefficients are lower than 0.80, indicating that there is no serious collinearity. A logistic regression model and 3 machine learning predictive models were trained with the above 7 clinical features . After model training and optimization, the optimal hyperparameters of each model are shown in the following table:
Table 1: Optimal hyperparameters for each model
Experimental results
From the overall performance of each model :
- The Brier score for the logistic regression model (LR) is 0.116
- Gradient Boosting Machine Model (GBM) has a Brier score of 0.114
- The decision tree model (DT) has a Brier score of 0.143
- Random Forest model (RF) has a Brier score of 0.126
The following figure shows the analysis results of each model :
Figure 5: AUC, calibration curve, decision curve, SHAP value of each model
D : Overall performance of each model
E : Calibration curve of each model
F : Decision curve of each model
G : Heat map of SHAP value
H : Feature importance analysis based on SHAP
According to Figure 5, the following conclusions can be drawn:
1. The AUCs of LR, GBM, DT and RF models are 0.827, 0.836, 0.760 and 0.829, respectively.
2. Calibration curves show that all models are well calibrated. Among them, the GBM model works best.
3. Decision curve analysis shows that both GBM model and LR model have good clinical practicability.
4. Based on the GBM model, the researchers further analyzed the importance of salient clinical features in the entire population. The top 3 features associated with one-year mortality were NT-proBNP, albumin, and statins, respectively, by analyzing both individual and mean SHAP values.
| SHAPE : Shaley Additive exPlanation, feature contribution. By analyzing SHAP values, researchers can obtain explanations for prediction results, understand how each feature affects the prediction of the model, and then better understand and explain the behavior of the model.
In summary, the researchers pointed out that although the models in previous studies had high predictive performance, they were not suitable for clinical application due to too many variables. In this study, the researchers successfully used 7 features to develop a model for predicting one-year mortality. The results showed that the GBM model had an AUC of 0.836 and a Brier score of 0.116, the best overall predictive performance .
It is worth noting that, in order to further facilitate clinical applications, the researchers also designed an online application that only requires doctors to fill in patient parameters to predict the probability of death within one year. Favorable measures to improve the survival probability of patients.
The future of AI in healthcare is bright, but don't be blindly optimistic
With the gradual maturity of technologies such as AI voice interaction, computer vision and cognitive computing, and deep learning, AI medical application scenarios are becoming more and more abundant, involving medical imaging, virtual assistants, drug development, health management, medical record/document analysis, and disease prediction Management and many other directions .
According to the "2020 Blue Book on the Development of Artificial Intelligence Medical Industry" issued by the China Academy of Information and Communications Technology, although the domestic AI medical field started relatively late, the market demand is strong and the future development prospects are broad . Among them, it is worth noting that as of the end of 2019, the proportion of the country's elderly population aged 65 and over has reached 12.6%, which means that China has officially entered an aging society. As a result, the incidence of chronic diseases is also increasing year by year.
In this context, the results related to disease prediction represented by this study emerged as the times require, which can effectively help doctors and patients to better manage their health. However, on the other hand, we also need to see that in terms of the overall market situation, AI-related technologies have not yet been applied on a large scale in hospitals, and hospitals are not very willing to pay, which is related to users' usage and payment habits, medical insurance policies, etc. There is an important connection between supporting infrastructure and the high complexity of clinical application scenarios. Therefore, for the AI medical field, there is still a long way to go.
Reference links:
[1] https://doi.org/10.5334/gh.934
[2] https://doi.org/10.1111/1753-0407.13175
[3] https://doi.org/10.1007/s001250051352
[ 4] https://doi.org/10.1186/1475-2840-5-15
[5] https://rs.yiigle.com/CN112148202107/1328929.htm
[6] http://www.caict.ac. cn/kxyj/qwfb/ztbg/202009/P020200910495521359097.pdf
This article was first published by HyperAI on the WeChat public platform~