PDF

Int J Med Sci 2022; 19(6):1049-1055. doi:10.7150/ijms.71341 This issue Cite

Research Paper

A Clinical Decision Support System for Diabetes Patients with Deep Learning: Experience of a Taiwan Medical Center

Ting-Ying Chien^1,2,3, Hsien-Wei Ting^3,4, Chih-Fang Chen⁵, Cheng-Zen Yang^1,3 , Chong-Yi Chen¹

1. Department of Computer Science and Engineering, Yuan Ze University, Taoyuan City, Taiwan.
2. Innovation Center for Big Data and Digital Convergence, Yuan Ze University, Taoyuan City, Taiwan.
3. Graduated Program in Biomedical Informatics, Yuan Ze University, Taoyuan City, Taiwan.
4. Department of Neurosurgery, Taipei Hospital, Ministry of Health and Welfare, New Taipei City, Taiwan.
5. Department of Pharmacy, MacKay Memorial Hospital, Taipei City, Taiwan.

✉ Corresponding author: Cheng-Zen Yang, Department of Computer Science and Engineering, Yuan Ze University, Taoyuan City, Taiwan. E-mail: czyangyzu.edu.tw.

Abstract

Background: Diabetes mellitus (DM) is a major public health problem worldwide. It involves dysfunction of blood sugar regulation resulting from insulin resistance, inadequate insulin secretion, or excessive glucagon secretion.

Methods: This study collated 971,401 drug usage records of 51,009 DM patients. These data include patient identification code, age, gender, outpatient visiting dates, visiting code, medication features (included items, doses, and frequencies of drugs), HbA1c results, and testing time. We apply a random forest (RF) model for feature selection and implement a regression model with the bidirectional long short-term memory (Bi-LSTM) deep learning architecture. Finally, we use the root mean square error (RMSE) as the evaluation index for the prediction model.

Results: After data cleaning, the data included 8,729 male and 9,115 female cases. Metformin was the most important feature suggested by the RF model, followed by glimepiride, acarbose, pioglitazone, glibenclamide, gliclazide, repaglinide, nateglinide, sitagliptin, and vildagliptin. The model performed better with the past two seasons in the training data than with additional seasons. Further, the Bi-LSTM architecture model performed better than support vector machines (SVMs).

Discussion & Conclusion: This study found that Bi-LSTM models is a well kernel in a CDSS which help physicians' decision-making, and the increasing the number of seasons will negative impact the performance. In addition, this study found that the most important drug is metformin, which is recommended as first-line treatment OHA in various situations for DM patients.

Keywords: Bidirectional Long Short-Term Memory (Bi-LSTM), Clinical Decision Support System (CDSS), Deep Learning, Diabetes Mellitus, Electronic Health Record, Oral Hypoglycemic Agents

Introduction

Diabetes mellitus (DM) is a major public health problem worldwide. It involves dysfunction of blood sugar regulation resulting from insulin resistance, inadequate insulin secretion, or excessive glucagon secretion [1]. There are two types of DM. Type 1 DM is usually due to an autoimmune disorder and involves the destruction of pancreatic beta-cells. Type 2 DM is caused by impairment of glucose regulation due to the malfunction of pancreatic beta cells or insulin resistance [1]. Treatment using oral hypoglycemic agents (OHA) for type 2 DM may have negative side effects, such as hypoglycemia. Therefore, it is crucial to ensure the safety and efficacy of OHA usage [1-7].

Clinical decision support systems (CDSSs), which integrate electronic health records (EHRs) and expert knowledge, have improved the decision-making of physicians and medical caregivers [8-12]. Many methods have been developed for CDSSs, including linear/logistic regression, support vector machines (SVMs), decision trees, random forest (RF), rough sets, and trajectory methods [13-19]. Glycated hemoglobin (HbA1c) is extensively studied in these approaches because it is a good indicator of DM control. DM patients with higher Hba1c measures are more likely to experience renal diseases, macrovascular events, cardiovascular diseases, retinopathies, skin ulceration/gangrene, and high mortality [3]. A well-controlled HbA1c value plays an important role in DM management. CDSSs have been shown to be effective in supporting HbA1c control. For example, O'Connor et al. showed that the HbA1c of DM patients significantly improve when the physicians use a CDSS compared with when physicians do not use a CDSS (p < 0.01). Moreover, 94% of physicians using the CDSS were satisfied for this application and physicians continued to use the CDSS for more than one year without research funding support [20].

Recently, deep learning methods have dramatically improved different fields of medical care and research [21, 22]. They have also been used as the core methods to build the CDSS [23, 24]. For example, convolutional neural networks (CNNs) are used to process image data and recurrent neural networks (RNNs) are used for sequential pattern problems [23, 25]. Sun et al. proposed a method to predict blood sugar levels at four intervals, namely 15, 30, 45, and 60 minutes, using the long short-term memory (LSTM) model and the bidirectional-LSTM (Bi-LSTM) model [26]. Therefore, we devise a CDSS using a Bi-LSTM model with HbA1c as the outcome index for managing OHA usage. The structure of the proposed CDSS, the LSTM model, and the Bi-LSTM model are shown in Figure 1.

Materials and Methods

We collated 971,401 drug usage records of 51,009 diabetes mellitus (DM) patients from January 2012 to December 2014 (12 seasons) and 313,165 laboratory records of 74,792 DM patients in a medical center from January 2012 to June 2015 (14 seasons). These data included patient identification code, age, gender, outpatient visiting dates, visiting code, medication features (included items, doses, and frequencies of drugs), HbA1c results, and testing time. The data were combined and cleansed. Twelve seasons of data and 17,844 DM patients were included in this study. The data were evaluated with five-fold cross-validation (training data = 80% and testing data = 20%) (Figure 2). We applied an RF model with mean square error (MSE) for feature selection where higher mean decrease MSE indicated more important parameters [27-29]. OHA dosages and codes were collected. This study was approved by the Institutional Review Board (IRB) of the MacKay Memorial Hospital (IRB approval number: 15MMHIS143e).

We implemented the Bi-LSTM structure using PyTorch running on two personal computers equipped, respectively, with Ubuntu 16.04 and Ubuntu 18.04. The GPU environments were NVIDIA GeForce GTX 980 and GTX 1080 Ti. We used Grid search to adjust the model parameters: epoch (100); batch size (32, 64, 128); hidden layer neuron numbers of Bi-LSTM (32, 64, 128); dropout rate (0, 0.2) between Bi-LSTM; optimizers AMSBound [30] and Adadelda [31]; and learning rate (AMSBound used 0.001 and the Adadelda used 1.0). We used early stopping (patience = 30) and L2 normalization (weight decay = 0.0005) to solve the problem of overfitting and applied gradient clipping (clip norm = 5) to prevent exploding gradient problems. We also implemented a support vector regression (SVR) model using Rgtsvm in R to compare with the Bi-LSTM model [18].

To evaluate the models, we used root mean square error (RMSE) (), sensitivity (), specificity (), and the Matthews correlation coefficient (MCC) (). We applied Pearson's chi-squared test and the student t-test for data analysis. The statistical analysis was conducted using SPSS version 19.0 (SPSS Inc., Chicago, IL, USA). Statistical significance was defined as p < 0.05.

Results

Of the included 17,844 cases, 8729 (49.0%) were male and 9115 (51.0%) were female. The mean age was 62.3 years old (SD = 11.9) overall, 60.4 (SD = 11.8) for males, and 64.2 (11.7) for females. The 45 to 64 year old age had the most cases (8,507 cases), followed by those aged above 65(6,966 cases). The mean Hba1c was 7.6% (SD = 1.7). There were 13,346 cases with Hba1c higher than 6.5% and 4,498 cases whose Hba1c were less than 6.5% (Table 1).

 Figure 1 

Structure of the clinical decision support system (CDSS), Long short time model (LSTM) and bidirectional-LSTM (Bi-LSTM) model. CDSS: clinical decision support system; EHR: electronic health records; Bi-LSTM: Bidirectional long short-term memory network model.

The data included 11 types of drugs. Compound drugs glimepiride (25,719), pioglitazone (25,720), and vildagliptin (25,726) were combined with metformin dosages of 500, 850, and 1000 mg, respectively. Nateglinide had dosages of 60 and 120 mg.

The most important feature was metformin with mean decreased MSE = 171.8, followed by glimepiride (156.6), acarbose (151.8), pioglitazone (148.1), glibenclamide (143.7), gliclazide (114,1), repaglinide (93.3), nateglinide (80.6), sitagliptin (74.0), and vildagliptin (21.9) (Table 2).

 Figure 2 

Data management flowchart. R: visiting times; P: patient case numbers; Q: seasonal times; Lab. data: laboratory data.

This study treated every season as ground truth from 2013 Q1 to 2015 Q1 and constructed nine datasets, each having a different sample size. For example, the dataset of 2014 Q4 had 12,677 and 3169 cases as training and test samples, respectively. Using other data as independent factors, we designed three kinds of models. The first used two seasons of data to predict drug usage of the third season. For example, model 9 (2015 Q1) used 2014 Q3 and 2014 Q4 to predict 2015 Q1. The other two types of model used three/four seasons to predict the drugs of the fourth/fifth seasons.

This study also evaluated differences in Hba1c between seasons. For example, we calculated the differences in mean Hba1c between 2015 Q1 and 2014 Q4 (0.87%), 2014 Q3 (0.98%) and 2014 Q2 (1.09%). We found that longer time distances had greater differences in Hba1c (Table 3).

We compared Bi-LSTM and SVM in the two-, three-, and four-season models. The RMSE of both two-season models (Bi-LSTM = 1.05±0.17 and. SVM = 1.05±0.07) was the best, followed by the three-season models (Bi-LSTM = 1.10±0.25 and. SVM = 1.12±0.03) and four-season models (Bi-LSTM = 1.09±0.21 and SVM = 1.16±0.04).

The sensitivity of the SVM models were not significantly different to each other (two seasons: 0.88±0.03, three seasons: 0.88±0.02, four seasons: 0.89±0.02). The sensitivity of the Bi-LSTM models gradually decreased non-significantly (two seasons: 0.83±0.16, three seasons: 0.80±0.21, four seasons: 0.77±0.23), but performed worse than the SVM models.

 Table 4 

Comparison of Bi-LSTM and SVM models with different seasons

Seasonal model	Evaluation index	SVM	Bi-LSTM
Two seasons	RMSE	1.05±0.07	1.05±0.17
	Sensitivity	0.88±0.03	0.83±0.16
	Specificity	0.68±0.05	0.69±0.32
	MCC	0.57±0.05	0.56±0.15
	Duration (Hours)	0.39±0.06	3.39±0.64
Three seasons	RMSE	1.12±0.03	1.10±0.25
	Sensitivity	0.88±0.02	0.80±0.21
	specificity	0.64±0.05	0.71±0.31
	MCC	0.54±0.05	0.55±0.15
	Duration (Hours)	0.47±0.09	5.18±1.07
Four seasons	RMSE	1.16±0.04	1.09±0.21
	sensitivity	0.89±0.02	0.77±0.23
	specificity	0.59±0.04	0.71±0.30
	MCC	0.50±0.04	0.54±0.12
	Duration (Hours)	0.52±0.06	5.36±1.17

Bi-LSTM: Bidirectional long short-term model. SVM: support vector machine.

Root mean square error (RMSE) =; Sensitivity = ; Specificity = ; Matthews correlation coefficient (MCC) = .

The specificity of the SVM models gradually decreased as the included seasons increased (two seasons: 0.68±0.05, three seasons: 0.64±0.05, four seasons: 0.59±0.04). The specificity of the Bi-LSTM models was not significant for any approach (two seasons: 0.69±0.32, three seasons: 0.71±0.31, four seasons: 0.71±0.30). According to the MCC evaluation, there were no significant differences between the six models. The two-season SVM model (0.39±0.06) had the shortest run time, followed by the three-season (0.47±0.09) and four-season (0.52±0.06) SVM models. The Bi-LSTM models had significantly longer run times (two seasons: 3.39±0.64, three seasons: 5.18±1.07, four seasons: 5.36±1.17) than the SVM models (Table 4).

Discussion & Conclusion

Studies have found that higher Hba1c is linked with increased risk of complications in DM patients [3]. A physician-pharmacist collaboration is useful for OHA adjustment to manage Hba1c owing to physician knowledge and experience [32]. However, it is a challenge to leverage the knowledge of these experts. The current study found that Bi-LSTM models performed better than SVM models for a CDSS to support physicians' decision-making related to OHA adjustment for DM patients. Many CDSS and classification models have used SVM and other artificial intelligence technologies [18, 33-36].

We also found that increasing the number of seasons used in the prediction negatively impacted accuracy and RMSE. Physicians reference the most recent Hba1c value to adjust OHA dosage and may choose to maintain the dosage if the Hba1c value is only slightly higher than 7% for the first time. Thus, it is not necessary to reference three or more seasons of Hba1c data, as validated by our experimental results.

We calculated the importance of these drugs and used RF to translate this information into the CDSS [27-29, 33, 35]. The most important drug was metformin, which is recommended as first-line treatment OHA in various situations for DM patients. This drug improves lipids and inflammatory markers and reduces cardiovascular events, but may be contraindicated for patients with mild to moderate chronic kidney disease. Recent research indicates that metformin requires caution in these kinds of DM patients [5]. Sulfonylureas is an important DM drug. We found that the glimepiride (2^nd) glibenclamide (5^th), and gliclazide (6^th) are also important OHA for DM patients [2, 4]. Acarbose was also found to have some gastrointestinal adverse effects, which were similar to metformin [6]. Pioglitazone is an important DM drug that may reduce Hba1c and improve both metabolic syndrome and nonalcoholic fatty liver disease/nonalcoholic steatohepatitis [7]. It also has a side effect of weight loss for some patients, which is sometimes treated as a benefit. Glimepiride, pioglitazone, and vildagliptin were all combined with metformin (Table 2).

This study has some limitations and areas for extension. Yanase et al. reported that low HbA1c is linked with frailty and suspected malnutrition in elderly type 2 DM patients [37]. Around 25% of cases in this study had Hba1c < 6.5%, indicating that the DM control was too strict for some patients, potentially leading to malnutrition or hypoglycemia. Our CDSS defined “good control” as Hba1c ≤ 7. Although our approach worked well, future versions could be enhanced by considering 6.5 ≤ Hba1c ≤ 7.

Acknowledgements

This study was supported by the MacKay Memorial Hospital (MMH-E-10403) and the Ministry of Science and Technology—MOST110-2221-E-155-051 and MOST110-2627-H-155-001. The authors would like to thank the Mackay Memorial Hospital for providing the dataset used in our study.

Author Contributions

T.Y. Chien and C.Z. Yang designed the study; C.F. Chen collected data; C.F. Chen and H.W. Ting provided support regarding knowledge of medicines; T.Y. Chien and C.Z. Yang provided support regarding information technology knowledge; C.Y. Chen constructed the decision support system; T.Y. Chien and H.W. Ting wrote the manuscript; C.Z. Yang made the decision to publish.

Competing Interests

The authors have declared that no competing interest exists.

References

1. Blair M. Diabetes Mellitus Review. Urol Nurs. 2016;36:27-36

2. Chen C-F, Ting H-W, Yang C-Z, Yeh H-I, Hung T-C. Effectiveness of Brand-Name and Generic Versions of Glimepiride for Diabetes Mellitus Care: Experience at a Medical Center in Taiwan. Int J Gerontol. 2019;13:116-20

3. Gorst C, Kwok CS, Aslam S, Buchan I, Kontopantelis E, Myint PK. et al. Long-term Glycemic Variability and Risk of Adverse Outcomes: A Systematic Review and Meta-analysis. Diabetes Care. 2015;38:2354-69

4. Kalra S, Bahendeka S, Sahay R, Ghosh S, Md F, Orabi A. et al. Consensus Recommendations on Sulfonylurea and Sulfonylurea Combinations in the Management of Type 2 Diabetes Mellitus - International Task Force. Indian J Endocrinol Metab. 2018;22:132-57

5. Sanchez-Rangel E, Inzucchi SE. Metformin: clinical use in type 2 diabetes. Diabetologia. 2017;60:1586-93

6. Gu S, Shi J, Tang Z, Sawhney M, Hu H, Shi L. et al. Comparison of glucose lowering effect of metformin and acarbose in type 2 diabetes mellitus: a meta-analysis. PLoS One. 2015;10:e0126704

7. DeFronzo RA, Inzucchi S, Abdul-Ghani M, Nissen SE. Pioglitazone: The forgotten, cost-effective cardioprotective drug for type 2 diabetes. Diab Vasc Dis Res. 2019;16:133-43

8. Légat L, Van Laere S, Nyssen M, Steurbaut S, Dupont AG, Cornu P. Clinical Decision Support Systems for Drug Allergy Checking: Systematic Review. J Med Internet Res. 2018;20:e258

9. Cho I, Bates DW. Behavioral Economics Interventions in Clinical Decision Support Systems. Yearb Med Inform. 2018;27:114-21

10. Kilsdonk E, Peute LW, Jaspers MW. Factors influencing implementation success of guideline-based clinical decision support systems: A systematic review and gaps analysis. Int J Med Inform. 2017;98:56-64

11. Bright TJ, Wong A, Dhurjati R, Bristow E, Bastian L, Coeytaux RR. et al. Effect of clinical decision-support systems: a systematic review. Ann Intern Med. 2012;157:29-43

12. Spänig S, Emberger-Klein A, Sowa J-P, Canbay A, Menrad K, Heider D. The virtual doctor: An interactive clinical-decision-support system based on deep learning for non-invasive prediction of diabetes. Artif Intell Med. 2019;100:101706

13. Breiman L. Random Forests. Machine Learning. 2001;45:5-32

14. Ting HW, Chien TY, Lai KR, Pan RH, Wu KH, Chen JM. et al. Differences in Spontaneous Intracerebral Hemorrhage Cases between Urban and Rural Regions of Taiwan: Big Data Analytics of Government Open Data. Int J Environ Res Public Health. 2017;14:1548

15. Tseng TL, Huang CC, Fraser K, Ting HW. Rough set based rule induction in decision making using credible classification and preference from medical application perspective. Comput Methods Programs Biomed. 2016;127:273-89

16. Ting HW, Wu JT, Chan CL, Lin SL, Chen MS. Decision Model for Acute Appendicitis Treatment With Decision Tree Technology-A Modification of the Alvarado Scoring System J Chin Med Assoc 2010; 73: 401-6

17. Chien TY, Lee ML, Wu WL, Ting HW. Exploration of Medical Trajectories of Stroke Patients Based on Group-Based Trajectory Modeling. Int J Environ Res Public Health. 2019;16:3472

18. Chan C-L, Chen C-L, Ting H-W, Phan D-V. An Agile Mortality Prediction Model: Hybrid Logarithm Least-Squares Support Vector Regression with Cautious Random Particle Swarm Optimization. Int J Comput Intell Syst. 2018;11:873-81

19. Teixeira AL, Leal JP, Falcao AO. Random forests for feature selection in QSPR Models - an application for predicting standard enthalpy of formation of hydrocarbons. J Cheminform. 2013;5:9

20. O'Connor PJ, Sperl-Hillen JM, Rush WA, Johnson PE, Amundson GH, Asche SE. et al. Impact of electronic health record clinical decision support on diabetes care: a randomized trial. Ann Fam Med. 2011;9:12-21

21. LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015;521:436-44

22. Yuan Q, Zhang H, Deng T, Tang S, Yuan X, Tang W. et al. Role of Artificial Intelligence in Kidney Disease. Int J Med Sci. 2020;17:970-84

23. Ting HW, Chung SL, Chen CF, Chiu HY, Hsieh YW. A drug identification model developed using deep learning technologies: experience of a medical center in Taiwan. BMC Health Serv Res. 2020;20:312

24. Feng Y, Wang Y, Zeng C, Mao H. Artificial Intelligence and Machine Learning in Chronic Airway Diseases: Focus on Asthma and Chronic Obstructive Pulmonary Disease. International Journal of Medical Sciences. 2021;18:2871

25. Yasaka K, Akai H, Kunimatsu A, Kiryu S, Abe O. Deep learning with convolutional neural network in radiology. Jpn J Radiol. 2018;36:257-72

26. Sun Q, Jankovic MV, Bally L, Mougiakakou SG. Predicting Blood Glucose with an LSTM and Bi-LSTM Based Deep Neural Network. 2018 14th Symposium on Neural Networks and Applications (NEUREL). 2018 p. 1-5

27. Dalal V, Carmicheal J, Dhaliwal A, Jain M, Kaur S, Batra SK. Radiomics in stratification of pancreatic cystic lesions: Machine learning in action. Cancer Lett. 2020;469:228-37

28. Ferreira JR Jr, Oliveira MC, de Azevedo-Marques PM. Characterization of Pulmonary Nodules Based on Features of Margin Sharpness and Texture. J Digit Imaging. 2018;31:451-63

29. Boostani R, Karimzadeh F, Nami M. A comparative review on sleep stage classification methods in patients and healthy individuals. Comput Methods Programs Biomed. 2017;140:77-91

30. Luo L, Xiong Y, Liu Y, Sun X. Adaptive Gradient Methods with Dynamic Bound of Learning Rate. 2019: arXiv:1902.09843.

31. Zeiler MD. ADADELTA: An Adaptive Learning Rate Method. 2012: arXiv:1212.5701.

32. Hwang AY, Gums TH, Gums JG. The benefits of physician-pharmacist collaboration. J Fam Pract. 2017;66:E1-e8

33. Huang S, Cai N, Pacheco PP, Narrandes S, Wang Y, Xu W. Applications of Support Vector Machine (SVM) Learning in Cancer Genomics. Cancer Genomics Proteomics. 2018;15:41-51

34. Krell MM, Wilshusen N, Seeland A, Kim SK. Classifier transfer with data selection strategies for online support vector machine classification with class imbalance. J Neural Eng. 2017;14:025003

35. Jiang F, Jiang Y, Zhi H, Dong Y, Li H, Ma S. et al. Artificial intelligence in healthcare: past, present and future. Stroke Vasc Neurol. 2017;2:230-43

36. Han H, Jiang X. Overcome support vector machine diagnosis overfitting. Cancer Inform. 2014;13:145-58

37. Yanase T, Yanagita I, Muta K, Nawata H. Frailty in elderly diabetes patients. Endocr J. 2018;65:1-11

Author contact

Corresponding author: Cheng-Zen Yang, Department of Computer Science and Engineering, Yuan Ze University, Taoyuan City, Taiwan. E-mail: czyangyzu.edu.tw.

Received 2022-1-23
Accepted 2022-5-19
Published 2022-6-13

Citation styles

©2024 Ivyspring International Publisher. Terms of use