Peningkatan Keberagaman Data untuk Klasifikasi Penyakit Diabetes Berbasis Stacking Ensemble Learning

nur kholis majid; Catur Supriyanto; Aris Marjuni

doi:10.30591/jpit.v10i1.7375

Peningkatan Keberagaman Data untuk Klasifikasi Penyakit Diabetes Berbasis Stacking Ensemble Learning

nur kholis majid, Catur Supriyanto, Aris Marjuni

Abstract

Diabetes cases are becoming more common in the late years. Diabetes attacks not only parents, but also children. The development of diabetes is not far from the lifestyle and diet that we live on a daily basis. Therefore, early detection of diabetes is essential because the earlier the disease is detected, the easier it is to treat. In the process of detecting disease based on factors, the cause can be predicted with data mining. The aim of this research is to increase data diversity so that it can be processed to the maximum in data mining. In the process of data upgrading, we used the imbalance learning method SMOTE-ENN combined with the method Stacking Ensemble Learning. In the search for a powerful stacking model, seven classification algorithms were involved in the experiments carried out on this study, namely: Random Forest, Decision Tree, Gradient Boosting, Naïve Bayes, Extreme Gradiant Boost, Logistic Regression, and k-Nearest Neighbor. Four algorithms were used to be classifiers level 0 (base model), namely kNN, Gradient Boosting, decision tree, and random forest, while Random Forest was used again to be classifier level 1. (meta model). With these combinations, the accuracy obtained is 97.3%. These are the highest results when compared to individual algorithms.

Full Text:

References

“Diabetes estimates (20-79 y): People with diabetes, in 1,000s.” Accessed: Mar. 22, 2024. [Online]. Available: https://diabetesatlas.org/data/

“Diabetes basics,” International Diabetes Federation. Accessed: Jun. 15, 2024. [Online]. Available: https://idf.org/about-diabetes/what-is-diabetes/

“Complications,” International Diabetes Federation. Accessed: Jun. 15, 2024. [Online]. Available: https://idf.org/about-diabetes/diabetes-complications/

X. Hu, L. Chu, J. Pei, W. Liu, and J. Bian, “Model Complexity of Deep Learning: A Survey,” Aug. 02, 2021, arXiv: arXiv:2103.05127. doi: 10.48550/arXiv.2103.05127.

A. Holzinger, “Machine Learning for Health Informatics,” vol. 9605, 2016, pp. 1–24. doi: 10.1007/978-3-319-50478-0_1.

R. Longadge and S. Dongre, “Class Imbalance Problem in Data Mining Review,” May 07, 2013, arXiv: arXiv:1305.1707. doi: 10.48550/arXiv.1305.1707.

Z. Xie, O. Nikolayeva, J. Luo, and D. Li, “Building Risk Prediction Models for Type 2 Diabetes Using Machine Learning Techniques,” Prev Chronic Dis, vol. 16, p. E130, Sep. 2019, doi: 10.5888/pcd16.190109.

A. Daza Vergaray et al., “Stacking Ensemble Based Hyperparameters for the Prediction of Heart Disease,” Sep. 06, 2023, Rochester, NY: 4553047. doi: 10.2139/ssrn.4553047.

A. Carriero, L. Groenhoff, E. Vologina, P. Basile, and M. Albera, “Deep Learning in Breast Cancer Imaging: State of the Art and Recent Advancements in Early 2024,” Diagnostics, vol. 14, no. 8, Art. no. 8, Jan. 2024, doi: 10.3390/diagnostics14080848.

J. Han and M. Kamber, Data mining: concepts and techniques, 3rd ed. Burlington, MA: Elsevier, 2012.

J. Han, J. Pei, and H. Tong, Data Mining: Concepts and Techniques. Morgan Kaufmann, 2022.

K. Lai, N. Twine, A. O’Brien, Y. Guo, and D. Bauer, “Artificial Intelligence and Machine Learning in Bioinformatics,” in Encyclopedia of Bioinformatics and Computational Biology, S. Ranganathan, M. Gribskov, K. Nakai, and C. Schönbach, Eds., Oxford: Academic Press, 2019, pp. 272–286. doi: 10.1016/B978-0-12-809633-8.20325-7.

J. Amin, M. A. Anjum, M. Sharif, S. Kadry, and J. Kim, “Explainable Neural Network for Classification of Cotton Leaf Diseases,” Agriculture, vol. 12, no. 12, Art. no. 12, Dec. 2022, doi: 10.3390/agriculture12122029.

M. Beckmann, N. F. F. Ebecken, and B. S. L. P. de Lima, “A KNN Undersampling Approach for Data Balancing,” Journal of Intelligent Learning Systems and Applications, vol. 7, no. 4, Art. no. 4, Sep. 2015, doi: 10.4236/jilsa.2015.74010.

L. Rokach and O. Maimon, “Decision Trees,” in The Data Mining and Knowledge Discovery Handbook, vol. 6, 2005, pp. 165–192. doi: 10.1007/0-387-25465-X_9.

L. Breiman, “Random Forests,” Machine Learning, vol. 45, no. 1, pp. 5–32, Oct. 2001, doi: 10.1023/A:1010933404324.

T. Pan, J. Zhao, W. Wu, and J. Yang, “Learning imbalanced datasets based on SMOTE and Gaussian distribution,” Information Sciences, vol. 512, pp. 1214–1233, Feb. 2020, doi: 10.1016/j.ins.2019.10.048.

D. Elreedy and A. F. Atiya, “A Comprehensive Analysis of Synthetic Minority Oversampling Technique (SMOTE) for handling class imbalance,” Information Sciences, vol. 505, pp. 32–64, Dec. 2019, doi: 10.1016/j.ins.2019.07.070.

M. O. Ojo and A. Zahid, “Improving Deep Learning Classifiers Performance via Preprocessing and Class Imbalance Approaches in a Plant Disease Detection Pipeline,” Agronomy, vol. 13, no. 3, Art. no. 3, Mar. 2023, doi: 10.3390/agronomy13030887.

“CDC - BRFSS - BRFSS Frequently Asked Questions (FAQs).” Accessed: Jul. 19, 2024. [Online]. Available: https://www.cdc.gov/brfss/about/brfss_faq.htm

G. Bhola, A. Garg, and M. Kumari, “Comparative Study of Machine Learning Techniques for Chronic Disease Prognosis,” in Computer Networks and Inventive Communication Technologies, S. Smys, R. Palanisamy, Á. Rocha, and G. N. Beligiannis, Eds., Singapore: Springer Nature, 2021, pp. 131–144. doi: 10.1007/978-981-15-9647-6_10.

H. Hairani and D. Priyanto, “A New Approach of Hybrid Sampling SMOTE and ENN to the Accuracy of Machine Learning Methods on Unbalanced Diabetes Disease Data,” International Journal of Advanced Computer Science and Applications, vol. 14, pp. 585–590, Aug. 2023, doi: 10.14569/IJACSA.2023.0140864.

M. A. Latief, L. R. Nabila, W. Miftakhurrahman, S. Ma’rufatullah, and H. Tantyoko, “Handling Imbalance Data using Hybrid Sampling SMOTE-ENN in Lung Cancer Classification,” IJECSA, vol. 3, no. 1, pp. 11–18, Feb. 2024, doi: 10.30812/ijecsa.v3i1.3758.

M. Song, H. Jung, S. Lee, D. Kim, and M. Ahn, “Diagnostic Classification and Biomarker Identification of Alzheimer’s Disease with Random Forest Algorithm,” Brain Sciences, vol. 11, no. 4, p. 453, Apr. 2021, doi: 10.3390/brainsci11040453.

Z. Ullah et al., “Detecting High-Risk Factors and Early Diagnosis of Diabetes Using Machine Learning Methods,” Computational Intelligence and Neuroscience, vol. 2022, pp. 1–10, Sep. 2022, doi: 10.1155/2022/2557795.

P. Lu et al., “A Gradient Boosting Crash Prediction Approach for Highway-Rail Grade Crossing Crash Analysis,” Journal of Advanced Transportation, vol. 2020, pp. 1–10, Jun. 2020, doi: 10.1155/2020/6751728.

A. Ghasemieh, A. Lloyed, P. Bahrami, P. Vajar, and R. Kashef, “A novel machine learning model with Stacking Ensemble Learner for predicting emergency readmission of heart-disease patients,” Decision Analytics Journal, vol. 7, p. 100242, Jun. 2023, doi: 10.1016/j.dajour.2023.100242.

“Cara Convert String ke Integer di Python: Panduan Lengkap 2024 | RevoU.” Accessed: Jul. 02, 2024. [Online]. Available: https://revou.co/panduan-teknis/convert-string-to-int-python

Y. Kwon, M. Lemieux, J. McTavish, and N. Wathen, “Identifying and removing duplicate records from systematic review searches,” J Med Libr Assoc, vol. 103, no. 4, pp. 184–188, Oct. 2015, doi: 10.3163/1536-5050.103.4.004.

G. E. A. P. A. Batista, R. C. Prati, and M. C. Monard, “A study of the behavior of several methods for balancing machine learning training data,” SIGKDD Explor. Newsl., vol. 6, no. 1, pp. 20–29, Jun. 2004, doi: 10.1145/1007730.1007735.

A. Puri and M. Kumar Gupta, “Improved Hybrid Bag-Boost Ensemble With K-Means-SMOTE–ENN Technique for Handling Noisy Class Imbalanced Data,” The Computer Journal, vol. 65, no. 1, pp. 124–138, Jan. 2020, doi: 10.1093/comjnl/bxab039.

H. Patel, D. Singh Rajput, G. Thippa Reddy, C. Iwendi, A. Kashif Bashir, and O. Jo, “A review on classification of imbalanced data for wireless sensor networks,” International Journal of Distributed Sensor Networks, vol. 16, no. 4, p. 1550147720916404, Apr. 2020, doi: 10.1177/1550147720916404.

S. K. Singh, A. K. Tiwari, and H. K. Paliwal, “A state-of-the-art review on the utilization of machine learning in nanofluids, solar energy generation, and the prognosis of solar power,” Engineering Analysis with Boundary Elements, vol. 155, pp. 62–86, Oct. 2023, doi: 10.1016/j.enganabound.2023.06.003.

A. A. Khan, O. Chaudhari, and R. Chandra, “A review of ensemble learning and data augmentation models for class imbalanced problems: Combination, implementation and evaluation,” Expert Systems with Applications, vol. 244, p. 122778, Jun. 2024, doi: 10.1016/j.eswa.2023.122778.

X. Dong, Z. Yu, W. Cao, Y. Shi, and Q. Ma, “A survey on ensemble learning,” Frontiers of Computer Science, vol. 14, no. 2, pp. 241–258, Apr. 2020, doi: 10.1007/s11704-019-8208-z.

D. Wolpert, “Stacked Generalization,” Neural Networks, vol. 5, pp. 241–259, Dec. 1992, doi: 10.1016/S0893-6080(05)80023-1.

N. V. S, D. Sripada, S. V, and M. Aghaei, “Detection of visual faults in photovoltaic modules using a stacking ensemble approach,” Heliyon, vol. 10, no. 6, p. e27894, Mar. 2024, doi: 10.1016/j.heliyon.2024.e27894.

H. Qian, N. Bao, D. Meng, B. Zhou, H. Lei, and H. Li, “Mapping and classification of Liao River Delta coastal wetland based on time series and multi-source GaoFen images using stacking ensemble model,” Ecological Informatics, vol. 80, p. 102488, May 2024, doi: 10.1016/j.ecoinf.2024.102488.

M. Ali, M. N. Haider, S. A. Lashari, W. Sharif, A. Khan, and D. A. Ramli, “Stacking Classifier with Random Forest functioning as a Meta Classifier for Diabetes Diseases Classification,” Procedia Computer Science, vol. 207, pp. 3459–3468, Jan. 2022, doi: 10.1016/j.procs.2022.09.404.

S. A. Hicks et al., “On evaluation metrics for medical applications of artificial intelligence,” Sci Rep, vol. 12, p. 5979, Apr. 2022, doi: 10.1038/s41598-022-09954-8.

S. Coulibaly, B. Kamsu-Foguem, D. Kamissoko, and D. Traore, “Deep Convolution Neural Network sharing for the multi-label images classification,” Machine Learning with Applications, vol. 10, p. 100422, Dec. 2022, doi: 10.1016/j.mlwa.2022.100422.

Z.-H. Zhou, Ensemble Methods: Foundations and Algorithms, vol. 14. 2012. doi: 10.1201/b12207.

F. A. Rafrastara, C. Supriyanto, C. Paramita, and Y. P. Astuti, “Deteksi Malware menggunakan Metode Stacking berbasis Ensemble,” Jurnal Informatika: Jurnal Pengembangan IT, vol. 8, no. 1, Art. no. 1, Jan. 2023, doi: 10.30591/jpit.v8i1.4606.

“A Novel Dynamic Android Malware Detection System With Ensemble Learning | IEEE Journals & Magazine | IEEE Xplore.” Accessed: Aug. 01, 2024. [Online]. Available: https://ieeexplore.ieee.org/document/8374030

A. Ahad, A Stacking-based Ensemble Learning Method for Outlier Detection. 2021. doi: 10.21203/rs.3.rs-677845/v1.

Y. 1 Jia, H. 1 Zhang, H. 2 Liu, G. 1 Zhong, and G. 1 1 C. of C. A. Li, “Flight Delay Classification Prediction Based on Stacking Algorithm,” 2021, doi: 10.1155/2021/4292778.

“Papers with Code - LTG-ST at NADI Shared Task 1: Arabic Dialect Identification using a Stacking Classifier.” Accessed: Aug. 01, 2024. [Online]. Available: https://paperswithcode.com/paper/ltg-st-at-nadi-shared-task-1-arabic-dialect

H. Byeon, “Exploring Factors Associated with the Social Discrimination Experience of Children from Multicultural Families in South Korea by using Stacking with Non-linear Algorithm,” International Journal of Advanced Computer Science and Applications (IJACSA), vol. 12, no. 5, Art. no. 5, 58/31 2021, doi: 10.14569/IJACSA.2021.0120516.

Y. Wang, Q. Pan, X. Liu, and Y. Ding, “ET-MSF: a model stacking framework to identify electron transport proteins,” Front Biosci (Landmark Ed), vol. 27, no. 1, p. 12, Jan. 2022, doi: 10.31083/j.fbl2701012.

M. M. Chowdhury, R. S. Ayon, and M. S. Hossain, “An investigation of machine learning algorithms and data augmentation techniques for diabetes diagnosis using class imbalanced BRFSS dataset,” Healthcare Analytics, vol. 5, p. 100297, Jun. 2024, doi: 10.1016/j.health.2023.100297.

DOI: https://doi.org/10.30591/jpit.v10i1.7375