Monk Skin Tone Classification: RMSprop vs Adam Optimizer in MobileNetV2

Firman Naufal Aryaputra; Christy Atika Sari; Eko Hari Rachmawanto

doi:10.30591/jpit.v10i3.8886

Monk Skin Tone Classification: RMSprop vs Adam Optimizer in MobileNetV2

Firman Naufal Aryaputra, Christy Atika Sari, Eko Hari Rachmawanto

Abstract

The lack of accurate and accessible skin tone classification systems poses significant challenges in personalized fashion recommendations and inclusive technology development. This study aims to develop a skin tone classification system utilizing the Monk Skin Tone (MST) scale through the implementation of Convolutional Neural Network with MobileNetV2 architecture enhanced by transfer learning techniques. The MST scale encompasses ten distinct categories providing comprehensive representation of human skin color diversity. The methodology leverages efficient MobileNetV2 architecture suitable for web deployment, transfer learning to enhance accuracy despite limited training data, and strategic dataset balancing. A dataset of 1,729 facial photographs representing the complete MST spectrum was utilized. Preprocessing involved scaling images to 224×224 pixels, normalization, and augmentation through various transformations to address class imbalance challenges. The dataset was partitioned using a 70:15:15 ratio for training, validation, and testing respectively. The system was implemented as a web platform called SkinToneAI that enables users to upload facial images for skin tone analysis and receive personalized clothing color recommendations. Evaluation demonstrated classification accuracy of 97.83% on the test dataset with a loss value of 0.1166 when using Adam optimizer, while RMSprop optimizer achieved better performance with 98.26% accuracy and 0.0548 loss value. The implemented web application successfully translates technical capabilities into practical fashion assistance. The system provides users with customized apparel color suggestions based on their identified skin tone category, effectively connecting advanced AI technology with everyday fashion needs.

Keywords

Classification, Convolutional Neural Network, Deep Learning, MobileNetV2, Monk Skin Tone

Full Text:

References

F. W. Prabowo, A. Homaidi, and A. Lutfi, “Deteksi Warna Kulit Menggunakan Metode Deep Learning Dengan CNN (Convolutional Neural Network) Untuk Menentukan Kecocokan Warna Kulit dan Warna Busana,” E-Link: Jurnal Teknik Elektro dan Informatika, vol. 19, no. 2, p. 186, Jul. 2024, doi: 10.30587/e-link.v19i2.8128.

X. Su et al., “Personalized clothing recommendation fusing the 4-season color system and users’ biological characteristics,” Multimed Tools Appl, vol. 83, no. 5, pp. 12597–12625, Jul. 2023, doi: 10.1007/s11042-023-16014-4.

R. Fayyadhila, A. Junaidi, and N. A. Prasetyo, “Implementasi Deep Learning Untuk Klasifikasi Citra Undertone Menggunakan Algoritma Convolutional Neural Network,” Journal of Dinda : Data Science, Information Technology, and Data Analytics, vol. 1, no. 2, pp. 52–62, Aug. 2021, doi: 10.20895/dinda.v1i2.366.

M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen, “MobileNetV2: Inverted Residuals and Linear Bottlenecks,” in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, Jun. 2018, pp. 4510–4520. doi: 10.1109/CVPR.2018.00474.

S. Saiwaeo, S. Arwatchananukul, L. Mungmai, W. Preedalikit, and N. Aunsri, “Human skin type classification using image processing and deep learning approaches,” Heliyon, vol. 9, no. 11, p. e21176, Nov. 2023, doi: 10.1016/j.heliyon.2023.e21176.

V. D. Br Sebayang and I. G. N. L. W. Kusuma, “Klasifikasi Jenis Jerawat Berdasarkan Citra Menggunakan Convolutional Neural Network dengan Arsitektur MobileNetV2,” JURNAL FASILKOM, vol. 14, no. 3, pp. 766–774, Dec. 2024, doi: 10.37859/jf.v14i3.8202.

Y. Gulzar, “Fruit Image Classification Model Based on MobileNetV2 with Deep Transfer Learning Technique,” Sustainability, vol. 15, no. 3, p. 1906, Jan. 2023, doi: 10.3390/su15031906.

M. G. Somoal and A. R. Dzikrillah, “Komparasi MobileNETV2 dengan Kustomisasi Transfer Learning dan Hyperparameter untuk Identifikasi Tumor Otak,” Jurnal Teknologi Informasi dan Ilmu Komputer, vol. 12, no. 1, pp. 229–240, Feb. 2025, doi: 10.25126/jtiik.2025129582.

V. P. Matias and J. Batista Neto, “Enhancing Fairness in Machine Learning: Skin Tone Classification Using the Monk Skin Tone Scale,” in Anais Estendidos da XXXVII Conference on Graphics, Patterns and Images (SIBGRAPI Estendido 2024), Sociedade Brasileira de Computação - SBC, Sep. 2024, pp. 76–81. doi: 10.5753/sibgrapi.est.2024.31648.

C. Schumann, G. O. Olanubi, A. Wright, E. Monk, C. Heldreth, and S. Ricco, “Consensus and Subjectivity of Skin Tone Annotation for ML Fairness,” Jan. 2024. [Online]. Available: http://arxiv.org/abs/2305.09073

S. K. Mbatha, M. J. Booysen, and R. P. Theart, “Skin Tone Estimation under Diverse Lighting Conditions,” J Imaging, vol. 10, no. 5, p. 109, Apr. 2024, doi: 10.3390/jimaging10050109.

A. Mahbod, G. Schaefer, C. Wang, G. Dorffner, R. Ecker, and I. Ellinger, “Transfer learning using a multi-scale and multi-network ensemble for skin lesion classification,” Comput Methods Programs Biomed, vol. 193, p. 105475, Sep. 2020, doi: 10.1016/j.cmpb.2020.105475.

P. N. Srinivasu, J. G. SivaSai, M. F. Ijaz, A. K. Bhoi, W. Kim, and J. J. Kang, “Classification of Skin Disease Using Deep Learning Neural Networks with MobileNet V2 and LSTM,” Sensors, vol. 21, no. 8, p. 2852, Apr. 2021, doi: 10.3390/s21082852.

V. A. O. Nancy, P. Prabhavathy, M. S. Arya, and B. S. Ahamed, “Comparative study and analysis on skin cancer detection using machine learning and deep learning algorithms,” Multimed Tools Appl, vol. 82, no. 29, pp. 45913–45957, Dec. 2023, doi: 10.1007/s11042-023-16422-6.

K. Alomar, H. I. Aysel, and X. Cai, “Data Augmentation in Classification and Segmentation: A Survey and New Strategies,” J Imaging, vol. 9, no. 2, p. 46, Feb. 2023, doi: 10.3390/jimaging9020046.

R. J. Hendri Butar-Butar and N. L. Marpaung, “Deep Learning untuk Identifikasi Daun Tanaman Obat Menggunakan Transfer Learning MobileNetV2,” Jurnal Informatika: Jurnal Pengembangan IT, vol. 8, no. 2, pp. 142–148, May 2023, doi: 10.30591/jpit.v8i2.5217.

A. Furqon, K. Malik, and F. N. Fajri, “Detection of Eight Skin Diseases Using Convolutional Neural Network with MobileNetV2 Architecture for Identification and Treatment Recommendation on Android Application,” Jurnal Ilmiah Teknik Elektro Komputer dan Informatika, vol. 10, no. 2, pp. 373–384, Jul. 2024, doi: 10.26555/jiteki.v10i2.28817.

Tensorflow, “ModelCheckpoint,” Tensorflow. Accessed: Jun. 25, 2025. [Online]. Available: https://www.tensorflow.org/api_docs/python/tf/keras/callbacks/ModelCheckpoint

T. Ahmed, F. S. Mou, and A. Hossain, “SCCNet: An Improved Multi-Class Skin Cancer Classification Network using Deep Learning,” in 2024 3rd International Conference on Advancement in Electrical and Electronic Engineering (ICAEEE), IEEE, Apr. 2024, pp. 1–5. doi: 10.1109/ICAEEE62219.2024.10561672.

Qorry Aina Fitroh and Shofwatul ’Uyun, “Deep Transfer Learning untuk Meningkatkan Akurasi Klasifikasi pada Citra Dermoskopi Kanker Kulit,” Jurnal Nasional Teknik Elektro dan Teknologi Informasi, vol. 12, no. 2, pp. 78–84, May 2023, doi: 10.22146/jnteti.v12i2.6502.

W.-C. Cheng, “Monastic Color Reproduction: A Software Tool for Printing and Assessing the Monk Skin Tone Scale,” Journal of Imaging Science and Technology, vol. 68, no. 5, pp. 1–13, Sep. 2024, doi: 10.2352/J.ImagingSci.Technol.2024.68.5.050404.

K. Struniawski and R. Kozera, “TfELM: Extreme Learning Machines framework with Python and TensorFlow,” SoftwareX, vol. 27, p. 101833, Aug. 2024, doi: 10.1016/j.softx.2024.101833.

G. Kruseman, “A Flexible, Extensible, Machine-Readable, Human-Intelligible, and Ontology-Agnostic Metadata Schema (OIMS),” Front Sustain Food Syst, vol. 6, Mar. 2022, doi: 10.3389/fsufs.2022.767863.

S. S. N. Challapalli, P. Kaushik, S. Suman, B. D. Shivahare, V. Bibhu, and A. D. Gupta, “Web Development and performance comparison of Web Development Technologies in Node.js and Python,” in 2021 International Conference on Technological Advancements and Innovations (ICTAI), IEEE, Nov. 2021, pp. 303–307. doi: 10.1109/ICTAI53825.2021.9673464.

L. Albesher and R. Alfayez, “An Observational Study on Flask Web Framework Questions on Stack Overflow (SO),” IET Software, vol. 2024, no. 1, p. 1905538, Jan. 2024, doi: 10.1049/sfw2/1905538.

G. Kaur, H. Malhotra, and T. Gupta, “Choosing the Right Model: A Comprehensive Analysis of Outfit Recommendation Systems,” Int J Comput Appl, vol. 183, no. 12, pp. 13–20, Jun. 2021, doi: 10.5120/ijca2021921413.

S. S. Ali, J. H. Al’ Ameri, and T. Abbas, “Face Detection Using Haar Cascade Algorithm,” in 2022 Fifth College of Science International Conference of Recent Trends in Information Technology (CSCTIT), IEEE, Nov. 2022, pp. 198–201. doi: 10.1109/CSCTIT56299.2022.10145680.

M. Taqiyuddin, K. Adi, O. Dwi Nurhayati, and H. Ochi, “Comparison of Optimizers for Drone Signal Detection Using Convolutional Neural Networks (CNN),” E3S Web of Conferences, vol. 448, p. 02025, Nov. 2023, doi: 10.1051/e3sconf/202344802025.

G. A. Saputra and I. M. A. Agastya, “Betta Fish Identification System Based On Convolutional Neural Network,” Journal of Applied Informatics and Computing, vol. 8, no. 2, pp. 443–452, Nov. 2024, doi: 10.30871/jaic.v8i2.8449.

A. Murthy, P. S. Rao, N. S. Pallavi, N. Kharvi, B. R. Neha, and K. Poojary, “Optimizing Convolutional Neural Networks: A Comparative Study of Gradient-Descent, Adam, and RMSprop Optimizers for Accuracy and Loss in Apple Leaf Disease Detection,” in 2024 Second International Conference on Networks, Multimedia and Information Technology (NMITCON), IEEE, Aug. 2024, pp. 1–6. doi: 10.1109/NMITCON62075.2024.10699138.

Q. Zhang, Y. Zhou, and S. Zou, “Convergence Guarantees for RMSProp and Adam in Generalized-smooth Non-convex Optimization with Affine Noise Variance,” Transactions on Machine Learning Research (TMLR, Mar. 2025, Accessed: May 02, 2025. [Online]. Available: http://arxiv.org/abs/2404.01436

D. I. Perrett and R. Sprengelmeyer, “Clothing Aesthetics: Consistent Colour Choices to Match Fair and Tanned Skin Tones,” Iperception, vol. 12, no. 6, Nov. 2021, doi: 10.1177/20416695211053361.

DOI: https://doi.org/10.30591/jpit.v10i3.8886