Anomaly-Based Network Intrusion Detection Using Isolation Forest on the Imbalanced UNSW-NB15 Dataset

Syifaurachman Syifaurachman; Samso Supriyatna; Muhamad Ihsan Ashari

doi:10.30591/jpit.v11i2.10378

Anomaly-Based Network Intrusion Detection Using Isolation Forest on the Imbalanced UNSW-NB15 Dataset

Syifaurachman Syifaurachman, Samso Supriyatna, Muhamad Ihsan Ashari

Abstract

The increasing complexity of network traffic and the rapid evolution of cyber threats require adaptive intrusion detection systems capable of identifying anomalous behavior in large-scale network environments. Traditional signature-based detection methods are effective in recognizing known attack patterns but are limited in detecting emerging or zero-day threats. Consequently, anomaly-based approaches using machine learning have become an important research direction in modern intrusion detection systems. One of the widely used unsupervised algorithms for anomaly detection is Isolation Forest, which identifies anomalies by isolating observations through random partitioning mechanisms.This study investigates the capability of the Isolation Forest algorithm for intrusion detection using the UNSW-NB15 dataset under imbalanced data conditions. The dataset consists of 2,540,044 network traffic records with approximately 87% normal traffic and 13% attack traffic. A quantitative experimental approach was applied, including data preprocessing, model development using Isolation Forest, and performance measurement using several evaluation metrics, namely Accuracy, Precision, Recall, F1-score, Receiver Operating Characteristic Area Under the Curve (ROC-AUC), and Precision-Recall Area Under the Curve (PR-AUC).The experimental results show that the model achieved an overall accuracy of 79.3% and a ROC-AUC value of 0.6302, indicating moderate capability in distinguishing between normal and malicious traffic. However, the PR-AUC value of 0.1710 reveals limited sensitivity in detecting minority attack instances under highly imbalanced conditions. These findings highlight that evaluation of intrusion detection systems under imbalanced data should not rely solely on accuracy-based metrics. Isolation Forest can serve as a baseline anomaly detection mechanism, but additional strategies are required to improve detection sensitivity in real-world intrusion detection environments.

Keywords

imbalanced dataset, intrusion detection, isolation forest, unsupervised learning

Full Text:

References

V. Chandola, A. Banerjee, and V. Kumar, “Anomaly detection,” ACM Comput. Surv., vol. 41, no. 3, pp. 1–22, 2009, doi: 10.1145/1541880.1541882.

A. L. Buczak and E. Guven, “A Survey of Data Mining and Machine Learning Methods for Cyber Security Intrusion Detection,” IEEE Communications Surveys and Tutorials, vol. 18, no. 2, pp. 1153–1176, Apr. 2016, doi: 10.1109/COMST.2015.2494502.

F. T. Liu, K. M. Ting, and Z. H. Zhou, “Isolation forest,” in Proceedings - IEEE International Conference on Data Mining, ICDM, 2008, pp. 413–422. doi: 10.1109/ICDM.2008.17.

F. T. Liu, K. M. Ting, and Z. H. Zhou, “Isolation-based anomaly detection,” ACM Trans. Knowl. Discov. Data, vol. 6, no. 1, Mar. 2012, doi: 10.1145/2133360.2133363.

S. Hariri, M. C. Kind, and R. J. Brunner, “Extended Isolation Forest,” IEEE Trans. Knowl. Data Eng., vol. 33, no. 4, pp. 1479–1489, Apr. 2021, doi: 10.1109/TKDE.2019.2947676.

N. Moustafa and J. Slay, “UNSW-NB15: A Comprehensive Data set for Network Intrusion Detection systems (UNSW-NB15 Network Data Set).” [Online]. Available: https://cve.mitre.org/

S. Meftah, T. Rachidi, and N. Assem, “Network based intrusion detection using the UNSW-NB15 dataset,” International Journal of Computing and Digital Systems, vol. 8, no. 5, pp. 477–487, 2019, doi: 10.12785/ijcds/080505.

S. M. Kasongo and Y. Sun, “A deep learning method with wrapper based feature extraction for wireless intrusion detection system,” Comput. Secur., vol. 92, May 2020, doi: 10.1016/j.cose.2020.101752.

Z. Zoghi and G. Serpen, “Building an intrusion detection system on UNSW-NB15: Reducing the margin of error to deal with data overlap and imbalance,” Concurr. Comput., vol. 36, no. 25, Nov. 2024, doi: 10.1002/cpe.8242.

M. A. Ambusaidi, X. He, P. Nanda, and Z. Tan, “Building an intrusion detection system using a filter-based feature selection algorithm,” IEEE Transactions on Computers, vol. 65, no. 10, pp. 2986–2998, Oct. 2016, doi: 10.1109/TC.2016.2519914.

N. Kumar and S. Sharma, “A Hybrid Modified Deep Learning Architecture for Intrusion Detection System with Optimal Feature Selection,” Electronics (Switzerland), vol. 12, no. 19, Oct. 2023, doi: 10.3390/electronics12194050.

T. Kieu, B. Yang, C. Guo, and C. S. Jensen, “Outlier Detection for Time Series with Recurrent Autoencoder Ensembles.”

N. Sharma, N. S. Yadav, and S. Sharma, “Classification of UNSW-NB15 dataset using Exploratory Data Analysis using Ensemble Learning,” EAI Endorsed Transactions on Industrial Networks and Intelligent Systems, vol. 8, no. 29, 2021, doi: 10.4108/EAI.13-10-2021.171319.

“14. Enhancing Industrial Control System Security An Isolation Forest-based Anomaly Detection Model for Mitigating Cyber Threats”.

X. Tao, Y. Peng, F. Zhao, P. Zhao, and Y. Wang, “A parallel algorithm for network traffic anomaly detection based on Isolation Forest,” Int. J. Distrib. Sens. Netw., vol. 14, no. 11, Nov. 2018, doi: 10.1177/1550147718814471.

S. Khalifa, M. Marie, and W. Mohamed, “An Optimized Deep Learning Approach for Multiclass Anomaly Detection,” Information (Switzerland), vol. 17, no. 2, Feb. 2026, doi: 10.3390/info17020183.

X. Nong, K. Qin, and X. Xie, “Anomaly Detection in Imbalanced Network Traffic Using a ResCAE-BiGRU Framework,” Symmetry (Basel)., vol. 17, no. 12, Dec. 2025, doi: 10.3390/sym17122087.

D. Paolini, P. Dini, E. Soldaini, and S. Saponara, “One-Class Anomaly Detection for Industrial Applications: A Comparative Survey and Experimental Study,” Computers, vol. 14, no. 7, Jul. 2025, doi: 10.3390/computers14070281.

“19. Enhancing early attack detection novel hybrid density-based isolation forest for improved anomaly detection”.

P. Verma, D. O’Shea, T. Newe, N. Mehta, N. Bharot, and J. G. Breslin, “ABIDS-VEM: leveraging an equilibrium optimizer and data ramification in association with ensemble learning for anomaly-based intrusion detection system,” Journal of Supercomputing, vol. 81, no. 7, May 2025, doi: 10.1007/s11227-025-07292-w.

“21. Optimizing cloud-based intrusion detection systems through hybrid data sampling and feature selection for enhanced anomaly detection”.

K. Vani and S. P. Swornambiga, “Adaptive intrusion detection framework for enhanced cloud security in fog and edge computing environments,” International Journal of Advanced Technology and Engineering Exploration, vol. 11, no. 121, pp. 1613–1640, Dec. 2024, doi: 10.19101/IJATEE.2024.111100395.

“23. Honey-block Edge assisted ensemble learning model for intrusion detection and prevention using defense mechanism in IoT”.

M. Mohy-Eddine, A. Guezzaz, S. Benkirane, M. Azrour, and Y. Farhaoui, “An Ensemble Learning Based Intrusion Detection Model for Industrial IoT Security,” Big Data Mining and Analytics, vol. 6, no. 3, pp. 273–287, Sep. 2023, doi: 10.26599/BDMA.2022.9020032.

A. K. Shukla, S. Srivastav, S. Kumar, and P. K. Muhuri, “UInDeSI4.0: An efficient Unsupervised Intrusion Detection System for network traffic flow in Industry 4.0 ecosystem,” Eng. Appl. Artif. Intell., vol. 120, Apr. 2023, doi: 10.1016/j.engappai.2023.105848.

“26. Fusion-based anomaly detection system using modified isolation forest for internet of things”.

L. Qi, Y. Yang, X. Zhou, W. Rafique, and J. Ma, “Fast Anomaly Identification Based on Multiaspect Data Streams for Intelligent Intrusion Detection Toward Secure Industry 4.0,” IEEE Trans. Industr. Inform., vol. 18, no. 9, pp. 6503–6511, Sep. 2022, doi: 10.1109/TII.2021.3139363.

DOI: https://doi.org/10.30591/jpit.v11i2.10378