A Dual Hybrid Approach for Log Anomaly Detection using Deep Learning
DOI:
https://doi.org/10.29304/jqcsm.2026.18.22645Keywords:
Deep Learning, Log Anomaly detection, CNNs, Bi-LSTMAbstract
The growing complexity of computer software and the explosive growth in the log data have made the detection of anomalies and the identification of problems in the system from the massive logs a major research field. However, existing log anomaly detection techniques cannot identify context-dependent semantic connections in unstructured logs and do not have explicit decision making processes and therefore they cannot be used in very dynamic systems. In this paper, we propose a dual hybrid technique for the anomaly detection task including the feature representation and the classification stages. During the feature representation stage, the statistical TF-IDF technique is used to extract features and merge them with the semantic representations of SBERT to obtain comprehensive representations with both statistical importance and semantic significance. In the classification stage, the hybrid deep learning strategy is used consisting of Convolutional Neural Networks (CNNs) for local pattern extraction and Bidirectional Long Short-Term Memory (Bi-LSTM) network for dealing with temporal relations in the data. This helps to improve the accuracy and effectiveness of the model in detecting anomalies in system logs. The approach has been trained and tested on the commonly used HDFS and BGL datasets. Experimental results showed that the proposed approach achieved an anomaly detection accuracy of up to 0.9989 and therefore demonstrated the potential and effectiveness of this approach compared with previous methodologies.
Downloads
References
Z. T. M. Al-Ta’i and S. M. Sadoon, “Visual cryptography based on chaotic logistic map in multi-cloud,” in AIP Conference Proceedings, 2024, vol. 3097, no. 1.
M. Alabadi and Y. Celik, “Anomaly Detection for Cyber-Security Based on Convolution Neural Network: A survey,” HORA 2020 - 2nd Int. Congr. Human-Computer Interact. Optim. Robot. Appl. Proc., no. January, 2020, doi: 10.1109/HORA49412.2020.9152899.
M. Landauer, S. Onder, F. Skopik, and M. Wurzenberger, “Deep learning for anomaly detection in log data: A survey,” Mach. Learn. with Appl., vol. 12, no. March, p. 100470, 2023, doi: 10.1016/j.mlwa.2023.100470.
R. Foorthuis, “On the nature and types of anomalies: A review of deviations in data,” SN Computer Science, vol. 12, no. 4, 2021, doi: 10.1007/s41060-021-00265-1.
C. Sánchez-Zas, X. Larriva-Novo, V. A. Villagrá, M. S. Rodrigo, and J. I. Moreno, “Design and Evaluation of Unsupervised Machine Learning Models for Anomaly Detection in Streaming Cybersecurity Logs,” Mathematics, vol. 10, no. 21, 2022, doi: 10.3390/math10214043.
Z. A. Khan, D. Shin, D. Bianculli, and L. C. Briand, “Impact of log parsing on deep learning-based anomaly detection,” Empirical Software Engineering, vol. 29, no. 6, 2024, doi: 10.1007/s10664-024-10533-w.
M. Goldstein and S. Uchida, “Behavior Analysis Using Unsupervised Anomaly Detection,” 10th Jt. Work. Mach. Percept. Robot., no. October, 2014.
A. B. Nassif, M. A. Talib, Q. Nasir, and F. M. Dakalbab, “Machine Learning for Anomaly Detection: A Systematic Review,” IEEE Access, vol. 9, pp. 78658–78700, 2021, doi: 10.1109/ACCESS.2021.3083060.
T. Rajendran, N. Mohamed Imtiaz, K. Jagadeesh, and B. Sampathkumar, “Cybersecurity Threat Detection Using Deep Learning and Anomaly Detection Techniques,” 2024 Int. Conf. Knowl. Eng. Commun. Syst. ICKECS 2024, vol. 1, pp. 1–7, 2024, doi: 10.1109/ICKECS61492.2024.10617347.
M. Du, F. Li, G. Zheng, and V. Srikumar, “Deeplog: Anomaly detection and diagnosis from system logs through deep learning,” in Proceedings of the 2017 ACM SIGSAC conference on computer and communications security, 2017, pp. 1285–1298.
Z. Liu, T. Qin, X. Guan, H. Jiang, and C. Wang, “An integrated method for anomaly detection from massive system logs,” IEEE Access, vol. 6, pp. 30602–30611, 2018.
W. Meng et al., “Loganomaly: Unsupervised detection of sequential and quantitative anomalies in unstructured logs.,” in IJCAI, 2019, vol. 19, no. 7, pp. 4739–4745.
C. Zhang et al., “LayerLog: Log sequence anomaly detection based on hierarchical semantics,” Appl. Soft Comput., vol. 132, p. 109860, 2023, doi: 10.1016/j.asoc.2022.109860.
Y. Duan et al., “LogEDL: Log Anomaly Detection via Evidential Deep Learning,” Appl. Sci., vol. 14, no. 16, pp. 1–18, 2024, doi: 10.3390/app14167055.
A. Aziz and K. Munir, “Anomaly Detection in Logs Using Deep Learning,” IEEE Access, vol. 12, no. November, pp. 176124–176135, 2024, doi: 10.1109/ACCESS.2024.3506332.
R. Jassim, “Review of Computer Engineering Research Artificial intelligence methods for identification of ADHD in children based on EEG signals Keyword s,” vol. 12, no. 2, pp. 80–93, 2025, doi: 10.18488/76. v12i2.4217.
A. Falini, “A review on the selection criteria for the truncated SVD in Data Science applications,” J. Comput. Math. Data Sci., vol. 5, p. 100064, 2022.
G. Haixiang, L. Yijing, J. Shang, G. Mingyun, H. Yuanyue, and G. Bing, “Learning from class-imbalanced data: Review of methods and applications,” Expert Systems with Applications, vol. 73, pp. 220–239, 2017.
J. da Silva Freitas Junior and P. H. Pisani, “Performance and model complexity on imbalanced datasets: An experimental comparison of cost-sensitive and resampling methods,” Proceedings of Machine Learning Research, vol. 183, 2022.
“Utility analysis about log data anomaly detection based on federated learning,” Applied Sciences, 2023.
W. Xu, L. Huang, A. Fox, D. Patterson, and M. I. Jordan, “Detecting large-scale system problems by mining console logs,” in Proceedings of the ACM SIGOPS 22nd symposium on Operating systems principles, 2009, pp. 117–132.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Harbi Mahmood Abas, Ziyad Tariq Mustafa Al-Ta'i, Shumoos jamal Rashid
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
