Optimizing the Architecture of Convolutional Neural Networks Using Modified Salp Swarm Algorithm
DOI:
https://doi.org/10.29304/jqcsm.2024.16.11450Keywords:
Deep learning, Convolutional neural networks, Hyperparameters optimization, Salp swarm algorithmAbstract
Deep learning is highly effective in dealing with complex tasks such as image classification and recognition. However, finding the optimal architecture's hyperparameters for Convolutional Neural Networks (CNNs) to achieve the best performance and parameter regularization can be challenging. Metaheuristic optimization algorithms can be utilized to find solutions in this context. In this research, a computerized CNN was adjusted using an improved Salp Swarm Algorithm (SSA) to enhance crucial CNN settings, like dropout rate, hidden units, learning rate, and batch size. The refined design was tested on two standard datasets. MNIST and Fashion MNIST. The outcomes displayed model performance achieving accuracy levels of 99.6% for MNIST and 94.08% for Fashion MNIST. This tuned system outperformed the existing practices by 0.2% and 0.04% for each dataset while also cutting down on computational expenses. The fusion of SSA with CNNs displayed adaptability and resilience opening up possibilities, in image classification and consistently delivering outstanding outcomes.
Downloads
References
Y. Bengio, I. Goodfellow, and A. Courville, "Deep learning (Vol. 1)," MIT Press Cambridge, MA, USA, vol. 22, pp. 23-24, 2017. DOI: 10.1007/s10710-017-9314-z
S. Dong, P. Wang, and K. Abbas, "A survey on deep learning and its applications," Computer Science Review, vol. 40, p. 100379, 2021. doi https://doi.org/10.1016/j.cosrev.2021.100379.
M. Abdulla and A. Marhoon, "Agriculture based on Internet of Things and Deep Learning," Iraqi Journal for Electrical and Electronic Engineering, vol. 18, no. 2, pp. 1-8, 2022. http://dx.doi.org/10.37917/ijeee.18.2.1.
A. Shrestha and A. Mahmood, "Review of deep learning algorithms and architectures," IEEE access, vol. 7, pp. 53040-53065, 2019. doi: http://dx.doi.org/10.1109/access.2019.2912200.
N. F. A. Hassan, A. A. Abed, and T. Y. Abdalla, "Face mask detection using deep learning on NVIDIA Jetson Nano," International Journal of Electrical & Computer Engineering (2088-8708), vol. 12, no. 5, 2022. doi: http://dx.doi.org/10.11591/ijece.v12i5.pp5427-5434.
Y. Wang, H. Zhang, and G. Zhang, "cPSO-CNN: An efficient PSO-based algorithm for fine-tuning hyper-parameters of convolutional neural networks," Swarm and Evolutionary Computation, vol. 49, pp. 114-123, 2019. doi: http://dx.doi.org/10.1016/j.swevo.2019.06.002.
A. Darwish, D. Ezzat, and A. E. Hassanien, "An optimized model based on convolutional neural networks and orthogonal learning particle swarm optimization algorithm for plant diseases diagnosis," Swarm and evolutionary computation, vol. 52, p. 100616, 2020, doi: http://dx.doi.org/10.1016/j.swevo.2019.100616.
L. Alzubaidi et al., "Review of deep learning: concepts, CNN architectures, challenges, applications, future directions," J Big Data, vol. 8, no. 1, p. 53, 2021, doi: 10.1186/s40537-021-00444-8.
S. Mirjalili, A. H. Gandomi, S. Z. Mirjalili, S. Saremi, H. Faris, and S. M. Mirjalili, "Salp Swarm Algorithm: A bio-inspired optimizer for engineering design problems," Advances in engineering software, vol. 114, pp. 163-191, 2017, doi: https://doi.org/10.1016/j.advengsoft.2017.07.002.
H. Zhang et al., "Differential evolution-assisted salp swarm algorithm with chaotic structure for real-world problems," Engineering with Computers, vol. 39, no. 3, pp. 1735-1769, 2023, doi: https://doi.org/10.1007/s00366-021-01545-x.
H. Shao, E. Ma, M. Zhu, X. Deng, and S. Zhai, "MNIST Handwritten Digit Classification Based on Convolutional Neural Network with Hyperparameter Optimization," Intelligent Automation & Soft Computing, vol. 36, no. 3, 2023. https://doi.org/10.32604/iasc.2023.036323
H. Xiao, K. Rasul, and R. Vollgraf, "Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms," arXiv preprint arXiv:1708.07747, 2017. doi: https://doi.org/10.48550/arXiv.1708.07747.
J.-H. Yoo, H.-i. Yoon, H.-G. Kim, H.-S. Yoon, and S.-S. Han, "Optimization of hyper-parameter for CNN model using genetic algorithm," in 2019 1st International conference on electrical, control and instrumentation engineering (ICECIE), Kuala Lumpur, Malaysia, 2019: IEEE, 2019, pp. 1-6, doi: 10.1109/ICECIE47765.2019.8974762.
Y. Guo, J.-Y. Li, and Z.-H. Zhan, "Efficient Hyperparameter Optimization for Convolution Neural Networks in Deep Learning: A Distributed Particle Swarm Optimization Approach," Cybernetics and Systems, vol. 52, no. 1, pp. 36-57, 2020, doi: 10.1080/01969722.2020.1827797.
N. Bacanin, T. Bezdan, E. Tuba, I. Strumberger, and M. Tuba, "Optimizing Convolutional Neural Network Hyperparameters by Enhanced Swarm Intelligence Metaheuristics," Algorithms, vol. 13, no. 3, 2020, doi: 10.3390/a13030067.
T. Serizawa and H. Fujita, "Optimization of convolutional neural network using the linearly decreasing weight particle swarm optimization," arXiv preprint arXiv:2001.05670, 2020, doi: https://doi.org/10.48550/arXiv.2001.05670.
K. Greeshma and J. V. Gripsy, "Image classification using HOG and LBP feature descriptors with SVM and CNN," Int J Eng Res Technol, vol. 8, no. 4, pp. 1-4, 2020.
Y. Ying, N. Zhang, P. He, and S. Peng, "Improving convolutional neural networks with competitive activation function," Security and Communication Networks, vol. 2021, pp. 1-9, 13 May 2021 2021, Art no. 1933490 doi: https://doi.org/10.1155/2021/1933490.
S. C. Nistor and G. Czibula, "IntelliSwAS: Optimizing deep neural network architectures using a particle swarm-based approach," Expert Systems with Applications, vol. 187, p. 115945, 2022. https://doi.org/10.1016/j.eswa.2021.115945
E. E. Moodie and D. A. Stephens, "Comment: Clarifying endogeneous data structures and consequent modelling choices using causal graphs," 2020. https://doi.org/10.1214/20-sts777.
J. R. Challapalli and N. Devarakonda, "A novel approach for optimization of convolution neural network with hybrid particle swarm and grey wolf algorithm for classification of Indian classical dances," Knowledge and Information Systems, vol. 64, no. 9, pp. 2411-2434, 2022. https://doi.org/10.1007/s10115-022-01707-3
I. D. Raji, H. Bello-Salau, I. J. Umoh, A. J. Onumanyi, M. A. Adegboye, and A. T. Salawudeen, "Simple deterministic selection-based genetic algorithm for hyperparameter tuning of machine learning models," Applied Sciences, vol. 12, no. 3, p. 1186, 2022. https://doi.org/10.3390/app12031186
O. Nocentini, J. Kim, M. Z. Bashir, and F. Cavallo, "Image classification using multiple convolutional neural networks on the fashion-MNIST dataset," Sensors, vol. 22, no. 23, p. 9544, 2022. https://doi.org/10.3390/s22239544.
S. R. Sumera, N. Anjum, and K. Vaidehi, "Implementation of CNN and ANN for Fashion-MNIST-Dataset using Different Optimizers," Indian Journal of Science and Technology, vol. 15, no. 47, pp. 2639-2645, 2022. https://doi.org/10.17485/ijst/v15i47.1821.
S.-Y. Shin, G. Jo, and G. Wang, "A Novel Method for Fashion Clothing Image Classification Based on Deep Learning," Journal of Information and Communication Technology, vol. 22, no. 1, pp. 127-148, 2023, doi: https://doi.org/10.32890/jict2023.22.1.6.
D. Liu, H. Ouyang, S. Li, C. Zhang, and Z.-H. Zhan, "Hyperparameters Optimization of Convolutional Neural Network Based on Local Autonomous Competition Harmony Search Algorithm," Journal of Computational Design and Engineering, p. qwad050, 2023. https://doi.org/10.1093/jcde/qwad050
F. M. Talaat and S. A. Gamel, "RL based hyper-parameters optimization algorithm (ROA) for convolutional neural network," Journal of Ambient Intelligence and Humanized Computing, vol. 14, no. 10, pp. 13349-13359, 2023. https://doi.org/10.1007/s12652-022-03788-y
N. Altwaijry and I. Al-Turaiki, "Arabic handwriting recognition system using convolutional neural network," Neural Computing and Applications, vol. 33, no. 7, pp. 2249-2261, 2021. https://doi.org/10.1007/s00521-020-05070-8
L. Ren, J. Dong, X. Wang, Z. Meng, L. Zhao, and M. J. Deen, "A data-driven auto-CNN-LSTM prediction model for lithium-ion battery remaining useful life," IEEE Transactions on Industrial Informatics, vol. 17, no. 5, pp. 3478-3487, 2020. https://doi.org/10.1109/tii.2020.3008223
A. H. Ashraf et al., "Weapons detection for security and video surveillance using cnn and YOLO-v5s," CMC-Comput. Mater. Contin, vol. 70, pp. 2761-2775, 2022. https://doi.org/10.32604/cmc.2022.018785
M. Zamir et al., "Face Detection & Recognition from Images & Videos Based on CNN & Raspberry Pi," Computation, vol. 10, no. 9, p. 148, 2022. https://doi.org/10.3390/computation10090148
C. Li, W. Xia, Y. Yan, B. Luo, and J. Tang, "Segmenting objects in day and night: Edge-conditioned CNN for thermal image semantic segmentation," IEEE Transactions on Neural Networks and Learning Systems, vol. 32, no. 7, pp. 3069-3082, 2020. https://doi.org/10.1109/tnnls.2020.3009373
M. A. Haque, A. Verma, J. S. R. Alex, and N. Venkatesan, "Experimental evaluation of CNN architecture for speech recognition," in First International Conference on Sustainable Technologies for Computational Intelligence: Proceedings of ICTSCI 2019, 2020: Springer, pp. 507-514. https://doi.org/10.1007/978-981-15-0029-9_40
R. S. Khudeyer and N. M. Almoosawi, "Combination of machine learning algorithms and Resnet50 for Arabic Handwritten Classification," Informatica, vol. 46, no. 9, 2023 https://doi.org/10.31449/inf.v46i9.4375.
J. Fregoso, C. I. Gonzalez, and G. E. Martinez, "Optimization of convolutional neural networks architectures using PSO for sign language recognition," Axioms, vol. 10, no. 3, p. 139, 2021.
X. Li, Z. Hu, M. Xu, Y. Wang, and J. Ma, "Transfer learning based intrusion detection scheme for Internet of vehicles," Information Sciences, vol. 547, pp. 119-135, 2021. https://doi.org/10.1016/j.ins.2020.05.130
O. D. Okey, D. C. Melgarejo, M. Saadi, R. L. Rosa, J. H. Kleinschmidt, and D. Z. Rodríguez, "Transfer learning approach to IDS on cloud IoT devices using optimized CNN," IEEE Access, vol. 11, pp. 1023-1038, 2023. https://doi.org/10.1109/access.2022.3233775
N. M. Almoosawi and R. S. Khudeyer, "ResNet-34/DR: a residual convolutional neural network for the diagnosis of diabetic retinopathy," Informatica, vol. 45, no. 7, 2021. https://doi.org/10.31449/inf.v45i7.3774
E. Abdulsaed, M. Alabbas, and R. Khudeyer, "Hyperparameter Optimization for Convolutional Neural Networks using the Salp Swarm Algorithm," Informatica, vol. 47, no. 9, 2023. https://doi.org/10.31449/inf.v47i9.5148
H. Faris, S. Mirjalili, I. Aljarah, M. Mafarja, and A. A. Heidari, "Salp swarm algorithm: theory, literature review, and application in extreme learning machines," Nature-inspired optimizers: theories, literature reviews and applications, pp. 185-199, 2020. https://doi.org/10.1007/978-3-030-12127-3_11
A. Q. Obaid and M. Alabbas, "Hybrid Variable-Length Spider Monkey Optimization with Good-Point Set Initialization for Data Clustering," Informatica, vol. 47, no. 8, 2023, doi: https://doi.org/10.31449/inf.v47i8.4872.
D. Liu, S. Zhang, B. Wang, and Z. Li, "Seagull algorithm based on good point set and dual hybrid strategy," in International Conference on Cloud Computing, Performance Computing, and Deep Learning (CCPCDL 2023), 2023, vol. 12712: SPIE, pp. 79-84. https://doi.org/10.1117/12.2678849
N. Bacanin, T. Bezdan, E. Tuba, I. Strumberger, and M. Tuba, "Optimizing convolutional neural network hyperparameters by enhanced swarm intelligence metaheuristics," Algorithms, vol. 13, no. 3, p. 67, 2020. https://doi.org/10.3390/a13030067
B. Rosner and D. Grove, "Use of the Mann–Whitney U‐test for clustered data," Statistics in medicine, vol. 18, no. 11, pp. 1387-1400, 1999. https://doi.org/10.1002/(sici)1097-0258(19990615)18:11<1387::aid-sim126>3.0.co;2-v
S. Ioannou, H. Chockler, A. Hammers, A. P. King, and A. s. D. N. Initiative, "A study of demographic bias in CNN-based brain MR segmentation," in International Workshop on Machine Learning in Clinical Neuroimaging, 2022: Springer, pp. 13-22, doi: https://doi.org/10.48550/arXiv.2208.06613.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Entesar H. Abdulsaed, Maytham Alabbas, Raidah S. Khudeyer
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
