A Deep Dive into Deep Learning and Machine Learning: A Review Study
DOI:
https://doi.org/10.29304/jqcsm.2025.17.32381Keywords:
Deep Learning, Machine Learning, Neural NetworksAbstract
The current comprehensive review paper focuses on the analysis of the differences and similarities between deep learning and traditional machine learning. This paper analyses the architectural differences between DL and ML systems and include a comparison of the suitability of the two for tasks such as computer vision, NLP, speech recognition and Reinforcement Learning. The differences of data demand, training procedures, and computational power necessary for each method are discussed. The paper also focuses on the difficulties of hyperparameter tuning / optimization and research methodologies in both paradigms: ML and DL. Additionally, it introduces aspects of deployment and a range of size and speed values for models, inference, and resources. Problems as to how interconnections between ML and DL can provide the greatest performance are analyzed along with the viability of the mentioned integrated models. Moving to the future advancements, we discuss about the trends and the future work in both the fields which gives some idea about the upcoming building of artificial intelligence. In this extensive analysis, we hope that the target audience of researchers, practitioners, and decision-makers will be equipped with the knowledge regarding when and how to appropriately use ML and DL approaches while considering the complexity of a given task, availability of data, and computing resources. This work in turn, adds its voice to the ever-increasing discussion in the AI world with regards to the relative advantages and suitable uses of these strong methods. This review quantitatively compares performance (e.g., 92% DL vs. 85% ML accuracy in image tasks) and resource requirements across domains.
Downloads
References
J. Platt, "Sequential minimal optimization: A fast algorithm for training support vector machines.," in "Microsoft Research Technical Report " Microsoft, USA, 1998. [Online]. Available: https://www.microsoft.com/en-us/research/publication/sequential-minimal-optimization-a-fast-algorithm-for-training-support-vector-machines/
D. E. Rumelhart, G. E. Hinton, and R. J. Williams, "Learning representations by back-propagating errors," Nature, vol. 323, no. 6088, pp. 533-536, 1986/10/01 1986, doi: 10.1038/323533a0.
S. Hong and J. Chae, "Active Learning With Multiple Kernels," IEEE Transactions on Neural Networks and Learning Systems, vol. 33, no. 7, pp. 2980-2994, 2022, doi: 10.1109/TNNLS.2020.3047953.
T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Second Edition (Springer Series in Statistics). New York, NY: Springer 2019.
E. Scornet, "Trees, forests, and impurity-based variable importance in regression," Annales de l'institut Henri Poincare (B) Probability and Statistics, vol. 59, no. 1, pp. 21-52, 2023, doi: 10.1214/21-AIHP1240.
A. Vaswani et al., "Attention is All you Need," in Advances in Neural Information Processing Systems, vol. 30, I. Guyon et al. Eds. Glasgow, Scotland, United Kingdom: Curran Associates, Inc., 2017.
S. Wang, B. Z. Li, M. Khabsa, H. Fang, and H. Ma, "Linformer: Self-Attention with Linear Complexity," arXiv preprint vol. arXiv:2006.04768v3, 2020, doi: https://doi.org/10.48550/arXiv.2006.04768.
J. Gupta, S. Pathak, and G. Kumar, "Deep Learning (CNN) and Transfer Learning: A Review," Journal of Physics: Conference Series, vol. 2273, no. 1, pp. 12029-12029, 2022, doi: 10.1088/1742-6596/2273/1/012029.
G. I. Parisi, R. Kemker, J. L. Part, C. Kanan, and S. Wermter, "Continual lifelong learning with neural networks: A review," Neural Networks, vol. 113, pp. 54-71, 2019/05/01/ 2019, doi: https://doi.org/10.1016/j.neunet.2019.01.012.
H. Li, Machine Learning Methods. Singapore: Springer Nature Singapore, 2024.
Y. Xiong and R. Zuo, "Robust Feature Extraction for Geochemical Anomaly Recognition Using a Stacked Convolutional Denoising Autoencoder," Mathematical Geosciences, vol. 54, no. 3, pp. 623-644, 2022, doi: 10.1007/s11004-021-09935-z.
F. Lyu et al., "Feature Representation Learning for Click-through Rate Prediction: A Review and New Perspectives," arXiv preprint, vol. arXiv:2302.02241, 2023, doi: 10.48550/arXiv.2302.02241.
I. Goodfellow, Y. Bengio, and A. Courville, Deep learning. Cambridge, MA, UK: MIT Press, 2016.
R. Heckel and F. F. Yilmaz, "Early stopping in deep networks: Double descent and how to eliminate it.," arXiv preprint, vol. arXiv:2007.10099, 2021, doi: https://doi.org/10.48550/arXiv.2007.10099.
L. Ouyang et al., " InstructGPT: Training language models to follow instructions with human feedback," arXiv preprint, vol. arXiv:2203.02155v1 2022, doi: https://doi.org/10.48550/arXiv.2203.02155.
A. Liaw and M. Wiener, "Classification and Regression by RandomForest," R news, vol. 2, no. 3, pp. 18-22, 11/30 2001.
Z. H. Zhou, Ensemble methods: Foundations and algorithms. CRC press, 2012, pp. 1-218.
H. A. Salman, A. Kalakech, and A. Steiti, "Random Forest Algorithm Overview," Babylonian Journal of Machine Learning, vol. 2024, pp. 69-79, 2024, doi: 10.58496/bjml/2024/007.
Y. LeCun, Y. Bengio, and G. Hinton, "Deep learning," Nature, vol. 521, no. 7553, pp. 436-444, 2015/05/01 2015, doi: 10.1038/nature14539.
J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, "You Only Look Once: Unified, Real-Time Object Detection," presented at the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016. [Online]. Available: https://doi.ieeecomputersociety.org/10.1109/CVPR.2016.91.
Y. Tay et al., "Long Range Arena: A Benchmark for Efficient Transformers," arXiv preprint, vol. arXiv:2011.04006v1, 2020, doi: https://doi.org/10.48550/arXiv.2011.04006.
O. Sagi and L. Rokach, "Ensemble learning: A survey," WIREs Data Mining and Knowledge Discovery, vol. 8, no. 4, p. e1249, 2018, doi: https://doi.org/10.1002/widm.1249.
D. M. C. Nieto, E. A. P. Quiroz, and M. A. Cano Lengua, "A systematic literature review on support vector machines applied to regression," in 2021 IEEE Sciences and Humanities International Research Conference (SHIRCON), 2021 2021: IEEE, pp. 1-4, doi: 10.1109/SHIRCON53068.2021.9652268.
M. I. Jordan and T. M. Mitchell, "Machine learning: Trends, perspectives, and prospects," Science, vol. 349, no. 6245, pp. 255-260, 2015, doi: doi:10.1126/science.aaa8415.
M. T. Ribeiro, S. Singh, and C. Guestrin, ""Why Should I Trust You?": Explaining the Predictions of Any Classifier," presented at the Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, California, USA, 2016. [Online]. Available: https://doi.org/10.1145/2939672.2939778.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Hind Khalid
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
