A Comprehensive Survey on Multi-Object Tracking: From Classical to Deep Learning-Based Methods
DOI:
https://doi.org/10.29304/jqcsm.2026.18.32838Keywords:
Deep Learning, Multi-Object Tracking (MOT), Computer Vision, Kalman Filter, Evaluation Metrics, Sensor FusionAbstract
Computer Vision is one of the most prominent fields within the area of artificial intelligence, as it aims to provide computers with the same visual understanding as humans. Object detection and tracking play a critical role in the development of intelligent systems, as they allow for intelligent decision-making from the information gathered from vision systems. Multiple Object Tracking (MOT) is one of the most challenging yet important problems within the field of computer vision. MOT requires the system to detect and then keep track of multiple objects in a sequence of video frames. Some of the applications of MOT include autonomous vehicles, security systems, healthcare, and sports. The use of deep learning techniques, such as Convolutional Neural Networks (CNN), Transformers, and self-supervised learning, has improved MOT techniques. Additionally, the fusion of deep learning and sensor data with classical methods like the Kalman and Particle Filter has created methods with a good balance between accuracy and speed. Despite the improvements in these methods, there are still some challenges to the optimal performance of MOT algorithms. A survey of these methods, including classical, deep learning, and hybrid methods, as well as an overview of the datasets, metrics, applications, and current state-of-the-art methods and their evaluations, is presented in this paper.
Downloads
References
N. Zhou, L. Lau, R. Bai, and T. Moore, "A genetic optimization resampling-based particle filtering algorithm for indoor target tracking," Remote Sensing, vol. 13, no. 1, pp. 1–22, Jan. 2021, doi: 10.3390/rs13010132.
L. Liu, Y. Dai, and Z. Liu, "Real-time pose estimation and motion tracking for motion performance using deep learning models," Journal of Intelligent Systems, vol. 33, no. 1, Jan. 2024, doi: 10.1515/jisys-2023-0288.
J. Shen and H. Yang, "Multi-object Tracking Model Based on Detection Tracking Paradigm in Panoramic Scenes," Applied Sciences, vol. 14, no. 10, May 2024, doi: 10.3390/app14104146.
H. Liu, "Research Advanced in 2D Multi-object Tracking based on Deep Learning," Highlights in Science, Engineering and Technology, vol. 72, 2024, doi: 10.54097/h7jpeh64.
W. Farhat, O. Ben Rhaiem, H. Faiedh, and C. Souani, "Pedestrian detection and tracking using an enhanced YOLOv9 model for automotive vehicles," Measurement, vol. 254, Oct. 2025, doi: 10.1016/j.measurement.2025.118009.
X. Xiao and X. Feng, "Multi-object Pedestrian Tracking Using Improved YOLOv8 and OC-SORT," Sensors, vol. 23, no. 20, p. 8439, Oct. 2023, doi: 10.3390/s23208439.
[7] Y. Liu, B. Li, D. Liu, and Q. Duan, "Adaptive spatial aggregation and viewpoint alignment for three-dimensional online multiple fish tracking," Computers and Electronics in Agriculture, vol. 236, Sep. 2025, doi: 10.1016/j.compag.2025.110408.
K. Salazar and H. V. Li, "Multi-object Tracking Sensor Fusion: Radar and LiDAR Integration with Kalman Filtering Techniques: Beyond Sight: Radar-LiDAR Fusion for Robust Multi-Object Tracking," Master of Science thesis, Department of Electrical Engineering, Uppsala University, Uppsala, Sweden, 2024.
X. Wang, K. Ma, Q. Liu, Y. Zou, and Y. Fu, "Multi-object tracking in the dark," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024, pp. 382-392.
L. Sun, B. Li, D. Gao, and B. Fan, "Adaptive multi-object tracking algorithm based on split trajectory," The Journal of Supercomputing, vol. 80, no. 15, pp. 22287–22314, Oct. 2024, doi: 10.1007/s11227-024-06285-5.
E. E. Gohari, R. Ramezani, and A. Baraani, "Adaptive probabilistic multi-model framework for robust 3D multi-object tracking under uncertainty," Expert Systems with Applications, vol. 272, May 2025, doi: 10.1016/j.eswa.2025.126719.
G. S. Walia, A. Kumar, A. Saxena, K. Sharma, and K. Singh, "Robust object tracking with crow search optimized multi-cue particle filter," Pattern Analysis and Applications, vol. 23, no. 3, pp. 1127–1144, Aug. 2020, doi: 10.1007/s10044-019-00847-7.
A. Ratre, "GMM-based Imbalanced Fractional Whale Particle Filter for Multiple Object Tracking in Surveillance Videos," International Journal of Computer Network and Information Security, vol. 17, no. 2, pp. 34–50, Apr. 2025, doi: 10.5815/ijcnis.2025.02.03.
M. A. Awal, M. A. R. Refat, F. Naznin, and M. Z. Islam, "A particle filter based visual object tracking: A systematic review of current trends and research challenges," International Journal of Advanced Computer Science and Applications, vol. 14, no. 11, pp. 1291–1300, 2023, doi: 10.14569/IJACSA.2023.01411131.
X. Bian, W. Zhao, L. Tang, H. Zhao, and X. Mei, "An Enhanced Particle Filtering Method Leveraging Particle Swarm Optimization for Simultaneous Localization and Mapping in Mobile Robots Navigating Unknown Environments," Applied Sciences, vol. 14, no. 20, Oct. 2024, doi: 10.3390/app14209426.
S. Chen and X. Ren, "Multi-object tracking algorithm based on improved TrackFormer," in Association for Computing Machinery (ACM) International Conference Proceeding Series, Nov. 2023, pp. 194–202, doi: 10.1145/3652628.3652661.
S. Wu, A. Hadachi, C. Lu, and D. Vivet, "MOTT: A new model for multi-object tracking based on green learning paradigm," AI Open, vol. 4, pp. 145–153, Jan. 2023, doi: 10.1016/j.aiopen.2023.09.002.
G. Zhao, F. Meng, C. Yang, H. Wei, D. Zhang, and Z. Zheng, "A review of object tracking based on deep learning," Neurocomputing, vol. 651, p. 130988, Oct. 2025, doi: 10.1016/j.neucom.2025.130988.
R. Zeng and M. Ramezani, "A dynamic-confidence 3D multi-object tracking method based on spatio-temporal association," Measurement, vol. 256, Dec. 2025, doi: 10.1016/j.measurement.2025.117864.
L. Zhang et al., "UniRTL: A universal RGBT and low-light benchmark for object tracking," Pattern Recognition, vol. 158, Feb. 2025, doi: 10.1016/j.patcog.2024.110984.
A. Prasannakumar and D. Mishra, "Deep Efficient Data Association for Multi-object Tracking: Augmented with SSIM-Based Ambiguity Elimination," Journal of Imaging, vol. 10, no. 7, Jul. 2024, doi: 10.3390/jimaging10070171.
J. Cao, Y. Peng, H. Wei, L. Mo, L. Fan, and L. Wang, "KPTr: Key point transformer for LiDAR-based 3D object detection," Measurement, vol. 242, Jan. 2025, doi: 10.1016/j.measurement.2024.115820.
H.-W. Huang et al., "Iterative scale-up expansion and deep feature association for multi-object tracking in sports," in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2024, pp. 163-172.
S. P. K. Reddy, J. Harikiran, and B. S. Chandana, "Deep CNN Based Multi-object Detection and Tracking in Video Frames with Mean Distributed Feature Set," in Procedia Computer Science, Elsevier B.V., 2024, pp. 723–734, doi: 10.1016/j.procs.2024.04.069.
M. Li, M. Liu, W. Zhang, W. Guo, E. Chen, and C. Zhang, "A Robust Multi-Camera Vehicle Tracking Algorithm in Highway Scenarios Using Deep Learning," Applied Sciences, vol. 14, no. 16, Aug. 2024, doi: 10.3390/app14167071.
L. Cheng, A. Sengupta, and S. Cao, "Deep Learning-Based Robust Multi-object Tracking via Fusion of mmWave Radar and Camera Sensors," IEEE Transactions on Intelligent Transportation Systems, vol. 25, no. 11, pp. 17218–17233, 2024, doi: 10.1109/TITS.2024.3421339.
M. Li, Y. Zhang, Y. Jia, and Y. Yang, "Advancing multi-object tracking through occlusion-awareness and trajectory optimization," Knowledge-Based Systems, vol. 310, Feb. 2025, doi: 10.1016/j.knosys.2024.112930.
D. Wu, W. Han, T. Wang, X. Dong, X. Zhang, and J. Shen, "Referring Multi-object Tracking," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023, pp. 14633–14642, doi: 10.1109/CVPR52729.2023.01406.
C. Yang, A. Xu, Y. Li, and X. Wu, "Multi-pedestrian tracking based on iterative filtering and observation discrimination," Signal, Image and Video Processing, vol. 19, no. 11, Nov. 2025, doi: 10.1007/s11760-025-04504-x.
W. Cao, X. Wang, X. Liu, and Y. Xu, "A deep learning framework for multi-object tracking in team sports videos," IET Computer Vision, vol. 18, no. 5, pp. 574–590, Aug. 2024, doi: 10.1049/cvi2.12266.
F. Ngeni, J. Mwakalonge, and S. Siuhi, "Solving traffic data occlusion problems in computer vision algorithms using DeepSORT and quantum computing," Journal of Traffic and Transportation Engineering (English Edition), vol. 11, no. 1, pp. 1–15, Feb. 2024, doi: 10.1016/j.jtte.2023.05.006.
Y. Li, L. Zhan, L. Wu, H. Liu, J. Min, and X. Wang, "Re-identification assistance and multi-stage association for pedestrian multi-object tracking," Scientific Reports, vol. 15, no. 1, Dec. 2025, doi: 10.1038/s41598-025-07276-z.
J. Liu, "RainyTrack: Enhancing Object Tracking in Adverse Weather Conditions with Siamese Networks," in Association for Computing Machinery (ACM) International Conference Proceeding Series, Jan. 2024, pp. 308–315, doi: 10.1145/3647649.3647698.
H. Pang, R. Ma, J. Su, C. Liu, Y. Gao, and Q. Jin, "Blinding and blurring the multi-object tracker with adversarial perturbations," Neural Networks, vol. 176, Aug. 2024, doi: 10.1016/j.neunet.2024.106331.
K. Ren, C. Hu, H. Xi, Y. Li, J. Fan, and L. Liu, "SemTG-Track: Multi-modal fine-grained semantic-unit temporal guidance for multi-object tracking," Expert Systems with Applications, vol. 289, Sep. 2025, doi: 10.1016/j.eswa.2025.128359.
X. Feng, Y. Liu, H. Yang, X. Jiao, and Z. Liu, "Self-supervised multi-object tracking based on metric learning," Complex and Intelligent Systems, vol. 10, no. 5, pp. 7077–7088, Oct. 2024, doi: 10.1007/s40747-024-01475-3.
X. Wang, Z. Sun, A. Chehri, G. Jeon, and Y. Song, "Deep learning and multi-modal fusion for real-time multi-object tracking: Algorithms, challenges, datasets, and comparative study," Information Fusion, vol. 105, May 2024, doi: 10.1016/j.inffus.2024.102247.
M. Á. Naya, E. Sanjurjo, A. J. Rodríguez, and J. Cuadrado, "Kalman filters based on multi-body models: linking simulation and real world. A comprehensive review," Multibody System Dynamics, vol. 58, no. 3-4, pp. 241–280, Aug. 2023, doi: 10.1007/s11044-023-09893-w.
P. Gao, S. M. Li, F. Wang, H. Fujita, H. Aljuaid, and R. Y. Yuan, "Learning multi-level graph attentional representation for thermal infrared object tracking," Engineering Applications of Artificial Intelligence, vol. 155, Sep. 2025, doi: 10.1016/j.engappai.2025.110957.
Y. Wang, Y. Qing, K. Huang, C. Dang, and Z. Wu, "Preformer MOT: A transformer-based approach for multi-object tracking with global trajectory prediction," Fundamental Research, 2025, doi: 10.1016/j.fmre.2024.06.015.
R. K. Saha, R. Singhal, and M. Nambiar, "CAMOT: Content Aware Multi-object Tracking," in ACM International Conference Proceeding Series, Association for Computing Machinery, Oct. 2023, doi: 10.1145/3639856.3639866.
X. Guo, Y. Zheng, and D. Wang, "PMTrack: Multi-object Tracking with Motion-Aware," in Proceedings of the Asian Conference on Computer Vision (ACCV), 2024, pp. 3091-3106.
R. N. Razak and H. N. Abdullah, "Improving multi-object detection and tracking with deep learning, DeepSORT, and frame cancellation techniques," Open Engineering, vol. 14, no. 1, Jan. 2024, doi: 10.1515/eng-2024-0056.
M. Sukkar, M. Shukla, D. Kumar, V. C. Gerogiannis, A. Kanavos, and B. Acharya, "Enhancing Pedestrian Tracking in Autonomous Vehicles by Using Advanced Deep Learning Techniques," Information (Switzerland), vol. 15, no. 2, Feb. 2024, doi: 10.3390/info15020104.
I. Daramouskas, D. Meimetis, V. Patrinopoulou, V. Lappas, V. Kostopoulos, and V. Kapoulas, "Camera-Based Local and Global Target Detection, Tracking, and Localization Techniques for UAVs," Machines, vol. 11, no. 2, Feb. 2023, doi: 10.3390/machines11020315.
S. Meng, D. Shao, J. Guo, and S. Gao, "Tracking without label: Unsupervised multiple object tracking via contrastive similarity learning," in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Oct. 2023, pp. 16264-16273.
Y. Cui, C. Zeng, X. Zhao, Y. Yang, G. Wu, and L. Wang, "SportsMOT: A large multi-object tracking dataset in multiple sports scenes," in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Oct. 2023, pp. 9921-9931.
Y. Zhu, Z. Liu, K. Li, Y. Li, X. Qi, and N. Ding, "Research on Multi-Objective Pedestrian Tracking Algorithm Based on Full-Size Feature Fusion," IEEE Access, vol. 13, pp. 39107–39116, 2025, doi: 10.1109/ACCESS.2025.3546818.
Z. Meng, X. Xia, R. Xu, W. Liu, and J. Ma, "HYDRO-3D: Hybrid Object Detection and Tracking for Cooperative Perception Using 3D LiDAR," IEEE Transactions on Intelligent Vehicles, vol. 8, no. 8, pp. 4069–4080, Aug. 2023, doi: 10.1109/TIV.2023.3282567.
N. Senel, K. Kefferpütz, K. Doycheva, and G. Elger, "Multi-Sensor Data Fusion for Real-Time Multi-object Tracking," Processes, vol. 11, no. 2, Feb. 2023, doi: 10.3390/pr11020501.
K. Vinoth and P. Sasikumar, "Multi-sensor fusion and segmentation for autonomous vehicle multi-object tracking using deep Q networks," Scientific Reports, vol. 14, no. 1, Dec. 2024, doi: 10.1038/s41598-024-82356-0.
S. Ren, "Multi-camera association tracking algorithm for pedestrian target based on difference image," Systems and Soft Computing, vol. 7, Dec. 2025, doi: 10.1016/j.sasc.2025.200282.
Y. Guo, Y. Li, S. Wang, K. Sun, M. Liu, and Z. Wang, "Pedestrian multi-object tracking combining appearance and spatial characteristics," Expert Systems with Applications, vol. 272, May 2025, doi: 10.1016/j.eswa.2025.126772.
Q. Hu, A. Scott, C. Yeung, and K. Fujii, "Basketball-SORT: an association method for complex multi-object occlusion problems in basketball multi-object tracking," Multimedia Tools and Applications, vol. 83, no. 38, pp. 86281–86297, Oct. 2024, doi: 10.1007/s11042-024-20360-2.
F. Li, Y. Chen, M. Hu, M. Luo, and G. Wang, "Helmet-Wearing Tracking Detection Based on StrongSORT," Sensors, vol. 23, no. 3, Feb. 2023, doi: 10.3390/s23031682.
V. K. Gupta, P. Rathore, P. Prajapati, and R. Shukla, "Automated PPE Detection Using YOLOv8 for Real-Time Workplace Safety Monitoring," International Journal of Innovative Science and Research Technology, pp. 2228–2233, May 2025, doi: 10.38124/ijisrt/25apr1699.
S. Lu et al., "Surgical instrument posture estimation and tracking based on LSTM," ICT Express, vol. 10, no. 3, pp. 465–471, Jun. 2024, doi: 10.1016/j.icte.2024.01.002.
M. Shili, S. Jayasingh, and S. Hammedi, "Advanced Customer Behavior Tracking and Heatmap Analysis with YOLOv5 and DeepSORT in Retail Environment," Electronics, vol. 13, no. 23, Dec. 2024, doi: 10.3390/electronics13234730.
T. D. Nguyen et al., "A retail store customer behavior analysis system: Design and implementation," in Artificial Intelligence Applications and Innovations (AIAI 2024), Cham: Springer, 2024, pp. 305–318, doi: 10.1007/978-3-031-63223-5_23.
R. Irampaye, J. Chen, X. Zhu, and Y. Zhou, "Visual multi-object tracking by interaction network," in Proceedings of the 2024 7th International Symposium on Autonomous Systems (ISAS), Institute of Electrical and Electronics Engineers (IEEE), 2024, doi: 10.1109/ISAS61044.2024.10552490.
S. Li, H. Ren, Y. Cao, et al., "A Review of Multi-Object Tracking in Recent Times," IET Computer Vision, vol. 18, no. 1, pp. 1–25, Feb. 2024, doi: 10.1049/cvi2.12231.
A. Ramachandran and A. K. Sangaiah, "A review on object detection in unmanned aerial vehicle surveillance," International Journal of Cognitive Computing in Engineering, vol. 2, pp. 215–228, 2021, doi: 10.1016/j.ijcce.2021.11.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 BATOOL B. ABOUD, Hikmat Z. Neima, Maytham Alabbas
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
