Unusual Activity Detection in Surveillance Video Scene: Review
DOI:
https://doi.org/10.29304/jqcm.2021.13.3.848Keywords:
Abnormal activity, Human-computer interaction, activity analysis, Deep learning, handmade features, Automated detection, surveillance video scenesAbstract
Abnormal activity may indicate threats and risks to others. An anomaly can be defined as something that deviates from what is expected, common, or normal. Because it is difficult to continuously monitor public spaces, intelligent video surveillance is necessary, which can monitor human actions in real-time and categorize them as ordinary or exceptional, as well as create an alarm. Human activities in public and sensitive regions such as bus stations, airports, railway stations, malls, banks, universities, car parks, roads, and other regions can be monitored using visual surveillance to prevent crime, theft, terrorism, vandalism, accidents, and other suspicious activities. This makes video surveillance a key to increasing public security. The main objective of event discovery is to discover the occurrence of events and categorize them into normal or abnormal actions. This discovery requires identifying and tracking objects and then learning what is going around those tracked objects. Recent research is based on one of two technologies: handcrafted features and deep learning models. Handmade features are based on extracting low-level features, and their strength is based on selecting the best features, that produce the best results. After the success of deep learning techniques for classifying images, the researchers examined the ability of deep learning techniques to detect, which bypasses the manual step of feature extraction and works directly with images. This paper presents a survey of both handmade and deep learning models to detect abnormal events.
Downloads
Download data is not yet available.
References
[1] Sabokrou M., Fayyaz M., Fathy M., Moayed Z., & Klette R. (2018). Deep-anomaly: Fully convolutional neural network for fast anomaly detection in crowded scenes. Computer Vision and Image Understanding, 172, 88-97.
[2] Hamdi S., Bouindour S., Loukil K., Snoussi H., & Abid M. (2019). Hybrid deep learning and HOF for Anomaly Detection. In 2019 6th International Conference on Control, Decision and Information Technologies (CoDIT) (pp. 575-580).
[3] Cheng, K. W., Chen, Y. T., & Fang, W. H. (2015). Gaussian process regression-based video anomaly detection and localization with hierarchical feature representation. IEEE Transactions on Image Processing, 24(12), 5288-5301.
[4] Sargano, A. B., Angelov, P., & Habib, Z. (2017). A comprehensive review on handcrafted and learning-based action representation approaches for human activity recognition. applied sciences, 7(1), 110.
[5] Sabokrou, M., Fathy, M., Zhao, G., & Adeli, E. (2020). Deep End-to-End One-Class Classifier. IEEE Transactions on Neural Networks and Learning Systems.
[6] Nawaratne, R., Alahakoon, D., De Silva, D., & Yu, X. (2019). Spatiotemporal anomaly detection using deep learning for real-time video surveillance. IEEE Transactions on Industrial Informatics, 16(1), 393-402.
[7] OMahony, N., Campbell, S., Carvalho, A., Harapanahalli, S., Hernandez, G. V., Krpalkova, L. & Walsh, J. (2019, April). Deep learning vs. traditional computer vision. In Science and Information Conference (pp. 128-144). Springer, Cham.
[8] Zhao, Y., Deng, B., Shen, C., Liu, Y., Lu, H., & Hua, X. S. (2017). Spatio-temporal autoencoder for video anomaly detection. In Proceedings of the 25th ACM international conference on Multimedia (pp. 1933-1941).
[9] Xia, H., Li, T., Liu, W., Zhong, X., & Yuan, J. (2019). Abnormal event detection method in surveillance video based on temporal CNN and sparse optical flow. In Proceedings of the 2019 5th International Conference on Computing and Data Engineering (pp. 90-94).
[10] Wang, J., Ma, Y., Zhang, L., Gao, R. X., & Wu, D. (2018). Deep learning for smart manufacturing: Methods and applications. Journal of Manufacturing Systems, 48, 144-156.
[11] Lin, W., Hasenstab, K., Cunha, G. M., & Schwartzman, A. (2020). Comparison of handcrafted features and convolutional neural networks for liver MR image adequacy assessment. Scientific Reports, 10(1), 1-11.
[12] Yang, M., Rajasegarar, S., Erfani, S. M., & Leckie, C. (2019). Deep Learning and One-class SVM-based Anomalous Crowd Detection. In 2019 International Joint Conference on Neural Networks (IJCNN) (pp. 1-8). IEEE.
[13] Nguyen, T. N., & Meunier, J. (2019). Anomaly detection in a video sequence with appearance-motion correspondence. In Proceedings of the IEEE International Conference on Computer Vision (pp. 1273-1283).
[14] Huang, S., Huang, D., & Zhou, X. (2018). Learning multimodal deep representations for crowd anomaly event detection. Mathematical Problems in Engineering.
[15] Deepak, K., Vignesh, L. K. P., & Chandrakala, S. (2020). Autocorrelation of gradients-based violence detection in surveillance videos. ICT Express.
[16] Yang, Y., Li, L., Liu, Z., & Liu, G. (2020). Abnormal behavior recognition based on Spatio-temporal context. Journal of Information Processing Systems, 16(3), 612-628.
[17] Das, D., & Mishra, D. (2018). Unsupervised Anomalous Trajectory Detection for Crowded Scenes. In 2018 IEEE 13th International Conference on Industrial and Information Systems (ICIIS) (pp. 27-31). IEEE.
[18] Zhu, S., Hu, J., & Shi, Z. (2016). Local abnormal behavior detection based on optical flow and Spatio-temporal gradient. Multimedia Tools and Applications, 75(15), 9445-9459.
[19] Wang, T., Qiao, M., Zhu, A., Niu, Y., Li, C., & Snoussi, H. (2018). Abnormal event detection via covariance matrix for the optical flow-based feature. Multimedia Tools and Applications, 77(13), 17375-17395.
[20] Hu, X., Huang, Y., Duan, Q., Ci, W., Dai, J., & Yang, H. (2018). Abnormal event detection in crowded scenes using histogram of oriented contextual gradient descriptor. EURASIP Journal on Advances in Signal Processing, 2018(1), 54.
[21] Amraee, S., Vafaei, A., Jamshidi, K., & Adibi, P. (2018). Abnormal event detection in crowded scenes using one-class SVM. Signal, Image and Video Processing, 12(6), 1115-1123.
[22] Pang, G., Yan, C., Shen, C., Hengel, A. V. D., & Bai, X. (2020). Self-trained Deep Ordinal Regression for End-to-End Video Anomaly Detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 12173-12182).
[23] Ruff, L., Vandermeulen, R. A., Gornitz, N., Binder, A., Muller, E., & Kloft, M. (2019). Deep support vector data description for unsupervised and semi-supervised anomaly detection. In Proceedings of the ICML 2019 Workshop on Uncertainty and Robustness in Deep Learning, Long Beach, CA, USA.
[24] Ravanbakhsh, M., Nabi, M., Mousavi, H., Sangineto, E., & Sebe, N. (2018). Plug-and-play CNN for crowd motion analysis: An application in abnormal event detection. In 2018 IEEE Winter Conference on Applications of Computer Vision (WACV) (pp. 1689-1698).
[25] Gkountakos, K., Ioannidis, K., Tsikrika, T., Vrochidis, S., & Kompatsiaris, I. (2020). A Crowd Analysis Framework for Detecting Violence Scenes. In Proceedings of the 2020 International Conference on Multimedia Retrieval (pp.276-280).
[26] Xu, D., Yan, Y., Ricci, E., & Sebe, N. (2017). Detecting anomalous events in videos by learning deep representations of appearance and motion. Computer Vision and Image Understanding, 156, 117-127.
[27] Ramchandran, A., & Sangaiah, A. K. (2019). Unsupervised deep learning system for local anomaly event detection in crowded scenes. Multimedia Tools and Applications, 1-21.
[28] Kamoona, A. M., Gostar, A. K., Bab-Hadiashar, A., & Hoseinnezhad, R. (2019). Sparsity-Based Naive Bayes Approach for Anomaly Detection in Real Surveillance Videos. In 2019 International Conference on Control, Automation and Information Sciences (ICCAIS) (pp. 1-6).
[29] Zhang, T., Jia, W., Yang, B., Yang, J., He, X., & Zheng, Z. (2017). MoWLD: a robust motion image descriptor for violence detection. Multimedia Tools and applications, 76(1), 1419-1438.
[30] Mabrouk, A. B., & Zagrouba, E. (2017). Spatiotemporal feature using optical flow-based distribution for violence detection. Pattern Recognition Letters, 92, 62-67.
[31] Yu, J., Yow, K. C., & Jeon, M. (2018). Joint representation learning of appearance and motion for abnormal event detection. Machine Vision and Applications, 29(7), 1157-1170.
[32] Wang, S., Zhu, E., Yin, J., & Porikli, F. (2016). Anomaly detection in crowded scenes by SL-HOF descriptor and foreground classification. In 2016 23rd International Conference on Pattern Recognition (ICPR) (pp. 3398-3403), IEEE.
[33] Sun, J., Shao, J., & He, C. (2019). Abnormal event detection for video surveillance using deep one-class learning. Multimedia Tools and Applications, 78(3), 3633-3647.
[34] Zhang, X., Yang, S., Zhang, J., & Zhang, W. (2020). Video Anomaly Detection and Localization using Motion-field Shape Description and Homogeneity Testing. Pattern Recognition, 107394.
[35] Fan, Y., Wen, G., Li, D., Qiu, S., Levine, M. D., & Xiao, F. (2020). Video anomaly detection and localization via Gaussian mixture fully convolutional variational autoencoder. Computer Vision and Image Understanding, 102920.
[36] Ravanbakhsh, M., Sangineto, E., Nabi, M., & Sebe, N. (2019, January). Training adversarial discriminators for cross-channel abnormal event detection in crowds. IEEE Winter Conference on Applications of Computer Vision (WACV), (pp. 1896-1904).
[37] Ye, M., Peng, X., Gan, W., Wu, W., & Qiao, Y. (2019). Video anomaly detection via deep predictive coding network. In Proceedings of the 27th ACM International Conference on Multimedia (pp. 1805-1813).
[38] Wu, D., Sharma, N., & Blumenstein, M. (2017, May). Recent advances in video-based human action recognition using deep learning: A review. In the 2017 International Joint Conference on Neural Networks (IJCNN) (pp. 2865-2872). IEEE.
[39] Dhar, S. K., Hasan, M. M., & Chowdhury, S. A. (2016, December). Human activity recognition based on Gaussian mixture model and directive local binary pattern. In 2016 2nd International Conference on Electrical, Computer & Telecommunication Engineering (ICECTE) (pp. 1-4), IEEE.
[40] Biradar, K. M., Gupta, A., Mandal, M., & Vipparthi, S. K. (2019). Challenges in time-stamp aware anomaly detection in traffic videos. arXiv preprint arXiv:1906.04574.
[41] Abdalrassul Jassim, M. (2017). Networks Security and data transmission using Multi-protocol Label Switching (MPLS) technology. Journal of Al-Qadisiyah for Computer Science and Mathematics, 9(1), 71-81. Retrieved from https://qu.edu.iq/journalcm/index.php/journalcm/article/view/17.
[42] M. Sahib, N. (2017). Video Compression Using Dpcm And Wt. Journal of Al-Qadisiyah for Computer Science and Mathematics, 1(2), 95-109. Retrieved from https://qu.edu.iq/journalcm/index.php/journalcm/article/view/202.
M. Sahib, N. (2017). Video Compression Using Dpcm And Wt. Journal of Al-Qadisiyah for Computer Science and Mathematics, 1(2), 95-109. Retrieved from https://qu.edu.iq/journalcm/index.php/journalcm/article/view/202.
[43] Jumaa Al-janabi, R., & Adil Nori, H. (2017). Utilizing Reverse Engineering in Tracking Software to diagnose its weaknesses. Journal of Al-Qadisiyah for Computer Science and Mathematics, 1(2), 71-83. Retrieved from https://qu.edu.iq/journalcm/index.php/journalcm/article/view/200.
[44] Kh. Alzubaidi, W., & H. Shaker, S. (2018). Methods of Secure Routing Protocol in Wireless Sensor Networks. Journal of Al-Qadisiyah for Computer Science and Mathematics, 10(3), Comp Page 38 - 55. https://doi.org/10.29304/jqcm.2018.10.3.437.
[45] Neamah Hussein, K. (2018). Video Frames Edge Detection of Red Blood Cells: A Performance Evaluation. Journal of Al-Qadisiyah for Computer Science and Mathematics, 10(1), Comp Page 16 - 27. https://doi.org/10.29304/jqcm.2018.10.1.347.
[2] Hamdi S., Bouindour S., Loukil K., Snoussi H., & Abid M. (2019). Hybrid deep learning and HOF for Anomaly Detection. In 2019 6th International Conference on Control, Decision and Information Technologies (CoDIT) (pp. 575-580).
[3] Cheng, K. W., Chen, Y. T., & Fang, W. H. (2015). Gaussian process regression-based video anomaly detection and localization with hierarchical feature representation. IEEE Transactions on Image Processing, 24(12), 5288-5301.
[4] Sargano, A. B., Angelov, P., & Habib, Z. (2017). A comprehensive review on handcrafted and learning-based action representation approaches for human activity recognition. applied sciences, 7(1), 110.
[5] Sabokrou, M., Fathy, M., Zhao, G., & Adeli, E. (2020). Deep End-to-End One-Class Classifier. IEEE Transactions on Neural Networks and Learning Systems.
[6] Nawaratne, R., Alahakoon, D., De Silva, D., & Yu, X. (2019). Spatiotemporal anomaly detection using deep learning for real-time video surveillance. IEEE Transactions on Industrial Informatics, 16(1), 393-402.
[7] OMahony, N., Campbell, S., Carvalho, A., Harapanahalli, S., Hernandez, G. V., Krpalkova, L. & Walsh, J. (2019, April). Deep learning vs. traditional computer vision. In Science and Information Conference (pp. 128-144). Springer, Cham.
[8] Zhao, Y., Deng, B., Shen, C., Liu, Y., Lu, H., & Hua, X. S. (2017). Spatio-temporal autoencoder for video anomaly detection. In Proceedings of the 25th ACM international conference on Multimedia (pp. 1933-1941).
[9] Xia, H., Li, T., Liu, W., Zhong, X., & Yuan, J. (2019). Abnormal event detection method in surveillance video based on temporal CNN and sparse optical flow. In Proceedings of the 2019 5th International Conference on Computing and Data Engineering (pp. 90-94).
[10] Wang, J., Ma, Y., Zhang, L., Gao, R. X., & Wu, D. (2018). Deep learning for smart manufacturing: Methods and applications. Journal of Manufacturing Systems, 48, 144-156.
[11] Lin, W., Hasenstab, K., Cunha, G. M., & Schwartzman, A. (2020). Comparison of handcrafted features and convolutional neural networks for liver MR image adequacy assessment. Scientific Reports, 10(1), 1-11.
[12] Yang, M., Rajasegarar, S., Erfani, S. M., & Leckie, C. (2019). Deep Learning and One-class SVM-based Anomalous Crowd Detection. In 2019 International Joint Conference on Neural Networks (IJCNN) (pp. 1-8). IEEE.
[13] Nguyen, T. N., & Meunier, J. (2019). Anomaly detection in a video sequence with appearance-motion correspondence. In Proceedings of the IEEE International Conference on Computer Vision (pp. 1273-1283).
[14] Huang, S., Huang, D., & Zhou, X. (2018). Learning multimodal deep representations for crowd anomaly event detection. Mathematical Problems in Engineering.
[15] Deepak, K., Vignesh, L. K. P., & Chandrakala, S. (2020). Autocorrelation of gradients-based violence detection in surveillance videos. ICT Express.
[16] Yang, Y., Li, L., Liu, Z., & Liu, G. (2020). Abnormal behavior recognition based on Spatio-temporal context. Journal of Information Processing Systems, 16(3), 612-628.
[17] Das, D., & Mishra, D. (2018). Unsupervised Anomalous Trajectory Detection for Crowded Scenes. In 2018 IEEE 13th International Conference on Industrial and Information Systems (ICIIS) (pp. 27-31). IEEE.
[18] Zhu, S., Hu, J., & Shi, Z. (2016). Local abnormal behavior detection based on optical flow and Spatio-temporal gradient. Multimedia Tools and Applications, 75(15), 9445-9459.
[19] Wang, T., Qiao, M., Zhu, A., Niu, Y., Li, C., & Snoussi, H. (2018). Abnormal event detection via covariance matrix for the optical flow-based feature. Multimedia Tools and Applications, 77(13), 17375-17395.
[20] Hu, X., Huang, Y., Duan, Q., Ci, W., Dai, J., & Yang, H. (2018). Abnormal event detection in crowded scenes using histogram of oriented contextual gradient descriptor. EURASIP Journal on Advances in Signal Processing, 2018(1), 54.
[21] Amraee, S., Vafaei, A., Jamshidi, K., & Adibi, P. (2018). Abnormal event detection in crowded scenes using one-class SVM. Signal, Image and Video Processing, 12(6), 1115-1123.
[22] Pang, G., Yan, C., Shen, C., Hengel, A. V. D., & Bai, X. (2020). Self-trained Deep Ordinal Regression for End-to-End Video Anomaly Detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 12173-12182).
[23] Ruff, L., Vandermeulen, R. A., Gornitz, N., Binder, A., Muller, E., & Kloft, M. (2019). Deep support vector data description for unsupervised and semi-supervised anomaly detection. In Proceedings of the ICML 2019 Workshop on Uncertainty and Robustness in Deep Learning, Long Beach, CA, USA.
[24] Ravanbakhsh, M., Nabi, M., Mousavi, H., Sangineto, E., & Sebe, N. (2018). Plug-and-play CNN for crowd motion analysis: An application in abnormal event detection. In 2018 IEEE Winter Conference on Applications of Computer Vision (WACV) (pp. 1689-1698).
[25] Gkountakos, K., Ioannidis, K., Tsikrika, T., Vrochidis, S., & Kompatsiaris, I. (2020). A Crowd Analysis Framework for Detecting Violence Scenes. In Proceedings of the 2020 International Conference on Multimedia Retrieval (pp.276-280).
[26] Xu, D., Yan, Y., Ricci, E., & Sebe, N. (2017). Detecting anomalous events in videos by learning deep representations of appearance and motion. Computer Vision and Image Understanding, 156, 117-127.
[27] Ramchandran, A., & Sangaiah, A. K. (2019). Unsupervised deep learning system for local anomaly event detection in crowded scenes. Multimedia Tools and Applications, 1-21.
[28] Kamoona, A. M., Gostar, A. K., Bab-Hadiashar, A., & Hoseinnezhad, R. (2019). Sparsity-Based Naive Bayes Approach for Anomaly Detection in Real Surveillance Videos. In 2019 International Conference on Control, Automation and Information Sciences (ICCAIS) (pp. 1-6).
[29] Zhang, T., Jia, W., Yang, B., Yang, J., He, X., & Zheng, Z. (2017). MoWLD: a robust motion image descriptor for violence detection. Multimedia Tools and applications, 76(1), 1419-1438.
[30] Mabrouk, A. B., & Zagrouba, E. (2017). Spatiotemporal feature using optical flow-based distribution for violence detection. Pattern Recognition Letters, 92, 62-67.
[31] Yu, J., Yow, K. C., & Jeon, M. (2018). Joint representation learning of appearance and motion for abnormal event detection. Machine Vision and Applications, 29(7), 1157-1170.
[32] Wang, S., Zhu, E., Yin, J., & Porikli, F. (2016). Anomaly detection in crowded scenes by SL-HOF descriptor and foreground classification. In 2016 23rd International Conference on Pattern Recognition (ICPR) (pp. 3398-3403), IEEE.
[33] Sun, J., Shao, J., & He, C. (2019). Abnormal event detection for video surveillance using deep one-class learning. Multimedia Tools and Applications, 78(3), 3633-3647.
[34] Zhang, X., Yang, S., Zhang, J., & Zhang, W. (2020). Video Anomaly Detection and Localization using Motion-field Shape Description and Homogeneity Testing. Pattern Recognition, 107394.
[35] Fan, Y., Wen, G., Li, D., Qiu, S., Levine, M. D., & Xiao, F. (2020). Video anomaly detection and localization via Gaussian mixture fully convolutional variational autoencoder. Computer Vision and Image Understanding, 102920.
[36] Ravanbakhsh, M., Sangineto, E., Nabi, M., & Sebe, N. (2019, January). Training adversarial discriminators for cross-channel abnormal event detection in crowds. IEEE Winter Conference on Applications of Computer Vision (WACV), (pp. 1896-1904).
[37] Ye, M., Peng, X., Gan, W., Wu, W., & Qiao, Y. (2019). Video anomaly detection via deep predictive coding network. In Proceedings of the 27th ACM International Conference on Multimedia (pp. 1805-1813).
[38] Wu, D., Sharma, N., & Blumenstein, M. (2017, May). Recent advances in video-based human action recognition using deep learning: A review. In the 2017 International Joint Conference on Neural Networks (IJCNN) (pp. 2865-2872). IEEE.
[39] Dhar, S. K., Hasan, M. M., & Chowdhury, S. A. (2016, December). Human activity recognition based on Gaussian mixture model and directive local binary pattern. In 2016 2nd International Conference on Electrical, Computer & Telecommunication Engineering (ICECTE) (pp. 1-4), IEEE.
[40] Biradar, K. M., Gupta, A., Mandal, M., & Vipparthi, S. K. (2019). Challenges in time-stamp aware anomaly detection in traffic videos. arXiv preprint arXiv:1906.04574.
[41] Abdalrassul Jassim, M. (2017). Networks Security and data transmission using Multi-protocol Label Switching (MPLS) technology. Journal of Al-Qadisiyah for Computer Science and Mathematics, 9(1), 71-81. Retrieved from https://qu.edu.iq/journalcm/index.php/journalcm/article/view/17.
[42] M. Sahib, N. (2017). Video Compression Using Dpcm And Wt. Journal of Al-Qadisiyah for Computer Science and Mathematics, 1(2), 95-109. Retrieved from https://qu.edu.iq/journalcm/index.php/journalcm/article/view/202.
M. Sahib, N. (2017). Video Compression Using Dpcm And Wt. Journal of Al-Qadisiyah for Computer Science and Mathematics, 1(2), 95-109. Retrieved from https://qu.edu.iq/journalcm/index.php/journalcm/article/view/202.
[43] Jumaa Al-janabi, R., & Adil Nori, H. (2017). Utilizing Reverse Engineering in Tracking Software to diagnose its weaknesses. Journal of Al-Qadisiyah for Computer Science and Mathematics, 1(2), 71-83. Retrieved from https://qu.edu.iq/journalcm/index.php/journalcm/article/view/200.
[44] Kh. Alzubaidi, W., & H. Shaker, S. (2018). Methods of Secure Routing Protocol in Wireless Sensor Networks. Journal of Al-Qadisiyah for Computer Science and Mathematics, 10(3), Comp Page 38 - 55. https://doi.org/10.29304/jqcm.2018.10.3.437.
[45] Neamah Hussein, K. (2018). Video Frames Edge Detection of Red Blood Cells: A Performance Evaluation. Journal of Al-Qadisiyah for Computer Science and Mathematics, 10(1), Comp Page 16 - 27. https://doi.org/10.29304/jqcm.2018.10.1.347.
Downloads
Published
2021-09-22
How to Cite
Mahdi, M. S., Mohammed, A. J., & Jafer, M. M. (2021). Unusual Activity Detection in Surveillance Video Scene: Review. Journal of Al-Qadisiyah for Computer Science and Mathematics, 13(3), Comp Page 92 – 98. https://doi.org/10.29304/jqcm.2021.13.3.848
Issue
Section
Computer Articles
