Securing ML Models: A Systematic Survey of Poisoning Attacks and Defense Mechanisms
DOI:
https://doi.org/10.29304/jqcsm.2024.16.41776Keywords:
Poisoning Attack, Label Flipping, Clean Lebel, Watermarking, Adversarial AttackAbstract
In recent years, Machine Learning (ML) has brought about a significant revolution in several fields such as medicine, justice, cybersecurity, and other vital fields that require intelligent and urgent decision-making. With this development, a type of adversarial attack targeting ML models called a Poisoning Attack (PA) has emerged. One realistic attack scenario is for an adversary to subtly update samples or reverse some labels of training data, causing degradation to the model's overall accuracy during the testing phase. To gain a deeper understanding of this scenario, a survey will be conducted about the attack and how it is carried out against different models. In addition to the protection techniques to identify their weaknesses. Finally, some solutions will be proposed to maintain the availability, robustness, and integrity of ML models.
Downloads
References
A. A. Springborg, M. K. Andersen, K. H. Hattel, and M. Albano, “Towards a secure API client generator for IoT devices”, 2022. http://arxiv.org/abs/2201.00270
A. Schwarzschild, M. Goldblum, A. Gupta, J. P. Dickerson, and T. Goldstein, “Just How Toxic is Data Poisoning? A Unified Benchmark for Backdoor and Data Poisoning Attacks”, 2020. http://arxiv.org/abs/2006.12557
A. H. Ali, M. Z. Abdullah, S. N. Abdul-Wahab, and M. Alsajri, “A Brief Review of Big Data Analytics Based on Machine Learning,” Iraqi Journal for Computer Science and Mathematics, vol. 1, no. 2, pp. 13–15, 2020. http://dx.doi.org/10.52866/ijcsm.2020.01.02.002
M. Naumov et al., “Deep Learning Recommendation Model for Personalization and Recommendation Systems,” May 2019. https://doi.org/10.48550/arXiv.1906.00091
H. Liu and B. Lang, “Machine learning and deep learning methods for intrusion detection systems: A survey,” Applied Sciences (Switzerland), vol. 9, no. 20. MDPI AG, 2019. https://doi.org/10.3390/app9204396
M. Haqi Al-Tai, B. M. Nema, and A. Al-Sherbaz, “Deep Learning for Fake News Detection: Literature Review,” Al-Mustansiriyah Journal of Science, vol. 34, no. 2, pp. 70–81, Jun. 2023. https://doi.org/10.23851/mjs.v34i2.1292
Charu C. Aggarwal, and Chandan K. Reddy, “Date Clustering Algorithms and Applications” Chapman & Hall/CRC Data Mining and Knowledge Discovery Series, 2014. https://doi.org/10.1201/9781315373515
M. M. Mijwil, I. E. Salem, and M. M. Ismaeel, “The Significance of Machine Learning and Deep Learning Techniques in Cybersecurity: A Comprehensive Review,” Iraqi Journal for Computer Science and Mathematics, vol. 4, no. 1. College of Education, Al-Iraqia University, pp. 87–101, 2023. https://doi.org/10.52866/ijcsm.2023.01.01.008
M. M. Taye, “Understanding of Machine Learning with Deep Learning: Architectures, Workflow, Applications and Future Directions”, Computers, vol. 12, no. 5. MDPI, 01, 2023. https://doi.org/10.3390/computers12050091
M. Naeem, S. T. H. Rizvi, and A. Coronato, “A Gentle Introduction to Reinforcement Learning and its Application in Different Fields,” IEEE Access, vol. 8, pp. 209320–209344, 2020. https://doi.org/10.1109/ACCESS.2020.3038605
J. Lin, L. Dang, M. Rahouti, and K. Xiong, “ML Attack Models: Adversarial Attacks and Data Poisoning Attacks”, 2021. https://doi.org/10.48550/arXiv.2112.02797
M. Barreno, B. Nelson, A. D. Joseph, and J. D. Tygar, “The security of machine learning,” Mach Learn, vol. 81, no. 2, pp. 121–148, 2010. http://doi.org/10.1007/s10994-010-5188-5
Y. Li, Y. Bai, Y. Jiang, Y. Yang, S.-T. Xia, and B. Li, “Untargeted Backdoor Watermark: Towards Harmless and Stealthy Dataset Copyright Protection”, Advances in Neural Information Processing Systems, vol. 35, pp. 13238-13250., 2022. https://doi.org/10.48550/arXiv.2210.00875
L. Cassiday, “Clean label: The next generation,” International News on Fats, Oils and Related Materials, vol. 28, no. 8, pp. 6–10, Sep. 2017. http://dx.doi.org/10.21748/inform.09.2017.06
J. Horkoff, “Non-functional requirements for machine learning: Challenges and new directions”, Proceedings of the IEEE International Conference on Requirements Engineering, IEEE Computer Society, pp. 386–391, 2019. https://doi.org/10.1109/RE.2019.00050
L. Obiora Nweke, “Using the CIA and AAA Models to explain Cybersecurity Activities”, PM World Journal, vol. 6, no.12, pp. 1-3., 2017. https://pmworldlibrary.net/article/using-the-cia-and-aaa-models-to-explain-cybersecurity-activities/
H. bediar Hashim, “Challenges and Security Vulnerabilities to Impact on Database Systems,” Al-Mustansiriyah Journal of Science, vol. 29, no. 2, pp. 117–125, Nov. 2018. https://doi.org/10.23851/mjs.v29i2.332
B. Biggio and F. Roli, “Wild patterns: Ten years after the rise of adversarial machine learning,” Pattern Recognit, vol. 84, pp. 317–331, 2018. https://doi.org/10.1016/j.patcog.2018.07.023
H. H. Ali, J. R. Naif, and W. R. Humood, “A New Smart Home Intruder Detection System Based on Deep Learning,” Al-Mustansiriyah Journal of Science, vol. 34, no. 2, pp. 60–69, 2023. https://doi.org/10.23851/mjs.v34i2.1267
M. Nasr, R. Shokri, and A. Houmansadr, “Comprehensive privacy analysis of deep learning: Passive and active white-box inference attacks against centralized and federated learning”, Proceedings - IEEE Symposium on Security and Privacy, Institute of Electrical and Electronics Engineers Inc., pp. 739–753, 2019. http://doi.org/ 10.1109/SP.2019.00065
B. Hitaj, G. Ateniese, and F. Perez-Cruz, “Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning”, Conference on Computer and Communications Security, pp.603-613, 2017. https://doi.org/10.1145/3133956.3134012
L. Melis, C. Song, E. De Cristofaro, and V. Shmatikov, “Exploiting unintended feature leakage in collaborative learning,” in Proceedings - IEEE Symposium on Security and Privacy, Institute of Electrical and Electronics Engineers Inc., pp. 691–706, 2019. https://doi.org/10.1109/SP.2019.00029
R. Shokri, M. Stronati, C. Song, and V. Shmatikov, “Membership Inference Attacks Against Machine Learning Models”, Proceedings - IEEE Symposium on Security and Privacy, Institute of Electrical and Electronics Engineers Inc., pp. 3–18, 2017. https://doi.org/10.1109/SP.2017.41
W. Xu, Y. Qi, and D. Evans, “Automatically Evading Classifiers A Case Study on PDF Malware Classifiers”, Network and Distributed System Security Symposium, 2016. http://dx.doi.org/10.14722/ndss.2016.23115
M. Mozaffari-Kermani, S. Sur-Kolay, A. Raghunathan, and N. K. Jha, “Systematic poisoning attacks on and defenses for machine learning in healthcare,” IEEE J Biomed Health Inform, vol. 19, no. 6, pp. 1893–1905, Nov. 2015. https://doi.org/10.1109/JBHI.2014.2344095
H. Xiao XIAOHU, G. Fumera, and F. Roli, “Is Feature Selection Secure against Training Data Poisoning?”, international conference on machine learning, pp. 1689-1698, 2015. https://pure.manchester.ac.uk/ws/portalfiles/portal/32297390/
R. Laishram and V. V. Phoha, “Curie: A method for protecting SVM Classifier from Poisoning Attack”, 2016. https://doi.org/10.48550/arXiv.1606.01584
B. Li, Y. Wang, A. Singh, and Y. Vorobeychik, “Data Poisoning Attacks on Factorization-Based Collaborative Filtering”, Conference on Neural Information Processing Systems, vol. 29, 2016. https://dl.acm.org/doi/10.5555/3157096.3157308
P. P. K. Chan, Z. M. He, H. Li, and C. C. Hsu, “Data sanitization against adversarial label contamination based on data complexity,” International Journal of Machine Learning and Cybernetics, vol. 9, no. 6, pp. 1039–1052, 2018. http://doi.org/10.1007/s13042-016-0629-5
N. Baracaldo, B. Chen, H. Ludwig, and J. A. Safavi, “Mitigating poisoning attacks on machine learning models: A data Provenance based approach”, AISec 2017 - Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security, co-located with CCS 2017, Association for Computing Machinery, Inc, pp. 103–110, 2017. http://dx.doi.org/10.1145/3128572.3140450
S. Chen et al., “Automated poisoning attacks and defenses in malware detection systems: An adversarial machine learning approach,” Comput Secur, vol. 73, pp. 326–344,2018. http://doi.org/10.1016/j.cose.2017.11.007
M. Jagielski, A. Oprea, B. Biggio, C. Liu, C. Nita-Rotaru, and B. Li, “Manipulating Machine Learning: Poisoning Attacks and Countermeasures for Regression Learning”, Proceedings - IEEE Symposium on Security and Privacy, Institute of Electrical and Electronics Engineers Inc., pp. 19–35, 2018. http://doi.org/10.1109/SP.2018.00057
A. Paudice, L. Muñoz-González, and E. C. Lupu, “Label Sanitization against Label Flipping Poisoning Attacks”, Conference on Machine Learning and Knowledge Discovery in Databases, vol. 11329, pp. 5-15, 2018. https://doi.org/10.1007/978-3-030-13453-2_1
D. Cao, S. Chang, Z. Lin, G. Liu, and D. Sun, “Understanding distributed poisoning attack in federated learning,” in Proceedings of the International Conference on Parallel and Distributed Systems - ICPADS, IEEE Computer Society, pp. 233–239, 2019. http://doi.org/10.1109/ICPADS47876.2019.00042
R. Taheri, R. Javidan, M. Shojafar, Z. Pooranian, A. Miri, and M. Conti, “On defending against label flipping attacks on malware detection systems,” Neural Comput. Appl, vol. 32, no. 18, pp. 14781–14800, 2020. http://doi.org/10.1007/s00521-020-04831-9
P. P. K. Chan, Z. He, X. Hu, E. C. C. Tsang, D. S. Yeung, and W. W. Y. Ng, “Causative label flip attack detection with data complexity measures”, International Journal of Machine Learning and Cybernetics, vol. 12, no. 1, pp. 103–116, 2021. http://doi.org/10.1007/s13042-020-01159-7
H. Liu, D. Li, and Y. Li, “Poisonous Label Attack: Black-Box Data Poisoning Attack with Enhanced Conditional DCGAN,” Neural Process Lett, vol. 53, no. 6, pp. 4117–4142, 2021. http://doi.org/10.1007/s11063-021-10584-w
H. Zhang, N. Cheng, Y. Zhang, and Z. Li, “Label flipping attacks against Naive Bayes on spam filtering systems,” Applied Intelligence, vol. 51, no. 7, pp. 4503–4514, 2021. http://doi.org/10.1007/s10489-020-02086-4
B. Zhao and Y. Lao, “Towards Class-Oriented Poisoning Attacks Against Neural Networks”, Conference on Applications of Computer Vision (WACV), 2022. https://doi.org/10.1109/WACV51458.2022.00230
R. Sharma, G. K. Sharma, and M. Pattanaik, “A CatBoost Based Approach to Detect Label Flipping Poisoning Attack in Hardware Trojan Detection Systems,” Journal of Electronic Testing: Theory and Applications (JETTA), vol. 38, no. 6, pp. 667–682, 2022. http://doi.org/10.1007/s10836-022-06035-6
Q. Li, X. Wang, F. Wang, and C. Wang, “A Label Flipping Attack on Machine Learning Model and Its Defense Mechanism,” in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Springer Science and Business Media Deutschland GmbH, pp. 490–506, 2023. http://doi.org/10.1007/978-3-031-22677-9_26
H. Mohammadian, A. Lashkari, and A. Ghorbani, “Evaluating Label Flipping Attack in Deep Learning-Based NIDS,” INSTICC, pp. 597–603, 2023. http://dx.doi.org/10.5220/0010867900003120
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Mahdi Nsaif Jasim, Hanan Abed Alwally Abed Allah
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
