Deploying Facial Segmentation Landmarks for Deepfake Detection
DOI:
https://doi.org/10.29304/jqcm.2023.15.1.1161Keywords:
Deepfake detection;, Digital image forensics;, Deep learning;, CNN;, DenseNet121;, Face segmentation;, FFHQ datasetAbstract
Deepfake is a type of artificial intelligence used to create convincing images, audio, and video hoaxes and it concerns celebrities and everyone because they are easy to manufacture. Deepfake are hard to recognize by people and current approaches, especially high-quality ones. As a defense against Deepfake techniques, various methods to detect Deepfake in images have been suggested. Most of them had limitations, like only working with one face in an image. The face has to be facing forward, with both eyes and the mouth open, depending on what part of the face they worked on. Other than that, a few focus on the impact of pre-processing steps on the detection accuracy of the models. This paper introduces a framework design focused on this aspect of the Deepfake detection task and proposes pre-processing steps to improve accuracy and close the gap between training and validation results with simple operations. Additionally, it differed from others by dealing with the positions of the face in various directions within the image, distinguishing the concerned face in an image containing multiple faces, and segmentation the face using facial landmarks points. All these were done using face detection, face box attributes, facial landmarks, and key points from the MediaPipe tool with the pre-trained model (DenseNet121). Lastly, the proposed model was evaluated using Deepfake Detection Challenge datasets, and after training for a few epochs, it achieved an accuracy of 97% in detecting the Deepfake
Downloads
Download data is not yet available.
References
[1] I. Goodfellow et al., "Generative adversarial nets," vol. 27, 2014.
[2] T. Karras, S. Laine, and T. Aila, "A Style-Based Generator Architecture for Generative Adversarial Networks," IEEE Transactions on Pattern
Analysis and Machine Intelligence, vol. PP, pp. 1-1, 01/31 2020.
[3] D. Zhang et al., "The AI index 2021 annual report," 2021.
[4] P. Korshunov and S. Marcel, "Vulnerability assessment and detection of deepfake videos," in 2019 International Conference on Biometrics (ICB),
2019, pp. 1-6: IEEE.
[5] A. Elhassan, M. Al-Fawa'reh, M. T. Jafar, M. Ababneh, and S. T. J. S. Jafar, "DFT-MF: Enhanced deepfake detection using mouth movement and
transfer learning," vol. 19, p. 101115, 2022.
[6] Yu, Chia-Mu, Ching-Tang Chang, and Yen-Wu Ti. "Detecting deepfake-forged contents with separable convolutional neural network and image
segmentation." arXiv preprint arXiv:1912.12184 (2019).
[7] G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, "Densely connected convolutional networks," in Proceedings of the IEEE conference
on computer vision and pattern recognition, 2017, pp. 4700-4708.
[8] Oraibi, Mohammed R., and Abdulkareem M. Radhi. "Enhancement Digital Forensic Approach for Inter-Frame Video Forgery Detection Using a
Deep Learning Technique." Iraqi Journal of Science (2022): 2686-2701.
[9] K. Y. Lim, H. Joan, and Y. Tew, "COMPUTER PERFORMANCE EVALUATION FOR VIRTUAL CLASSROOM WITH ARTIFICIAL
INTELLIGENCE FEATURES," in International Conference on Digital Transformation and Applications (ICDXA), 2021, vol. 25, p. 26.
[10] S. Das, S. Seferbekov, A. Datta, M. Islam, and M. Amin, "Towards solving the deepfake problem: An analysis on improving deepfake detection
using dynamic face augmentation," in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 3776-3785.
[11] Guo, Hui, et al. "Robust Attentive Deep Neural Network for Detecting GAN-Generated Faces." IEEE Access 10 (2022): 32574-32583.
[12] K. Zhang, Z. Zhang, Z. Li, and Y. Qiao, "Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks," IEEE Signal
Processing Letters, vol. 23, no. 10, pp. 1499-1503, 2016.
[13] King, Davis E. "Dlib-ml: A machine learning toolkit." The Journal of Machine Learning Research 10 (2009): 1755-1758.
[14] S. Milborrow and F. Nicolls, "Active shape models with SIFT descriptors and MARS," in 2014 International Conference on Computer Vision Theory and Applications (VISAPP), 2014, vol. 2, pp. 380-387: IEEE.
[15] J. T. Camillo Lugaresi, Hadon Nash, Chris McClanahan, Esha Uboweja, Michael Hays, Fan Zhang, Chuo-Ling Chang, Ming Guang Yong, Juhyun
Lee, Wan-Teh Chang, Wei Hua, Manfred Georg and Matthias Grundmann Google Research, "MediaPipe: A Framework for Building Perception
Pipelines," 2019.
[16] J. Deng, J. Guo, E. Ververas, I. Kotsia, and S. Zafeiriou, "Retinaface: Single-shot multi-level face localisation in the wild," in Proceedings of the
IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 5203-5212.
[17] T. Baltrusaitis, A. Zadeh, Y. C. Lim, and L.-P. Morency, "Openface 2.0: Facial behavior analysis toolkit," in 2018 13th IEEE international
conference on automatic face & gesture recognition (FG 2018), 2018, pp. 59-66: IEEE.
[18] A. I. Siam, N. F. Soliman, A. D. Algarni, F. E. Abd El-Samie, and A. Sedik, "Deploying Machine Learning Techniques for Human Emotion
Detection," Comput Intell Neurosci, vol. 2022, p. 8032673, 2022.
[19] G. Révy, D. Hadházi, and G. Hullám, "Towards Hand-Over-Face Gesture Detection," in 29th Minisymposium of the Department of Measurement
and Information Systems, 2022, pp. 58-61: Budapest University of Technology and Economics.
[20] Shorten, Connor, and Taghi M. Khoshgoftaar. "A survey on image data augmentation for deep learning." Journal of big data 6.1 (2019): 1-48.
[21] Silva, Samuel Henrique, et al. "Deepfake forensics analysis: An explainable hierarchical ensemble of weakly supervised models." Forensic Science
International: Synergy 4 (2022): 100217.
[22] N. H. Salman and S. N. Mohammed, "Image Segmentation Using PSO-Enhanced K-Means Clustering and Region Growing Algorithms," Iraqi
Journal of Science, pp. 4988-4998, 2021.
[23] Abdulateef, Salwa Khalid, and Mohanad Dawood Salman. "A Comprehensive Review of Image Segmentation Techniques." Iraqi Journal for
Electrical & Electronic Engineering 17.2 (2021).
[24] Symeon, Polychronis Charitidis Giorgos Kordopatis-Zilos, and Papadopoulos Ioannis Kompatsiaris. "AFace PREPROCESSING APPROACH FOR
IMPROVED DEEPFAKE DETECTION." arXiv preprint arXiv:2006.07084 (2020).
[25] Suganthi, S. T., et al. "Deep learning model for deep fake face recognition and detection." PeerJ Computer Science 8 (2022): e881.
[26] Nirkin, Yuval, et al. "DeepFake detection based on discrepancies between faces and their context." IEEE Transactions on Pattern Analysis and
Machine Intelligence (2021).
[27] S. Fang, S. Wang, and R. Ye, "DeepFake Video Detection through Facial Sparse Optical Flow based Light CNN," in Journal of Physics: Conference Series, 2022, vol. 2224, no. 1, p. 012014: IOP Publishing.
[28] Dhahir, Hayder Kadhim, and Nassir Hussein Salman. "A Review on Face Detection Based on Convolution Neural Network Techniques." Iraqi
Journal of Science (2022): 1823-1835.
[29] A. T. Kabakus, "An experimental performance comparison of widely used face detection tools," 2019.
[30] Y. Kartynnik, A. Ablavatski, I. Grishchenko, and M. J. a. p. a. Grundmann, "Real-time facial surface geometry from monocular video on mobile
GPUs," 2019.
[31] O. Russakovsky et al., "Imagenet large scale visual recognition challenge," vol. 115, no. 3, pp. 211-252, 2015.
[32] Kingma, Diederik P., and Jimmy Ba. "Adam: A method for stochastic optimization." arXiv preprint arXiv:1412.6980 (2014).
[33] Y. Wang, V. Zarghami, and S. Cui, "Fake Face Detection using Local Binary Pattern and Ensemble Modeling," in 2021 IEEE International Conference on Image Processing (ICIP), 2021, pp. 3917-3921: IEEE.
[2] T. Karras, S. Laine, and T. Aila, "A Style-Based Generator Architecture for Generative Adversarial Networks," IEEE Transactions on Pattern
Analysis and Machine Intelligence, vol. PP, pp. 1-1, 01/31 2020.
[3] D. Zhang et al., "The AI index 2021 annual report," 2021.
[4] P. Korshunov and S. Marcel, "Vulnerability assessment and detection of deepfake videos," in 2019 International Conference on Biometrics (ICB),
2019, pp. 1-6: IEEE.
[5] A. Elhassan, M. Al-Fawa'reh, M. T. Jafar, M. Ababneh, and S. T. J. S. Jafar, "DFT-MF: Enhanced deepfake detection using mouth movement and
transfer learning," vol. 19, p. 101115, 2022.
[6] Yu, Chia-Mu, Ching-Tang Chang, and Yen-Wu Ti. "Detecting deepfake-forged contents with separable convolutional neural network and image
segmentation." arXiv preprint arXiv:1912.12184 (2019).
[7] G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, "Densely connected convolutional networks," in Proceedings of the IEEE conference
on computer vision and pattern recognition, 2017, pp. 4700-4708.
[8] Oraibi, Mohammed R., and Abdulkareem M. Radhi. "Enhancement Digital Forensic Approach for Inter-Frame Video Forgery Detection Using a
Deep Learning Technique." Iraqi Journal of Science (2022): 2686-2701.
[9] K. Y. Lim, H. Joan, and Y. Tew, "COMPUTER PERFORMANCE EVALUATION FOR VIRTUAL CLASSROOM WITH ARTIFICIAL
INTELLIGENCE FEATURES," in International Conference on Digital Transformation and Applications (ICDXA), 2021, vol. 25, p. 26.
[10] S. Das, S. Seferbekov, A. Datta, M. Islam, and M. Amin, "Towards solving the deepfake problem: An analysis on improving deepfake detection
using dynamic face augmentation," in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 3776-3785.
[11] Guo, Hui, et al. "Robust Attentive Deep Neural Network for Detecting GAN-Generated Faces." IEEE Access 10 (2022): 32574-32583.
[12] K. Zhang, Z. Zhang, Z. Li, and Y. Qiao, "Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks," IEEE Signal
Processing Letters, vol. 23, no. 10, pp. 1499-1503, 2016.
[13] King, Davis E. "Dlib-ml: A machine learning toolkit." The Journal of Machine Learning Research 10 (2009): 1755-1758.
[14] S. Milborrow and F. Nicolls, "Active shape models with SIFT descriptors and MARS," in 2014 International Conference on Computer Vision Theory and Applications (VISAPP), 2014, vol. 2, pp. 380-387: IEEE.
[15] J. T. Camillo Lugaresi, Hadon Nash, Chris McClanahan, Esha Uboweja, Michael Hays, Fan Zhang, Chuo-Ling Chang, Ming Guang Yong, Juhyun
Lee, Wan-Teh Chang, Wei Hua, Manfred Georg and Matthias Grundmann Google Research, "MediaPipe: A Framework for Building Perception
Pipelines," 2019.
[16] J. Deng, J. Guo, E. Ververas, I. Kotsia, and S. Zafeiriou, "Retinaface: Single-shot multi-level face localisation in the wild," in Proceedings of the
IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 5203-5212.
[17] T. Baltrusaitis, A. Zadeh, Y. C. Lim, and L.-P. Morency, "Openface 2.0: Facial behavior analysis toolkit," in 2018 13th IEEE international
conference on automatic face & gesture recognition (FG 2018), 2018, pp. 59-66: IEEE.
[18] A. I. Siam, N. F. Soliman, A. D. Algarni, F. E. Abd El-Samie, and A. Sedik, "Deploying Machine Learning Techniques for Human Emotion
Detection," Comput Intell Neurosci, vol. 2022, p. 8032673, 2022.
[19] G. Révy, D. Hadházi, and G. Hullám, "Towards Hand-Over-Face Gesture Detection," in 29th Minisymposium of the Department of Measurement
and Information Systems, 2022, pp. 58-61: Budapest University of Technology and Economics.
[20] Shorten, Connor, and Taghi M. Khoshgoftaar. "A survey on image data augmentation for deep learning." Journal of big data 6.1 (2019): 1-48.
[21] Silva, Samuel Henrique, et al. "Deepfake forensics analysis: An explainable hierarchical ensemble of weakly supervised models." Forensic Science
International: Synergy 4 (2022): 100217.
[22] N. H. Salman and S. N. Mohammed, "Image Segmentation Using PSO-Enhanced K-Means Clustering and Region Growing Algorithms," Iraqi
Journal of Science, pp. 4988-4998, 2021.
[23] Abdulateef, Salwa Khalid, and Mohanad Dawood Salman. "A Comprehensive Review of Image Segmentation Techniques." Iraqi Journal for
Electrical & Electronic Engineering 17.2 (2021).
[24] Symeon, Polychronis Charitidis Giorgos Kordopatis-Zilos, and Papadopoulos Ioannis Kompatsiaris. "AFace PREPROCESSING APPROACH FOR
IMPROVED DEEPFAKE DETECTION." arXiv preprint arXiv:2006.07084 (2020).
[25] Suganthi, S. T., et al. "Deep learning model for deep fake face recognition and detection." PeerJ Computer Science 8 (2022): e881.
[26] Nirkin, Yuval, et al. "DeepFake detection based on discrepancies between faces and their context." IEEE Transactions on Pattern Analysis and
Machine Intelligence (2021).
[27] S. Fang, S. Wang, and R. Ye, "DeepFake Video Detection through Facial Sparse Optical Flow based Light CNN," in Journal of Physics: Conference Series, 2022, vol. 2224, no. 1, p. 012014: IOP Publishing.
[28] Dhahir, Hayder Kadhim, and Nassir Hussein Salman. "A Review on Face Detection Based on Convolution Neural Network Techniques." Iraqi
Journal of Science (2022): 1823-1835.
[29] A. T. Kabakus, "An experimental performance comparison of widely used face detection tools," 2019.
[30] Y. Kartynnik, A. Ablavatski, I. Grishchenko, and M. J. a. p. a. Grundmann, "Real-time facial surface geometry from monocular video on mobile
GPUs," 2019.
[31] O. Russakovsky et al., "Imagenet large scale visual recognition challenge," vol. 115, no. 3, pp. 211-252, 2015.
[32] Kingma, Diederik P., and Jimmy Ba. "Adam: A method for stochastic optimization." arXiv preprint arXiv:1412.6980 (2014).
[33] Y. Wang, V. Zarghami, and S. Cui, "Fake Face Detection using Local Binary Pattern and Ensemble Modeling," in 2021 IEEE International Conference on Image Processing (ICIP), 2021, pp. 3917-3921: IEEE.
Downloads
Published
2023-04-03
How to Cite
Abdullah, M. T., & M. Ali, N. H. (2023). Deploying Facial Segmentation Landmarks for Deepfake Detection. Journal of Al-Qadisiyah for Computer Science and Mathematics, 15(1), Comp Page 137–149. https://doi.org/10.29304/jqcm.2023.15.1.1161
Issue
Section
Computer Articles
