Detecting Frame Deletion in Videos Using Supervised and Unsupervised Learning with Convolutional Neural Networks
Keywords:
CNN, deep learning, frame deletion detection, temporal forgery, video forgery detectionAbstract
In recent years, videos have been susceptible not only to any edition but also to a variety of forgeries. One of the most popular video forgeries is frame deletion, in which a group of frames is removed to hide specific actions from the human eye. When frame deletion occurs, videos selected as evidence lose their evidentiary value. This highlights the necessity of automation, especially for analyzing large volumes of videos. Thus, we measure the performance of two deep learning approaches for frame deletion detection. Both of them use Convolutional Neural Networks (CNN): The first one, a supervised 3DCNN model and, the second one, is an unsupervised model compound of VGG-16 and Resnet-50. We evaluated them using 10-fold cross-validation in the following datasets: UCF-101, VIFFD and DTD (Driving Test Dataset), which is our contribution to the data community. To the best of our knowledge, no comparison of both approaches using 10-fold cross-validation has been found in the literature before. Afterward, we analyze the results and make recommendations for future work in this area.
Downloads
References
S. Fadl, Q. Han, and Q. Li, “CNN spatiotemporal features and fusion for surveillance video forgery detection,” Signal Processing: Image Communication, vol. 90, p. 116066, Jan. 2021, doi: 10.1016/j.image.2020.116066.
M. Munawar, I. Noreen, R. S. Alharthi, and N. Sarwar, “Forged Video Detection Using Deep Learning: A SLR,” Applied Computational Intelligence and Soft Computing, vol. 2023, p. e6661192, Oct. 2023, doi: 10.1155/2023/6661192.
A. Yadav and D. Kumar Vishwakarma, “Datasets, clues and stateof-the-arts for multimedia forensics: An extensive review,” Expert Systems with Applications, vol. 249, p. 123756, Sep. 2024, doi: 10.1016/j.eswa.2024.123756.
P. Johnston and E. Elyan, “A review of digital video tampering: From simple editing to full synthesis,” Digital Investigation, vol. 29, pp. 67–81, Jun. 2019, doi: 10.1016/j.diin.2019.03.006.
G. Mercier, F. Markatopoulou, R. Cozien, M. Zampoglou, E. Apostolidis, A. Metsai, S. Papadopoulos, V. Mezaris, I. Patras, and I. Kompatsiaris, “Detecting manipulations in video,” in Video Verif. in the Fake News Era. Springer International Publishing, 2019, pp. 161–189, journal Abbreviation: Video Verif. in the Fake News Era, doi: 10.1007/978-3-030-26752-0 6.
L. Jegaveerapandian, A. Rani, P. Periyaswamy, and S. Velusamy, “A survey on passive digital video forgery detection techniques,” vol. 13, no. 6, pp. 6324–6334, 2023, publisher: Institute of Advanced Engineering and Science, doi: 10.11591/ijece.v13i6.pp6324-6334.
A. R. Javed, Z. Jalil, W. Zehra, T. Gadekallu, D. Suh, and M. Jalil Piran, “A comprehensive survey on digital video forensics: Taxonomy, challenges, and future directions,” Engineering Applications of Artificial Intelligence, vol. 106, p. 104456, Nov. 2021, doi: 0.1016/j.engappai.2021.104456.
N. A. Shelke and S. S. Kasana, “A comprehensive survey on passive techniques for digital video forgery detection,” Multimedia Tools and Applications, vol. 80, no. 4, pp. 6247–6310, Feb. 2021, doi: 10.1007/s11042-020-09974-4.
W. El-Shafai, M. A. Fouda, E.-S. M. El-Rabaie, and N. A. El-Salam, “A comprehensive taxonomy on multimedia video forgery detection techniques: challenges and novel trends,” Multimedia Tools and Applications, May 2023, doi: 10.1007/s11042-023-15609-1.
R. Gowda and D. Pawar, “Deep learning-based forgery identification and localization in videos,” Signal, Image and Video Processing, vol. 17, no. 5, pp. 2185–2192, Jul. 2023, doi: 10.1007/s11760-022-02433-7.
N. Akhtar, M. Saddique, K. Asghar, U. Bajwa, M. Hussain, and Z. Habib, “Digital Video Tampering Detection and Localization: Review, Representations, Challenges and Algorithm,” Mathematics, vol. 10, no. 2, 2022, doi: 10.3390/math10020168.
S. Ferreira, M. Antunes, and M. Correia, “A dataset of photos and videos for digital forensics analysis using machine learning processing,” Data, vol. 6, no. 8, 2021, doi: 10.3390/data6080087.
X. Nguyen, Y. Hu, M. Amin, K. Hayat, V. Le, and D.-T. Truong, “Detecting Video Inter-Frame Forgeries Based on Convolutional Neural Network Model,” International Journal of Image, Graphics and Signal Processing, vol. 12, no. 3, pp. 1–12, 2020, doi: 10.5815/ijigsp.2020.03.01.
H. Yi Gong, F. Chun Hui, and B. Dan Dan, “IReF: Improved Residual Feature For Video Frame Deletion Forensics,” in 2022 4th International Conference on Data Intelligence and Security (ICDIS), Aug. 2022, pp. 248–253, doi: 10.1109/ICDIS55630.2022.00045.
J. H. Hong, Y. Yang, and B. T. Oh, “Detection of frame deletion in HEVC-Coded video in the compressed domain,” Digital Investigation, vol. 30, pp. 23–31, Sep. 2019, doi: 10.1016/j.diin.2019.06.002.
C. Long, E. Smith, A. Basharat, and A. Hoogs, “A C3D-Based Convolutional Neural Network for Frame Dropping Detection in a Single Video Shot,” in 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Honolulu, HI, USA: IEEE, Jul. 2017, pp. 1898–1906, doi: 10.1109/CVPRW.2017.237.
N. Singla, J. Singh, and S. Nagpal, “Video Frame Deletion Detection using Correlation Coefficients,” in 2021 8th International Conference on Signal Processing and Integrated Networks (SPIN). Noida, India: IEEE, Aug. 2021, pp. 796–799, doi: 10.1109/SPIN52536.2021.9565979.
M. Oraibi and A. Radhi, “Enhancement Digital Forensic Approach for Inter-Frame Video Forgery Detection Using a Deep Learning Technique,” Iraqi Journal of Science, vol. 63, no. 6, pp. 2686–2701, 2022, doi: 10.24996/ijs.2022.63.6.34.
J. Deng, W. Dong, R. Socher, L.-J. Li, Kai Li, and Li Fei-Fei, “ImageNet: A large-scale hierarchical image database,” in 2009 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 2009, pp. 248–255, doi: 10.1109/CVPR.2009.5206848.
M. Saddique, K. Asghar, U. Bajwa, M. Hussain, H. Aboalsamh, and Z. Habib, “Classification of authentic and tampered video using motion residual and parasitic layers,” IEEE Access, vol. 8, pp. 56782–56797, 2020, doi: 10.1109/ACCESS.2020.2980951.
V. Kumar, V. Kansal, and M. Gaur, “Multiple Forgery Detection in Video Using Convolution Neural Network,” Computers, Materials and Continua, vol. 73, no. 1, pp. 1347–1364, 2022, doi: 10.32604/cmc.2022.023545.
A. Bavrina, “Method for Frame Removal Detection in Static Camera Surveillance Video,” in Proc. - IEEE Int. Conf. Inf. Technol. Nanotechnol., ITNT. Institute of Electrical and Electronics Engineers Inc., 2023, doi: 10.1109/ITNT57377.2023.10139053.
Q. Abbas, M. E. A. Ibrahim, and M. A. Jaffar, “Video scene analysis: an overview and challenges on deep learning algorithms,” Multimedia Tools and Applications, vol. 77, no. 16, pp. 20415–20453, Aug. 2018, doi: 10.1007/s11042-017-5438-7.
S. Ferreira, M. Antunes, and M. E. Correia, “Exposing Manipulated Photos and Videos in Digital Forensics Analysis,” Journal of Imaging, vol. 7, no. 7, p. 102, Jul. 2021, doi: 10.3390/jimaging7070102.
X. Jin, Y. Su, and P. Jing, “Video frame deletion detection based on time–frequency analysis,” Journal of Visual Communication and Image Representation, vol. 83, p. 103436, Feb. 2022, doi: 10.1016/j.jvcir.2022.103436.
X. H. Nguyen, “VIFFD - a dataset for detecting video inter-frame forgeries,” 2020, doi: 10.17632/R3SS3V53SJ.6.
A. R. Z. Khurram Soomro and M. Shah, “Ucf101: A dataset of 101 human action classes from videos in the wild.” in CRCV, 2012, doi: 10.48550/arXiv.1212.0402.
C. Tinipuclla, J. Ceron, and P. Shiguihara, “Frame Deletion Detection in Videos Using Convolutional Neural Networks,” in 2024 IEEE ANDESCON. Cusco, Peru: IEEE, Sep. 2024, pp. 1–6, doi: 10.1109/ANDESCON61840.2024.10755836.
L. Yu, H. Wang, Q. Han, X. Niu, S. Yiu, J. Fang, and Z. Wang, “Exposing frame deletion by detecting abrupt changes in video streams,” Neurocomputing, vol. 205, pp. 84–91, Sep. 2016, doi: 10.1016/j.neucom.2016.03.051.
C. Feng, Z. Xu, S. Jia, W. Zhang, and Y. Xu, “Motion-Adaptive Frame Deletion Detection for Digital Video Forensics,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 27, no. 12, pp. 2543–2554, Dec. 2017, doi: 10.1109/TCSVT.2016.2593612.
C. C. Huang, C. E. Lee, and V. L. L. Thing, “A Novel Video Forgery Detection Model Based on Triangular Polarity Feature Classification:,” International Journal of Digital Crime and Forensics, vol. 12, no. 1, pp. 14–34, Jan. 2020, doi: 10.4018/IJDCF.2020010102.
Y. Wang, Y. Hu, A. W.-C. Liew, and C.-T. Li, “ENF Based Video Forgery Detection Algorithm:,” International Journal of Digital Crime and Forensics, vol. 12, no. 1, pp. 131–156, Jan. 2020, doi: 10.4018/IJDCF.2020010107.
S. Li and H. Huo, “Frame Deletion Detection Based on Optical Flow Orientation Variation,” IEEE Access, vol. 9, pp. 37196–37209, 2021, doi: 10.1109/ACCESS.2021.3061586.
J. Bakas, R. Naskar, and S. Bakshi, “Detection and localization of inter-frame forgeries in videos based on macroblock variation and motion vector analysis,” Computers & Electrical Engineering, vol. 89, p. 106929, Jan. 2021, doi: 10.1016/j.compeleceng.2020.106929.
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” 2015, doi: 10.48550/arXiv.1512.03385.
G. Huang, Z. Liu, L. van der Maaten, and K. Q. Weinberger, “Densely connected convolutional networks,” 2018, doi: 10.48550/arXiv.1608.06993.
B. Zoph, V. Vasudevan, J. Shlens, and Q. V. Le, “Learning transferable architectures for scalable image recognition,” 2018, doi: 10.48550/arXiv.1707.07012.
K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” 2015, doi: 10.48550/arXiv.1409.1556.
C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, “Rethinking the inception architecture for computer vision,” 2015, doi: 10.48550/arXiv.1512.00567.
