MUN: Image Forgery Localization Based on M³ Encoder and UN Decoder (original) (raw)

Authors

• Yaqi Liu Beijing Electronic Science and Technology Institute
• Shuhuan Chen Institute of Information Engineering, Chinese Academy of Sciences School of Cyber Security, University of Chinese Academy of Sciences
• Haichao Shi Institute of Information Engineering, Chinese Academy of Sciences
• Xiao-Yu Zhang Institute of Information Engineering, Chinese Academy of Sciences
• Song Xiao Beijing Electronic Science and Technology Institute
• Qiang Cai Beijing Key Laboratory of Big Data Technology for Food Safety, Beijing Technology and Business University

DOI:

https://doi.org/10.1609/aaai.v39i6.32606

Abstract

Image forgeries can entirely change the semantic information of an image, and can be used for unscrupulous purposes. In this paper, we propose a novel image forgery localization network named as MUN, which consists of an M^3 encoder and a UN decoder. Firstly, the M^3 encoder is constructed based on a Multi-scale Max-pooling query module to extract Multi-clue forged features. Noiseprint++ is adopted to assist the RGB clue, and its deployment methodology is discussed. A Multi-scale Max-pooling Query (MMQ) module is proposed to integrate RGB and noise features. Secondly, a novel UN decoder is proposed to extract hierarchical features from both top-down and bottom-up directions, reconstructing both high-level and low-level features at the same time. Thirdly, we formulate an IoU-recalibrated Dynamic Cross-Entropy (IoUDCE) loss to dynamically adjust the weights on forged regions according to IoU which can adaptively balance the influence of authentic and forged regions. Last but not least, we propose a data augmentation method, i.e., Deviation Noise Augmentation (DNA), which acquires accessible prior knowledge of RGB distribution to improve the generalization ability. Extensive experiments on publicly available datasets show that MUN outperforms the state-of-the-art works.

How to Cite

Liu, Y., Chen, S., Shi, H., Zhang, X.-Y., Xiao, S., & Cai, Q. (2025). MUN: Image Forgery Localization Based on M³ Encoder and UN Decoder. Proceedings of the AAAI Conference on Artificial Intelligence, 39(6), 5685-5693. https://doi.org/10.1609/aaai.v39i6.32606

Issue

Section

AAAI Technical Track on Computer Vision V