Multi-Level Multi-Modal Cross-Attention Network for Fake News Detection

• Main Authors: Long Ying, Hui Yu, Jinguang Wang, Yongze Ji, Shengsheng Qian
• Format: Article
• Language: English
• Published: IEEE 2021-01-01
• Series: IEEE Access
• Subjects: Multi-level neural networks; fake news detection; multi-modal fusion
• Online Access: https://ieeexplore.ieee.org/document/9541113/

With the development of the Mobile Internet, more and more users publish multi-modal posts on social media platforms. Fake news detection has become an increasingly challenging task. Although there are many works using deep schemes to extract and combine textual and visual representation in the post, most existing methods do not sufficiently utilize the complementary multi-modal information containing semantic concepts and entities to complement and enhance each modality. Moreover, these methods do not model and incorporate the rich multi-level semantics of text information to improve fake news detection tasks. In this paper, we propose a novel end-to-end <italic>Multi-level Multi-modal Cross-attention Network</italic> (MMCN) which exploits the multi-level semantics of textual content and jointly integrates the relationships of duplicate and different modalities (textual and visual modality) of social multimedia posts in a unified framework. Pre-trained BERT and ResNet models are employed to generate high-quality representations for text words and image regions respectively. A multi-modal cross-attention network is then designed to fuse the feature embeddings of the text words and image regions by simultaneously considering data relationships in duplicate and different modalities. Specially, due to different layers of the transformer architecture have different feature representations, we employ a multi-level encoding network to capture the rich multi-level semantics to enhance the presentations of posts. Extensive experiments on the two public datasets (WEIBO and PHEME) demonstrate that compared with the state-of-the-art models, the proposed MMCN has an advantageous performance.