Data and Code for "Seeing Through Deception: Uncovering Misleading Creator Intent in Multimodal News with Vision-Language Models" (ICLR 2026)

This repo contains the data and code for the following paper:

Jiaying Wu, Fanxiao Li, Zihang Fu, Min-Yen Kan, Bryan Hooi. Seeing Through Deception: Uncovering Misleading Creator Intent in Multimodal News with Vision-Language Models, International Conference on Learning Representations (ICLR) 2026. [Paper PDF]

Abstract

The impact of multimodal misinformation arises not only from factual inaccuracies but also from the misleading narratives that creators deliberately embed. Interpreting such creator intent is therefore essential for multimodal misinformation detection (MMD) and effective information governance. To this end, we introduce DeceptionDecoded, a large-scale benchmark of 12,000 image-caption pairs grounded in trustworthy reference articles, created using an intent-guided simulation framework that models both the desired influence and the execution plan of news creators. The dataset captures both misleading and non-misleading cases, spanning manipulations across visual and textual modalities, and supports three intent-centric tasks: (1) misleading intent detection, (2) misleading source attribution, and (3) creator desire inference. We evaluate 14 state-of-the-art vision-language models (VLMs) and find that they struggle with intent reasoning, often relying on shallow cues such as surface-level alignment, stylistic polish, or heuristic authenticity signals. To bridge this, our framework systematically synthesizes data that enables models to learn implication-level intent reasoning. Models trained on DeceptionDecoded demonstrate strong transferability to real-world MMD, validating our framework as both a benchmark to diagnose VLM fragility and a data synthesis engine that provides high-quality, intent-focused resources for enhancing robustness in real-world multimodal misinformation governance.

Requirements

python==3.11.5
openai==2.6.0

Data

The DeceptionDecoded benchmark is constructed based on the VisualNews repository of trustworthy multimodal news reports (Liu et al., 2021).

The full DeceptionDecoded dataset is available under gated access via the [HuggingFace Dataset Link].

After obtaining access and downloading the dataset from HuggingFace, place the following files directly under the main directory:

deceptiondecoded.json
visualnews_data_mapping.json
data.tar.gz

Then extract the data archive:

tar -xvzf data.tar.gz

This will create the data/ folder.

Additional dataset details are provided in the HuggingFace dataset card.

Run Evaluation

We provide an example evaluation script in src/get_predictions.py.

This script queries GPT-4o and GPT-4o-mini through the OpenAI API to obtain creator intent misleadingness predictions. It can be easily adapted to other VLMs by following the model configurations listed in Table 11 on Page 23 of our manuscript, and adapting the get_response function accordingly.

Model predictions are saved under the responses/ directory.

Contact

jiayingwu [at] u.nus.edu

Citation

If you find this repo or our DeceptionDecoded benchmark useful for your research, please consider citing our paper:

@inproceedings{
    wu2026seeing,
    title={Seeing Through Deception: Uncovering Misleading Creator Intent in Multimodal News with Vision-Language Models},
    author={Jiaying Wu and Fanxiao Li and Zihang Fu and Min-Yen Kan and Bryan Hooi},
    booktitle={The Fourteenth International Conference on Learning Representations},
    year={2026},
}