M4FC: a Multimodal, Multilingual, Multicultural, Multitask Real-World Fact-Checking Dataset

• URL: http://arxiv.org/abs/2510.23508v1
• Date: Mon, 27 Oct 2025 16:44:35 GMT
• Title: M4FC: a Multimodal, Multilingual, Multicultural, Multitask Real-World Fact-Checking Dataset
• Authors: Jiahui Geng, Jonathan Tonglet, Iryna Gurevych,
• Abstract summary: M4FC is a new real-world dataset comprising 4,982 images paired with 6,980 claims.<n>Each claim is available in one or two out of ten languages.<n>M4FC spans six multimodal fact-checking tasks: visual claim extraction, claimant intent prediction, fake detection, image contextualization, location verification, and verdict prediction.
• Score: 54.271294779453235
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Existing real-world datasets for multimodal automated fact-checking have multiple limitations: they contain few instances, focus on only one or two languages and tasks, suffer from evidence leakage, or depend on external sets of news articles for sourcing true claims. To address these shortcomings, we introduce M4FC, a new real-world dataset comprising 4,982 images paired with 6,980 claims. The images, verified by professional fact-checkers from 22 organizations, represent diverse cultural and geographic contexts. Each claim is available in one or two out of ten languages. M4FC spans six multimodal fact-checking tasks: visual claim extraction, claimant intent prediction, fake detection, image contextualization, location verification, and verdict prediction. We provide baseline results for all tasks and analyze how combining intermediate tasks influence downstream verdict prediction performance. We make our dataset and code available.

Related papers

• MultiCaption: Detecting disinformation using multilingual visual claims [10.69065586825833]
  We present MultiCaption, a dataset specifically designed for detecting contradictions in visual claims.<n>The resulting dataset comprises 11,088 visual claims in 64 languages.<n>The gains from multilingual training and testing highlight the dataset's potential for building effective multilingual fact-checking pipelines.
  arXiv  Detail & Related papers  (2026-01-16T11:57:07Z)
• MultiFinBen: Benchmarking Large Language Models for Multilingual and Multimodal Financial Application [118.63802040274999]
  MultiFinBen is the first expert-annotated multilingual (five languages) and multimodal benchmark for evaluating LLMs in realistic financial contexts.<n>Financial reasoning tests cross-lingual evidence integration from filings and news, and financial OCR, which extracts structured text from scanned documents.<n> evaluating 21 leading LLMs shows that even frontier multimodal models like GPT-4o achieve only 46.01% overall, stronger on vision and audio but dropping sharply in multilingual settings.
  arXiv  Detail & Related papers  (2025-06-16T22:01:49Z)
• Piecing It All Together: Verifying Multi-Hop Multimodal Claims [39.68850054331197]
  We introduce a new task: multi-hop multimodal claim verification.<n>This task challenges models to reason over multiple pieces of evidence from diverse sources, including text, images, and tables.<n>We construct MMCV, a large-scale dataset comprising 15k multi-hop claims paired with multimodal evidence, with additional input from human feedback.
  arXiv  Detail & Related papers  (2024-11-14T16:01:33Z)
• MMSci: A Dataset for Graduate-Level Multi-Discipline Multimodal Scientific Understanding [59.41495657570397]
  We present a comprehensive dataset compiled from Nature Communications articles covering 72 scientific fields.<n>We evaluated 19 proprietary and open-source models on two benchmark tasks, figure captioning and multiple-choice, and conducted human expert annotation.<n>Fine-tuning Qwen2-VL-7B with our task-specific data achieved better performance than GPT-4o and even human experts in multiple-choice evaluations.
  arXiv  Detail & Related papers  (2024-07-06T00:40:53Z)
• 3AM: An Ambiguity-Aware Multi-Modal Machine Translation Dataset [90.95948101052073]
  We introduce 3AM, an ambiguity-aware MMT dataset comprising 26,000 parallel sentence pairs in English and Chinese. Our dataset is specifically designed to include more ambiguity and a greater variety of both captions and images than other MMT datasets. Experimental results show that MMT models trained on our dataset exhibit a greater ability to exploit visual information than those trained on other MMT datasets.
  arXiv  Detail & Related papers  (2024-04-29T04:01:30Z)
• Multi-modal Stance Detection: New Datasets and Model [56.97470987479277]
  We study multi-modal stance detection for tweets consisting of texts and images. We propose a simple yet effective Targeted Multi-modal Prompt Tuning framework (TMPT) TMPT achieves state-of-the-art performance in multi-modal stance detection.
  arXiv  Detail & Related papers  (2024-02-22T05:24:19Z)
• Multilingual Multimodal Learning with Machine Translated Text [27.7207234512674]
  We investigate whether machine translating English multimodal data can be an effective proxy for the lack of readily available multilingual data. We propose two metrics for automatically removing such translations from the resulting datasets. In experiments on five tasks across 20 languages in the IGLUE benchmark, we show that translated data can provide a useful signal for multilingual multimodal learning.
  arXiv  Detail & Related papers  (2022-10-24T11:41:20Z)
• MuRAG: Multimodal Retrieval-Augmented Generator for Open Question Answering over Images and Text [58.655375327681774]
  We propose the first Multimodal Retrieval-Augmented Transformer (MuRAG) MuRAG accesses an external non-parametric multimodal memory to augment language generation. Our results show that MuRAG achieves state-of-the-art accuracy, outperforming existing models by 10-20% absolute on both datasets.
  arXiv  Detail & Related papers  (2022-10-06T13:58:03Z)
• CHEF: A Pilot Chinese Dataset for Evidence-Based Fact-Checking [55.75590135151682]
  CHEF is the first CHinese Evidence-based Fact-checking dataset of 10K real-world claims. The dataset covers multiple domains, ranging from politics to public health, and provides annotated evidence retrieved from the Internet.
  arXiv  Detail & Related papers  (2022-06-06T09:11:03Z)
• MM-Claims: A Dataset for Multimodal Claim Detection in Social Media [7.388174516838141]
  We introduce a novel dataset, MM-Claims, which consists of tweets and corresponding images over three topics: COVID-19, Climate Change and broadly Technology. We describe the dataset in detail, evaluate strong unimodal and multimodal baselines, and analyze the potential and drawbacks of current models.
  arXiv  Detail & Related papers  (2022-05-04T10:43:58Z)
• Logically at the Factify 2022: Multimodal Fact Verification [2.8914815569249823]
  This paper describes our participant system for the multi-modal fact verification (Factify) challenge at AAAI 2022. Two baseline approaches are proposed and explored including an ensemble model and a multi-modal attention network. Our best model is ranked first in leaderboard which obtains a weighted average F-measure of 0.77 on both validation and test set.
  arXiv  Detail & Related papers  (2021-12-16T23:34:07Z)
• X-FACT: A New Benchmark Dataset for Multilingual Fact Checking [21.2633064526968]
  We introduce X-FACT: the largest publicly available multilingual dataset for factual verification of naturally existing real-world claims. The dataset contains short statements in 25 languages and is labeled for veracity by expert fact-checkers.
  arXiv  Detail & Related papers  (2021-06-17T05:09:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.