MMClima: A Framework for Multimodal Climate Science Data and Evaluation

• URL: http://arxiv.org/abs/2606.10194v1
• Date: Mon, 08 Jun 2026 21:30:34 GMT
• Title: MMClima: A Framework for Multimodal Climate Science Data and Evaluation
• Abstract summary: MMClima is a large-scale multimodal climate question answering framework with 104k+ expert-validated question-answer pairs spanning articles, video transcriptions, and figures.<n>We benchmark state-of-the-art multimodal language models on tasks requiring factual recall, visual interpretation, and cross-modal synthesis.<n>We release the dataset, evaluation pipeline, fine-tuned model weights, and data creation framework to support standardized multimodal evaluation for climate science.
• Score: 18.546813902218265
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Climate change research increasingly requires AI systems that reason across text, dynamic visual content, and scientific figures, yet existing climate QA benchmarks are small, mostly textual, and cover a narrow range of models. We introduce MMClima, a large-scale multimodal climate question answering framework with 104k+ expert-validated question-answer pairs spanning articles, video transcriptions, and figures across five core climate science domains. MMClima is constructed via automated claim extraction and QA synthesis with human-in-the-loop validation to ensure both scale and reliability. Using MMClima, we benchmark state-of-the-art multimodal language models on tasks requiring factual recall, visual interpretation, and cross-modal synthesis. We additionally fine-tune on the textual split to produce mmclima-70b-txt, a domain-adapted baseline that outperforms strong open- and closed-source models on textual QA. We release the dataset, evaluation pipeline, fine-tuned model weights, and data creation framework to support standardized multimodal evaluation for climate science.

Related papers

• SciMDR: Benchmarking and Advancing Scientific Multimodal Document Reasoning [59.93568326270048]
  We introduce the synthesize synthesize framework, which generates faithful, isolated QA pairs on focused segments.<n>We construct SciMDR, a large-scale training dataset for crossmodal comprehension.<n>Experiments demonstrate that SciMDR models achieve significant improvements across multiple scientific QA benchmarks.
  arXiv  Detail & Related papers  (2026-03-12T17:57:52Z)
• RPC-Bench: A Fine-grained Benchmark for Research Paper Comprehension [65.81339691942757]
  RPC-Bench is a large-scale question-answering benchmark built from review-rebuttal exchanges of high-quality computer science papers.<n>We design a fine-grained taxonomy aligned with the scientific research flow to assess models' ability to understand and answer why, what, and how questions in scholarly contexts.
  arXiv  Detail & Related papers  (2026-01-14T11:37:00Z)
• ClimateBench-M: A Multi-Modal Climate Data Benchmark with a Simple Generative Method [61.76389719956301]
  We contribute a multi-modal climate benchmark, i.e., ClimateBench-M, which aligns time series climate data from ERA5, extreme weather events data from NOAA, and satellite image data from NASA.<n>Under each data modality, we also propose a simple but strong generative method that could produce competitive performance in weather forecasting, thunderstorm alerts, and crop segmentation tasks.
  arXiv  Detail & Related papers  (2025-04-10T02:22:23Z)
• CliME: Evaluating Multimodal Climate Discourse on Social Media and the Climate Alignment Quotient (CAQ) [14.065907685322097]
  CliME is a first-of-its-kind multimodal dataset, comprising 2579 Twitter and Reddit posts.<n>The benchmark features a diverse collection of humorous memes and skeptical posts, capturing how these formats distill complex issues into viral narratives that shape public opinion and policy discussions.<n>We present the Climate Alignment Quotient (CAQ), a novel metric comprising five distinct dimensions: Articulation, Evidence, Resonance, Transition, and Specificity.<n>Our findings, based on the CAQ metric, indicate that while most evaluated LLMs perform relatively well in Criticality and Justice, they consistently underperform on the Actionability axis.
  arXiv  Detail & Related papers  (2025-04-04T20:01:00Z)
• mmE5: Improving Multimodal Multilingual Embeddings via High-quality Synthetic Data [71.352883755806]
  Multimodal embedding models have gained significant attention for their ability to map data from different modalities, such as text and images, into a unified representation space.<n>However, the limited labeled multimodal data often hinders embedding performance.<n>Recent approaches have leveraged data synthesis to address this problem, yet the quality of synthetic data remains a critical bottleneck.
  arXiv  Detail & Related papers  (2025-02-12T15:03:33Z)
• AtmosSci-Bench: Evaluating the Recent Advance of Large Language Model for Atmospheric Science [9.757412158154514]
  We present a novel benchmark designed to assess large language models (LLMs) across five core categories of atmospheric science problems.<n>AtmosSci-Bench features a dual-format design comprising both multiple-choice questions (MCQs) and open-ended questions (OEQs)<n>We conduct a comprehensive evaluation of representative LLMs, categorized into four groups: instruction-tuned models, advanced reasoning models, math-augmented models, and domain-specific climate models.
  arXiv  Detail & Related papers  (2025-02-03T08:50:46Z)
• ClimaQA: An Automated Evaluation Framework for Climate Question Answering Models [38.05357439484919]
  We develop ClimaGen, an adaptive learning framework that generates question-answer pairs from graduate textbooks with climate scientists in the loop.<n>We present ClimaQA-Gold, an expert-annotated benchmark dataset alongside ClimaQA-Silver, a large-scale, comprehensive synthetic QA dataset for climate science.
  arXiv  Detail & Related papers  (2024-10-22T05:12:19Z)
• MMSci: A Dataset for Graduate-Level Multi-Discipline Multimodal Scientific Understanding [59.41495657570397]
  We present a comprehensive dataset compiled from Nature Communications articles covering 72 scientific fields.<n>We evaluated 19 proprietary and open-source models on two benchmark tasks, figure captioning and multiple-choice, and conducted human expert annotation.<n>Fine-tuning Qwen2-VL-7B with our task-specific data achieved better performance than GPT-4o and even human experts in multiple-choice evaluations.
  arXiv  Detail & Related papers  (2024-07-06T00:40:53Z)
• Synthetic Multimodal Question Generation [60.33494376081317]
  Multimodal Retrieval Augmented Generation (MMRAG) is a powerful approach to question-answering over multimodal documents. We propose SMMQG, a synthetic data generation framework that generates question and answer pairs directly from multimodal documents. We use SMMQG to generate an MMRAG dataset of 1024 questions over Wikipedia documents and evaluate state-of-the-art models using it.
  arXiv  Detail & Related papers  (2024-07-02T12:57:42Z)
• ClimateGPT: Towards AI Synthesizing Interdisciplinary Research on Climate Change [21.827936253363603]
  This paper introduces ClimateGPT, a model family of domain-specific large language models that synthesize interdisciplinary research on climate change. We trained two 7B models from scratch on a science-oriented dataset of 300B tokens. ClimateGPT-7B, 13B and 70B are continuously pre-trained from Llama2 on a domain-specific dataset of 4.2B tokens.
  arXiv  Detail & Related papers  (2024-01-17T23:29:46Z)
• Arabic Mini-ClimateGPT : A Climate Change and Sustainability Tailored Arabic LLM [77.17254959695218]
  Large Language Models (LLMs) like ChatGPT and Bard have shown impressive conversational abilities and excel in a wide variety of NLP tasks. We propose a light-weight Arabic Mini-ClimateGPT that is built on an open-source LLM and is specifically fine-tuned on a conversational-style instruction tuning Arabic dataset Clima500-Instruct. Our model surpasses the baseline LLM in 88.3% of cases during ChatGPT-based evaluation.
  arXiv  Detail & Related papers  (2023-12-14T22:04:07Z)
• ClimateSet: A Large-Scale Climate Model Dataset for Machine Learning [26.151056828513962]
  Climate models have been key for assessing the impact of climate change and simulating future climate scenarios. The machine learning (ML) community has taken an increased interest in supporting climate scientists' efforts on various tasks such as climate model emulation, downscaling, and prediction tasks. Here, we introduce ClimateSet, a dataset containing the inputs and outputs of 36 climate models from the Input4MIPs and CMIP6 archives.
  arXiv  Detail & Related papers  (2023-11-07T04:55:36Z)
• ClimaX: A foundation model for weather and climate [51.208269971019504]
  ClimaX is a deep learning model for weather and climate science. It can be pre-trained with a self-supervised learning objective on climate datasets. It can be fine-tuned to address a breadth of climate and weather tasks.
  arXiv  Detail & Related papers  (2023-01-24T23:19:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.