Chitrakshara: A Large Multilingual Multimodal Dataset for Indian languages

• URL: http://arxiv.org/abs/2603.23521v1
• Date: Fri, 06 Mar 2026 15:01:25 GMT
• Title: Chitrakshara: A Large Multilingual Multimodal Dataset for Indian languages
• Abstract summary: We introduce the Chitrakshara dataset series, covering 11 Indian languages sourced from Common Crawl.<n>It comprises (1) Chitrakshara-IL, a large-scale interleaved pretraining dataset with 193M images, 30B text tokens, and 50M multilingual documents, and (2) Chitrakshara-Cap, which includes 44M image-text pairs with 733M tokens.
• Score: 2.592717137976924
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multimodal research has predominantly focused on single-image reasoning, with limited exploration of multi-image scenarios. Recent models have sought to enhance multi-image understanding through large-scale pretraining on interleaved image-text datasets. However, most Vision-Language Models (VLMs) are trained primarily on English datasets, leading to inadequate representation of Indian languages. To address this gap, we introduce the Chitrakshara dataset series, covering 11 Indian languages sourced from Common Crawl. It comprises (1) Chitrakshara-IL, a large-scale interleaved pretraining dataset with 193M images, 30B text tokens, and 50M multilingual documents, and (2) Chitrakshara-Cap, which includes 44M image-text pairs with 733M tokens. This paper details the data collection pipeline, including curation, filtering, and processing methodologies. Additionally, we present a comprehensive quality and diversity analysis to assess the dataset's representativeness across Indic languages and its potential for developing more culturally inclusive VLMs.

Related papers

• Pragyaan: Designing and Curating High-Quality Cultural Post-Training Datasets for Indian Languages [2.403023083920947]
  Existing open-source datasets often lack multilingual coverage, cultural grounding, and task diversity gaps.<n>We introduce a human-in-the-loop pipeline that combines translations with synthetic expansion to produce reliable and diverse Indic post-training data.<n>Our dataset protocol incorporates several often-overlooked dimensions and emphasize task diversity, multi-turn dialogue, instruction fidelity, safety alignment, and preservation of cultural nuance.
  arXiv  Detail & Related papers  (2025-10-08T13:23:45Z)
• COSMMIC: Comment-Sensitive Multimodal Multilingual Indian Corpus for Summarization and Headline Generation [10.9454163542891]
  This study introduces COSMMIC, a comment-sensitive multimodal, multilingual dataset featuring nine major Indian languages.<n> COSMMIC comprises 4,959 article-image pairs and 24,484 reader comments, with ground-truth summaries available in all included languages.<n>To assess the dataset's effectiveness, we employ state-of-the-art language models such as LLama3 and GPT-4.
  arXiv  Detail & Related papers  (2025-06-18T11:38:23Z)
• mOSCAR: A Large-scale Multilingual and Multimodal Document-level Corpus [52.83121058429025]
  Multimodal Large Language Models (mLLMs) are trained on a large amount of text-image data.<n>mOSCAR is the first large-scale multilingual and multimodal document corpus crawled from the web.<n>It covers 163 languages, 303M documents, 200B tokens and 1.15B images.
  arXiv  Detail & Related papers  (2024-06-13T00:13:32Z)
• OmniCorpus: A Unified Multimodal Corpus of 10 Billion-Level Images Interleaved with Text [112.60163342249682]
  We introduce OmniCorpus, a 10 billion-scale image-text interleaved dataset. Our dataset has 15 times larger scales while maintaining good data quality. We hope this could provide a solid data foundation for future multimodal model research.
  arXiv  Detail & Related papers  (2024-06-12T17:01:04Z)
• CVQA: Culturally-diverse Multilingual Visual Question Answering Benchmark [68.21939124278065]
  Culturally-diverse multilingual Visual Question Answering benchmark designed to cover a rich set of languages and cultures. CVQA includes culturally-driven images and questions from across 30 countries on four continents, covering 31 languages with 13 scripts, providing a total of 10k questions. We benchmark several Multimodal Large Language Models (MLLMs) on CVQA, and show that the dataset is challenging for the current state-of-the-art models.
  arXiv  Detail & Related papers  (2024-06-10T01:59:00Z)
• Multilingual Diversity Improves Vision-Language Representations [97.16233528393356]
  Pre-training on this dataset outperforms using English-only or English-dominated datasets on ImageNet.<n>On a geographically diverse task like GeoDE, we also observe improvements across all regions, with the biggest gain coming from Africa.
  arXiv  Detail & Related papers  (2024-05-27T08:08:51Z)
• IndicSTR12: A Dataset for Indic Scene Text Recognition [33.194567434881314]
  This paper proposes the largest and most comprehensive real dataset - IndicSTR12 - and benchmarking STR performance on 12 major Indian languages. The size and complexity of the proposed dataset are comparable to those of existing Latin contemporaries. The dataset contains over 27000 word-images gathered from various natural scenes, with over 1000 word-images for each language.
  arXiv  Detail & Related papers  (2024-03-12T18:14:48Z)
• Large Multilingual Models Pivot Zero-Shot Multimodal Learning across Languages [76.35234803589412]
  MPM is an effective training paradigm for training large multimodal models in non-English languages. We build large multimodal models VisCPM in image-to-text and text-to-image generation, which achieve state-of-the-art (open-source) performance in Chinese.
  arXiv  Detail & Related papers  (2023-08-23T09:55:41Z)
• MMT: A Multilingual and Multi-Topic Indian Social Media Dataset [1.0413233169366503]
  Social media plays a significant role in cross-cultural communication. A vast amount of this occurs in code-mixed and multilingual form. We introduce a large-scale multilingual, and multi-topic dataset (MMT) collected from Twitter.
  arXiv  Detail & Related papers  (2023-04-02T21:39:00Z)
• Bloom Library: Multimodal Datasets in 300+ Languages for a Variety of Downstream Tasks [0.007696728525672149]
  In total, the initial release of the Bloom Library datasets covers 363 languages across 32 language families. Some of these first-of-their-kind baselines are comparable to state-of-the-art performance for higher-resourced languages.
  arXiv  Detail & Related papers  (2022-10-26T13:45:14Z)
• UC2: Universal Cross-lingual Cross-modal Vision-and-Language Pre-training [52.852163987208826]
  UC2 is the first machine translation-augmented framework for cross-lingual cross-modal representation learning. We propose two novel pre-training tasks, namely Masked Region-to-Token Modeling (MRTM) and Visual Translation Language Modeling (VTLM) Our proposed framework achieves new state-of-the-art on diverse non-English benchmarks while maintaining comparable performance to monolingual pre-trained models on English tasks.
  arXiv  Detail & Related papers  (2021-04-01T08:30:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.