A Benchmark Dataset and a Framework for Urdu Multimodal Named Entity Recognition

• URL: http://arxiv.org/abs/2505.05148v1
• Date: Thu, 08 May 2025 11:38:20 GMT
• Title: A Benchmark Dataset and a Framework for Urdu Multimodal Named Entity Recognition
• Authors: Hussain Ahmad, Qingyang Zeng, Jing Wan,
• Abstract summary: We introduce the U-MNER framework and release the Twitter2015-Urdu dataset.<n>Adapted from the widely used Twitter2015 dataset, it is annotated with Urdu-specific grammar rules.<n>Our model achieves state-of-the-art performance on the Twitter2015-Urdu dataset.
• Score: 1.9500421038452647
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The emergence of multimodal content, particularly text and images on social media, has positioned Multimodal Named Entity Recognition (MNER) as an increasingly important area of research within Natural Language Processing. Despite progress in high-resource languages such as English, MNER remains underexplored for low-resource languages like Urdu. The primary challenges include the scarcity of annotated multimodal datasets and the lack of standardized baselines. To address these challenges, we introduce the U-MNER framework and release the Twitter2015-Urdu dataset, a pioneering resource for Urdu MNER. Adapted from the widely used Twitter2015 dataset, it is annotated with Urdu-specific grammar rules. We establish benchmark baselines by evaluating both text-based and multimodal models on this dataset, providing comparative analyses to support future research on Urdu MNER. The U-MNER framework integrates textual and visual context using Urdu-BERT for text embeddings and ResNet for visual feature extraction, with a Cross-Modal Fusion Module to align and fuse information. Our model achieves state-of-the-art performance on the Twitter2015-Urdu dataset, laying the groundwork for further MNER research in low-resource languages.

Related papers

• Multimodal Coreference Resolution for Chinese Social Media Dialogues: Dataset and Benchmark Approach [21.475881921929236]
  Multimodal coreference resolution (MCR) aims to identify mentions referring to the same entity across different modalities.<n>We introduce TikTalkCoref, the first Chinese multimodal coreference dataset for social media in real-world scenarios.
  arXiv  Detail & Related papers  (2025-04-19T15:15:59Z)
• RoLargeSum: A Large Dialect-Aware Romanian News Dataset for Summary, Headline, and Keyword Generation [2.3577273565334522]
  RoLargeSum is a novel large-scale summarization dataset for the Romanian language.<n>It was crawled from various publicly available news websites from Romania and the Republic of Moldova.
  arXiv  Detail & Related papers  (2024-12-15T21:27:33Z)
• MURI: High-Quality Instruction Tuning Datasets for Low-Resource Languages via Reverse Instructions [54.08017526771947]
  Multilingual Reverse Instructions (MURI) generates high-quality instruction tuning datasets for low-resource languages. MURI produces instruction-output pairs from existing human-written texts in low-resource languages. Our dataset, MURI-IT, includes more than 2 million instruction-output pairs across 200 languages.
  arXiv  Detail & Related papers  (2024-09-19T17:59:20Z)
• TRINS: Towards Multimodal Language Models that Can Read [61.17806538631744]
  TRINS is a Text-Rich image INStruction dataset. It contains 39,153 text-rich images, captions, and 102,437 questions. We introduce a Language-vision Reading Assistant (LaRA) which is good at understanding textual content within images.
  arXiv  Detail & Related papers  (2024-06-10T18:52:37Z)
• 3AM: An Ambiguity-Aware Multi-Modal Machine Translation Dataset [90.95948101052073]
  We introduce 3AM, an ambiguity-aware MMT dataset comprising 26,000 parallel sentence pairs in English and Chinese. Our dataset is specifically designed to include more ambiguity and a greater variety of both captions and images than other MMT datasets. Experimental results show that MMT models trained on our dataset exhibit a greater ability to exploit visual information than those trained on other MMT datasets.
  arXiv  Detail & Related papers  (2024-04-29T04:01:30Z)
• IndicSTR12: A Dataset for Indic Scene Text Recognition [33.194567434881314]
  This paper proposes the largest and most comprehensive real dataset - IndicSTR12 - and benchmarking STR performance on 12 major Indian languages. The size and complexity of the proposed dataset are comparable to those of existing Latin contemporaries. The dataset contains over 27000 word-images gathered from various natural scenes, with over 1000 word-images for each language.
  arXiv  Detail & Related papers  (2024-03-12T18:14:48Z)
• UTRNet: High-Resolution Urdu Text Recognition In Printed Documents [5.179738379203527]
  We propose a novel approach to address the challenges of printed Urdu text recognition. Our proposed UTRNet architecture, a hybrid CNN-RNN model, demonstrates state-of-the-art performance on benchmark datasets.
  arXiv  Detail & Related papers  (2023-06-27T20:09:56Z)
• XTREME-UP: A User-Centric Scarce-Data Benchmark for Under-Represented Languages [105.54207724678767]
  Data scarcity is a crucial issue for the development of highly multilingual NLP systems. We propose XTREME-UP, a benchmark defined by its focus on the scarce-data scenario rather than zero-shot. XTREME-UP evaluates the capabilities of language models across 88 under-represented languages over 9 key user-centric technologies.
  arXiv  Detail & Related papers  (2023-05-19T18:00:03Z)
• Beyond Triplet: Leveraging the Most Data for Multimodal Machine Translation [53.342921374639346]
  Multimodal machine translation aims to improve translation quality by incorporating information from other modalities, such as vision. Previous MMT systems mainly focus on better access and use of visual information and tend to validate their methods on image-related datasets. This paper establishes new methods and new datasets for MMT.
  arXiv  Detail & Related papers  (2022-12-20T15:02:38Z)
• MultiCoNER: A Large-scale Multilingual dataset for Complex Named Entity Recognition [15.805414696789796]
  We present MultiCoNER, a large multilingual dataset for Named Entity Recognition that covers 3 domains (Wiki sentences, questions, and search queries) across 11 languages. This dataset is designed to represent contemporary challenges in NER, including low-context scenarios.
  arXiv  Detail & Related papers  (2022-08-30T20:45:54Z)
• Reinforced Iterative Knowledge Distillation for Cross-Lingual Named Entity Recognition [54.92161571089808]
  Cross-lingual NER transfers knowledge from rich-resource language to languages with low resources. Existing cross-lingual NER methods do not make good use of rich unlabeled data in target languages. We develop a novel approach based on the ideas of semi-supervised learning and reinforcement learning.
  arXiv  Detail & Related papers  (2021-06-01T05:46:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.