HaVQA: A Dataset for Visual Question Answering and Multimodal Research in Hausa Language

• URL: http://arxiv.org/abs/2305.17690v1
• Date: Sun, 28 May 2023 10:55:31 GMT
• Title: HaVQA: A Dataset for Visual Question Answering and Multimodal Research in Hausa Language
• Authors: Shantipriya Parida, Idris Abdulmumin, Shamsuddeen Hassan Muhammad, Aneesh Bose, Guneet Singh Kohli, Ibrahim Said Ahmad, Ketan Kotwal, Sayan Deb Sarkar, Ond\v{r}ej Bojar, Habeebah Adamu Kakudi
• Abstract summary: HaVQA is the first multimodal dataset for visual question-answering tasks in the Hausa language. The dataset was created by manually translating 6,022 English question-answer pairs, which are associated with 1,555 unique images from the Visual Genome dataset.
• Score: 1.3476084087665703
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: This paper presents HaVQA, the first multimodal dataset for visual question-answering (VQA) tasks in the Hausa language. The dataset was created by manually translating 6,022 English question-answer pairs, which are associated with 1,555 unique images from the Visual Genome dataset. As a result, the dataset provides 12,044 gold standard English-Hausa parallel sentences that were translated in a fashion that guarantees their semantic match with the corresponding visual information. We conducted several baseline experiments on the dataset, including visual question answering, visual question elicitation, text-only and multimodal machine translation.

Related papers

• WorldCuisines: A Massive-Scale Benchmark for Multilingual and Multicultural Visual Question Answering on Global Cuisines [74.25764182510295]
  Vision Language Models (VLMs) often struggle with culture-specific knowledge, particularly in languages other than English. We introduce World Cuisines, a massive-scale benchmark for multilingual and multicultural, visually grounded language understanding. This benchmark includes a visual question answering (VQA) dataset with text-image pairs across 30 languages and dialects, spanning 9 language families and featuring over 1 million data points.
  arXiv  Detail & Related papers  (2024-10-16T16:11:49Z)
• CVQA: Culturally-diverse Multilingual Visual Question Answering Benchmark [68.21939124278065]
  Culturally-diverse multilingual Visual Question Answering benchmark designed to cover a rich set of languages and cultures. CVQA includes culturally-driven images and questions from across 30 countries on four continents, covering 31 languages with 13 scripts, providing a total of 10k questions. We benchmark several Multimodal Large Language Models (MLLMs) on CVQA, and show that the dataset is challenging for the current state-of-the-art models.
  arXiv  Detail & Related papers  (2024-06-10T01:59:00Z)
• Dataset and Benchmark for Urdu Natural Scenes Text Detection, Recognition and Visual Question Answering [50.52792174648067]
  This initiative seeks to bridge the gap between textual and visual comprehension. We propose a new multi-task Urdu scene text dataset comprising over 1000 natural scene images. We provide fine-grained annotations for text instances, addressing the limitations of previous datasets.
  arXiv  Detail & Related papers  (2024-05-21T06:48:26Z)
• MTVQA: Benchmarking Multilingual Text-Centric Visual Question Answering [58.92057773071854]
  We introduce MTVQA, the first benchmark featuring high-quality human expert annotations across 9 diverse languages. MTVQA is the first benchmark featuring high-quality human expert annotations across 9 diverse languages.
  arXiv  Detail & Related papers  (2024-05-20T12:35:01Z)
• ViTextVQA: A Large-Scale Visual Question Answering Dataset for Evaluating Vietnamese Text Comprehension in Images [1.2529442734851663]
  We introduce the first large-scale dataset in Vietnamese specializing in the ability to understand text appearing in images. We uncover the significance of the order in which tokens in OCR text are processed and selected to formulate answers.
  arXiv  Detail & Related papers  (2024-04-16T15:28:30Z)
• ViCLEVR: A Visual Reasoning Dataset and Hybrid Multimodal Fusion Model for Visual Question Answering in Vietnamese [1.6340299456362617]
  We introduce the ViCLEVR dataset, a pioneering collection for evaluating various visual reasoning capabilities in Vietnamese. We conduct a comprehensive analysis of contemporary visual reasoning systems, offering valuable insights into their strengths and limitations. We present PhoVIT, a comprehensive multimodal fusion that identifies objects in images based on questions.
  arXiv  Detail & Related papers  (2023-10-27T10:44:50Z)
• Making the V in Text-VQA Matter [1.2962828085662563]
  Text-based VQA aims at answering questions by reading the text present in the images. Recent studies have shown that the question-answer pairs in the dataset are more focused on the text present in the image. The models trained on this dataset predict biased answers due to the lack of understanding of visual context.
  arXiv  Detail & Related papers  (2023-08-01T05:28:13Z)
• EVJVQA Challenge: Multilingual Visual Question Answering [1.4641199499831683]
  Visual Question Answering (VQA) is a challenging task of natural language processing (NLP) and computer vision (CV) EVJVQA is used as a benchmark dataset for the challenge of multilingual visual question answering at the 9th Workshop on Vietnamese Language and Speech Processing (VLSP 2022) We present details of the organization of the challenge, an overview of the methods employed by shared-task participants, and the results.
  arXiv  Detail & Related papers  (2023-02-23T02:38:39Z)
• Towards Complex Document Understanding By Discrete Reasoning [77.91722463958743]
  Document Visual Question Answering (VQA) aims to understand visually-rich documents to answer questions in natural language. We introduce a new Document VQA dataset, named TAT-DQA, which consists of 3,067 document pages and 16,558 question-answer pairs. We develop a novel model named MHST that takes into account the information in multi-modalities, including text, layout and visual image, to intelligently address different types of questions.
  arXiv  Detail & Related papers  (2022-07-25T01:43:19Z)
• Grounding Answers for Visual Questions Asked by Visually Impaired People [16.978747012406266]
  VizWiz-VQA-Grounding is the first dataset that visually grounds answers to visual questions asked by people with visual impairments. We analyze our dataset and compare it with five VQA-Grounding datasets to demonstrate what makes it similar and different.
  arXiv  Detail & Related papers  (2022-02-04T06:47:16Z)
• Visual Question Answering on Image Sets [70.4472272672716]
  We introduce the task of Image-Set Visual Question Answering (ISVQA), which generalizes the commonly studied single-image VQA problem to multi-image settings. Taking a natural language question and a set of images as input, it aims to answer the question based on the content of the images. The questions can be about objects and relationships in one or more images or about the entire scene depicted by the image set.
  arXiv  Detail & Related papers  (2020-08-27T08:03:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.