Multi-Image Visual Question Answering

• URL: http://arxiv.org/abs/2112.13706v1
• Date: Mon, 27 Dec 2021 14:28:04 GMT
• Title: Multi-Image Visual Question Answering
• Authors: Harsh Raj, Janhavi Dadhania, Akhilesh Bhardwaj
• Abstract summary: We present an empirical study of different feature extraction methods with different loss functions. We propose New dataset for the task of Visual Question Answering with multiple image inputs having only one ground truth. Our final model utilising Resnet + RCNN image features and Bert embeddings, inspired by stacked attention network gives 39% word accuracy and 99% image accuracy on CLEVER+TinyImagenet dataset.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While a lot of work has been done on developing models to tackle the problem of Visual Question Answering, the ability of these models to relate the question to the image features still remain less explored. We present an empirical study of different feature extraction methods with different loss functions. We propose New dataset for the task of Visual Question Answering with multiple image inputs having only one ground truth, and benchmark our results on them. Our final model utilising Resnet + RCNN image features and Bert embeddings, inspired from stacked attention network gives 39% word accuracy and 99% image accuracy on CLEVER+TinyImagenet dataset.

Related papers

• UNK-VQA: A Dataset and a Probe into the Abstention Ability of Multi-modal Large Models [55.22048505787125]
  This paper contributes a comprehensive dataset, called UNK-VQA. We first augment the existing data via deliberate perturbations on either the image or question. We then extensively evaluate the zero- and few-shot performance of several emerging multi-modal large models.
  arXiv  Detail & Related papers  (2023-10-17T02:38:09Z)
• Contrastive Learning of Features between Images and LiDAR [18.211513930388417]
  This work treats learning cross-modal features as a dense contrastive learning problem. To learn good features and not lose generality, we developed a variant of widely used PointNet++ architecture for images. We show that our models indeed learn information from both images as well as LiDAR by visualizing the features.
  arXiv  Detail & Related papers  (2022-06-24T04:35:23Z)
• Learning Enriched Features for Fast Image Restoration and Enhancement [166.17296369600774]
  This paper presents a holistic goal of maintaining spatially-precise high-resolution representations through the entire network. We learn an enriched set of features that combines contextual information from multiple scales, while simultaneously preserving the high-resolution spatial details. Our approach achieves state-of-the-art results for a variety of image processing tasks, including defocus deblurring, image denoising, super-resolution, and image enhancement.
  arXiv  Detail & Related papers  (2022-04-19T17:59:45Z)
• A Better Loss for Visual-Textual Grounding [74.81353762517979]
  Given a textual phrase and an image, the visual grounding problem is defined as the task of locating the content of the image referenced by the sentence. It is a challenging task that has several real-world applications in human-computer interaction, image-text reference resolution, and video-text reference resolution. We propose a model that is able to achieve a higher accuracy than state-of-the-art models thanks to the adoption of a more effective loss function.
  arXiv  Detail & Related papers  (2021-08-11T16:26:54Z)
• AugNet: End-to-End Unsupervised Visual Representation Learning with Image Augmentation [3.6790362352712873]
  We propose AugNet, a new deep learning training paradigm to learn image features from a collection of unlabeled pictures. Our experiments demonstrate that the method is able to represent the image in low dimensional space. Unlike many deep-learning-based image retrieval algorithms, our approach does not require access to external annotated datasets.
  arXiv  Detail & Related papers  (2021-06-11T09:02:30Z)
• Dealing with Missing Modalities in the Visual Question Answer-Difference Prediction Task through Knowledge Distillation [75.1682163844354]
  We address the issues of missing modalities that have arisen from the Visual Question Answer-Difference prediction task. We introduce a model, the "Big" Teacher, that takes the image/question/answer triplet as its input and outperforms the baseline.
  arXiv  Detail & Related papers  (2021-04-13T06:41:11Z)
• Learning Compositional Representation for Few-shot Visual Question Answering [93.4061107793983]
  Current methods of Visual Question Answering perform well on the answers with an amount of training data but have limited accuracy on the novel ones with few examples. We propose to extract the attributes from the answers with enough data, which are later composed to constrain the learning of the few-shot ones. Experimental results on the VQA v2.0 validation dataset demonstrate the effectiveness of our proposed attribute network.
  arXiv  Detail & Related papers  (2021-02-21T10:16:24Z)
• RSVQA: Visual Question Answering for Remote Sensing Data [6.473307489370171]
  This paper introduces the task of visual question answering for remote sensing data (RSVQA) We use questions formulated in natural language and use them to interact with the images. The datasets can be used to train (when using supervised methods) and evaluate models to solve the RSVQA task.
  arXiv  Detail & Related papers  (2020-03-16T17:09:31Z)
• Learning Enriched Features for Real Image Restoration and Enhancement [166.17296369600774]
  convolutional neural networks (CNNs) have achieved dramatic improvements over conventional approaches for image restoration task. We present a novel architecture with the collective goals of maintaining spatially-precise high-resolution representations through the entire network. Our approach learns an enriched set of features that combines contextual information from multiple scales, while simultaneously preserving the high-resolution spatial details.
  arXiv  Detail & Related papers  (2020-03-15T11:04:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.