Logically Consistent Loss for Visual Question Answering

• URL: http://arxiv.org/abs/2011.10094v1
• Date: Thu, 19 Nov 2020 20:31:05 GMT
• Title: Logically Consistent Loss for Visual Question Answering
• Authors: Anh-Cat Le-Ngo, Truyen Tran, Santu Rana, Sunil Gupta, Svetha Venkatesh
• Abstract summary: The current advancement in neural-network based Visual Question Answering (VQA) cannot ensure such consistency due to identically distribution (i.i.d.) assumption. We propose a new model-agnostic logic constraint to tackle this issue by formulating a logically consistent loss in the multi-task learning framework. Experiments confirm that the proposed loss formulae and introduction of hybrid-batch leads to more consistency as well as better performance.
• Score: 66.83963844316561
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Given an image, a back-ground knowledge, and a set of questions about an object, human learners answer the questions very consistently regardless of question forms and semantic tasks. The current advancement in neural-network based Visual Question Answering (VQA), despite their impressive performance, cannot ensure such consistency due to identically distribution (i.i.d.) assumption. We propose a new model-agnostic logic constraint to tackle this issue by formulating a logically consistent loss in the multi-task learning framework as well as a data organisation called family-batch and hybrid-batch. To demonstrate usefulness of this proposal, we train and evaluate MAC-net based VQA machines with and without the proposed logically consistent loss and the proposed data organization. The experiments confirm that the proposed loss formulae and introduction of hybrid-batch leads to more consistency as well as better performance. Though the proposed approach is tested with MAC-net, it can be utilised in any other QA methods whenever the logical consistency between answers exist.

Related papers

• Case-Based Reasoning Approach for Solving Financial Question Answering [5.10832476049103]
  FinQA introduced a numerical reasoning dataset for financial documents. We propose a novel approach to tackle numerical reasoning problems using case based reasoning (CBR) Our model retrieves relevant cases to address a given question, and then generates an answer based on the retrieved cases and contextual information.
  arXiv  Detail & Related papers  (2024-05-18T10:06:55Z)
• RetinaQA: A Robust Knowledge Base Question Answering Model for both Answerable and Unanswerable Questions [23.73807255464977]
  State-of-the-art Knowledge Base Question Answering (KBQA) models assume all questions to be answerable. We propose RetinaQA, a new model that unifies two key ideas in a single KBQA architecture. We show that RetinaQA significantly outperforms adaptations of state-of-the-art KBQA models in handling both answerable and unanswerable questions.
  arXiv  Detail & Related papers  (2024-03-16T08:08:20Z)
• An Empirical Comparison of LM-based Question and Answer Generation Methods [79.31199020420827]
  Question and answer generation (QAG) consists of generating a set of question-answer pairs given a context. In this paper, we establish baselines with three different QAG methodologies that leverage sequence-to-sequence language model (LM) fine-tuning. Experiments show that an end-to-end QAG model, which is computationally light at both training and inference times, is generally robust and outperforms other more convoluted approaches.
  arXiv  Detail & Related papers  (2023-05-26T14:59:53Z)
• Logical Implications for Visual Question Answering Consistency [2.005299372367689]
  We introduce a new consistency loss term that can be used by a wide range of the VQA models. We propose to infer these logical relations using a dedicated language model and use these in our proposed consistency loss function. We conduct extensive experiments on the VQA Introspect and DME datasets and show that our method brings improvements to state-of-the-art VQA models.
  arXiv  Detail & Related papers  (2023-03-16T16:00:18Z)
• Improving Unsupervised Question Answering via Summarization-Informed Question Generation [47.96911338198302]
  Question Generation (QG) is the task of generating a plausible question for a passage, answer> pair. We make use of freely available news summary data, transforming declarative sentences into appropriate questions using dependency parsing, named entity recognition and semantic role labeling. The resulting questions are then combined with the original news articles to train an end-to-end neural QG model.
  arXiv  Detail & Related papers  (2021-09-16T13:08:43Z)
• A New Score for Adaptive Tests in Bayesian and Credal Networks [64.80185026979883]
  A test is adaptive when its sequence and number of questions is dynamically tuned on the basis of the estimated skills of the taker. We present an alternative family of scores, based on the mode of the posterior probabilities, and hence easier to explain.
  arXiv  Detail & Related papers  (2021-05-25T20:35:42Z)
• Contrast and Classify: Training Robust VQA Models [60.80627814762071]
  We propose a novel training paradigm (ConClaT) that optimize both cross-entropy and contrastive losses. We find that optimizing both losses -- either alternately or jointly -- is key to effective training.
  arXiv  Detail & Related papers  (2020-10-13T00:23:59Z)
• Hierarchical Deep Multi-modal Network for Medical Visual Question Answering [25.633660028022195]
  We propose a hierarchical deep multi-modal network that analyzes and classifies end-user questions/queries. We integrate the QS model to the hierarchical deep multi-modal neural network to generate proper answers to the queries related to medical images.
  arXiv  Detail & Related papers  (2020-09-27T07:24:41Z)
• MUTANT: A Training Paradigm for Out-of-Distribution Generalization in Visual Question Answering [58.30291671877342]
  We present MUTANT, a training paradigm that exposes the model to perceptually similar, yet semantically distinct mutations of the input. MUTANT establishes a new state-of-the-art accuracy on VQA-CP with a $10.57%$ improvement.
  arXiv  Detail & Related papers  (2020-09-18T00:22:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.