Attribute Diversity Determines the Systematicity Gap in VQA

• URL: http://arxiv.org/abs/2311.08695v2
• Date: Mon, 24 Jun 2024 15:51:13 GMT
• Title: Attribute Diversity Determines the Systematicity Gap in VQA
• Authors: Ian Berlot-Attwell, Kumar Krishna Agrawal, A. Michael Carrell, Yash Sharma, Naomi Saphra,
• Abstract summary: We study the systematicity gap in visual question answering. We find that increased quantity of training data does not reduce the systematicity gap. In all, our experiments suggest that the more distinct attribute type combinations are seen during training, the more systematic we can expect the resulting model to be.
• Score: 7.433031036510163
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The degree to which neural networks can generalize to new combinations of familiar concepts, and the conditions under which they are able to do so, has long been an open question. In this work, we study the systematicity gap in visual question answering: the performance difference between reasoning on previously seen and unseen combinations of object attributes. To test, we introduce a novel diagnostic dataset, CLEVR-HOPE. We find that while increased quantity of training data does not reduce the systematicity gap, increased training data diversity of the attributes in the unseen combination does. In all, our experiments suggest that the more distinct attribute type combinations are seen during training, the more systematic we can expect the resulting model to be.

Related papers

• Bayesian Joint Additive Factor Models for Multiview Learning [7.254731344123118]
  A motivating application arises in the context of precision medicine where multi-omics data are collected to correlate with clinical outcomes. We propose a joint additive factor regression model (JAFAR) with a structured additive design, accounting for shared and view-specific components. Prediction of time-to-labor onset from immunome, metabolome, and proteome data illustrates performance gains against state-of-the-art competitors.
  arXiv  Detail & Related papers  (2024-06-02T15:35:45Z)
• D3: Data Diversity Design for Systematic Generalization in Visual Question Answering [6.033292707572802]
  We show that the diversity of simple tasks plays a key role in achieving systematic generalization. This implies that it may not be essential to gather a large and varied number of complex tasks, which could be costly to obtain.
  arXiv  Detail & Related papers  (2023-09-15T22:45:02Z)
• Rethinking Mitosis Detection: Towards Diverse Data and Feature Representation [30.882319057927052]
  We propose a novel generalizable framework (MitDet) for mitosis detection. Our proposed model outperforms all the SOTA approaches in several popular mitosis detection datasets.
  arXiv  Detail & Related papers  (2023-07-12T03:33:11Z)
• On the Joint Interaction of Models, Data, and Features [82.60073661644435]
  We introduce a new tool, the interaction tensor, for empirically analyzing the interaction between data and model through features. Based on these observations, we propose a conceptual framework for feature learning. Under this framework, the expected accuracy for a single hypothesis and agreement for a pair of hypotheses can both be derived in closed-form.
  arXiv  Detail & Related papers  (2023-06-07T21:35:26Z)
• Regularization Through Simultaneous Learning: A Case Study on Plant Classification [0.0]
  This paper introduces Simultaneous Learning, a regularization approach drawing on principles of Transfer Learning and Multi-task Learning. We leverage auxiliary datasets with the target dataset, the UFOP-HVD, to facilitate simultaneous classification guided by a customized loss function. Remarkably, our approach demonstrates superior performance over models without regularization.
  arXiv  Detail & Related papers  (2023-05-22T19:44:57Z)
• Heterogeneous Target Speech Separation [52.05046029743995]
  We introduce a new paradigm for single-channel target source separation where the sources of interest can be distinguished using non-mutually exclusive concepts. Our proposed heterogeneous separation framework can seamlessly leverage datasets with large distribution shifts.
  arXiv  Detail & Related papers  (2022-04-07T17:14:20Z)
• Leveraging the structure of dynamical systems for data-driven modeling [111.45324708884813]
  We consider the impact of the training set and its structure on the quality of the long-term prediction. We show how an informed design of the training set, based on invariants of the system and the structure of the underlying attractor, significantly improves the resulting models.
  arXiv  Detail & Related papers  (2021-12-15T20:09:20Z)
• The Role of Isomorphism Classes in Multi-Relational Datasets [6.419762264544509]
  We show that isomorphism leakage overestimates performance in multi-relational inference. We propose isomorphism-aware synthetic benchmarks for model evaluation. We also demonstrate that isomorphism classes can be utilised through a simple prioritisation scheme.
  arXiv  Detail & Related papers  (2020-09-30T12:15:24Z)
• Select-ProtoNet: Learning to Select for Few-Shot Disease Subtype Prediction [55.94378672172967]
  We focus on few-shot disease subtype prediction problem, identifying subgroups of similar patients. We introduce meta learning techniques to develop a new model, which can extract the common experience or knowledge from interrelated clinical tasks. Our new model is built upon a carefully designed meta-learner, called Prototypical Network, that is a simple yet effective meta learning machine for few-shot image classification.
  arXiv  Detail & Related papers  (2020-09-02T02:50:30Z)
• Learning What Makes a Difference from Counterfactual Examples and Gradient Supervision [57.14468881854616]
  We propose an auxiliary training objective that improves the generalization capabilities of neural networks. We use pairs of minimally-different examples with different labels, a.k.a counterfactual or contrasting examples, which provide a signal indicative of the underlying causal structure of the task. Models trained with this technique demonstrate improved performance on out-of-distribution test sets.
  arXiv  Detail & Related papers  (2020-04-20T02:47:49Z)
• Adversarial Feature Hallucination Networks for Few-Shot Learning [84.31660118264514]
  Adversarial Feature Hallucination Networks (AFHN) is based on conditional Wasserstein Generative Adversarial networks (cWGAN) Two novel regularizers are incorporated into AFHN to encourage discriminability and diversity of the synthesized features.
  arXiv  Detail & Related papers  (2020-03-30T02:43:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.