Consistency of Compositional Generalization across Multiple Levels

• URL: http://arxiv.org/abs/2412.13636v1
• Date: Wed, 18 Dec 2024 09:09:41 GMT
• Title: Consistency of Compositional Generalization across Multiple Levels
• Authors: Chuanhao Li, Zhen Li, Chenchen Jing, Xiaomeng Fan, Wenbo Ye, Yuwei Wu, Yunde Jia,
• Abstract summary: We propose a meta-learning based framework, for achieving consistent compositional generalization across multiple levels. We build a GQA-CCG dataset to quantitatively evaluate the consistency.
• Score: 31.77432446850103
• License:
• Abstract: Compositional generalization is the capability of a model to understand novel compositions composed of seen concepts. There are multiple levels of novel compositions including phrase-phrase level, phrase-word level, and word-word level. Existing methods achieve promising compositional generalization, but the consistency of compositional generalization across multiple levels of novel compositions remains unexplored. The consistency refers to that a model should generalize to a phrase-phrase level novel composition, and phrase-word/word-word level novel compositions that can be derived from it simultaneously. In this paper, we propose a meta-learning based framework, for achieving consistent compositional generalization across multiple levels. The basic idea is to progressively learn compositions from simple to complex for consistency. Specifically, we divide the original training set into multiple validation sets based on compositional complexity, and introduce multiple meta-weight-nets to generate sample weights for samples in different validation sets. To fit the validation sets in order of increasing compositional complexity, we optimize the parameters of each meta-weight-net independently and sequentially in a multilevel optimization manner. We build a GQA-CCG dataset to quantitatively evaluate the consistency. Experimental results on visual question answering and temporal video grounding, demonstrate the effectiveness of the proposed framework. We release GQA-CCG at https://github.com/NeverMoreLCH/CCG.

Related papers

• T2I-CompBench: A Comprehensive Benchmark for Open-world Compositional Text-to-image Generation [62.71574695256264]
  T2I-CompBench is a comprehensive benchmark for open-world compositional text-to-image generation. We propose several evaluation metrics specifically designed to evaluate compositional text-to-image generation. We introduce a new approach, Generative mOdel fine-tuning with Reward-driven Sample selection (GORS) to boost the compositional text-to-image generation abilities.
  arXiv  Detail & Related papers  (2023-07-12T17:59:42Z)
• Variational Cross-Graph Reasoning and Adaptive Structured Semantics Learning for Compositional Temporal Grounding [143.5927158318524]
  Temporal grounding is the task of locating a specific segment from an untrimmed video according to a query sentence. We introduce a new Compositional Temporal Grounding task and construct two new dataset splits. We argue that the inherent structured semantics inside the videos and language is the crucial factor to achieve compositional generalization.
  arXiv  Detail & Related papers  (2023-01-22T08:02:23Z)
• Compositional Temporal Grounding with Structured Variational Cross-Graph Correspondence Learning [92.07643510310766]
  Temporal grounding in videos aims to localize one target video segment that semantically corresponds to a given query sentence. We introduce a new Compositional Temporal Grounding task and construct two new dataset splits. We empirically find that they fail to generalize to queries with novel combinations of seen words. We propose a variational cross-graph reasoning framework that explicitly decomposes video and language into multiple structured hierarchies.
  arXiv  Detail & Related papers  (2022-03-24T12:55:23Z)
• Structurally Diverse Sampling Reduces Spurious Correlations in Semantic Parsing Datasets [51.095144091781734]
  We propose a novel algorithm for sampling a structurally diverse set of instances from a labeled instance pool with structured outputs. We show that our algorithm performs competitively with or better than prior algorithms in not only compositional template splits but also traditional IID splits. In general, we find that diverse train sets lead to better generalization than random training sets of the same size in 9 out of 10 dataset-split pairs.
  arXiv  Detail & Related papers  (2022-03-16T07:41:27Z)
• Compositional Generalization Requires Compositional Parsers [69.77216620997305]
  We compare sequence-to-sequence models and models guided by compositional principles on the recent COGS corpus. We show structural generalization is a key measure of compositional generalization and requires models that are aware of complex structure.
  arXiv  Detail & Related papers  (2022-02-24T07:36:35Z)
• Grounded Graph Decoding Improves Compositional Generalization in Question Answering [68.72605660152101]
  Question answering models struggle to generalize to novel compositions of training patterns, such as longer sequences or more complex test structures. We propose Grounded Graph Decoding, a method to improve compositional generalization of language representations by grounding structured predictions with an attention mechanism. Our model significantly outperforms state-of-the-art baselines on the Compositional Freebase Questions (CFQ) dataset, a challenging benchmark for compositional generalization in question answering.
  arXiv  Detail & Related papers  (2021-11-05T17:50:14Z)
• Improving Compositional Generalization in Classification Tasks via Structure Annotations [33.90268697120572]
  Humans have a great ability to generalize compositionally, but state-of-the-art neural models struggle to do so. First, we study ways to convert a natural language sequence-to-sequence dataset to a classification dataset that also requires compositional generalization. Second, we show that providing structural hints (specifically, providing parse trees and entity links as attention masks for a Transformer model) helps compositional generalization.
  arXiv  Detail & Related papers  (2021-06-19T06:07:27Z)
• COGS: A Compositional Generalization Challenge Based on Semantic Interpretation [27.89019245763459]
  We introduce COGS, a semantic parsing dataset based on a fragment of English. The evaluation portion of COGS contains multiple systematic gaps that can only be addressed by compositional generalization. In experiments with Transformers and LSTMs, we found that in-distribution accuracy on the COGS test set was near-perfect (96--99%), but generalization accuracy was substantially lower (16--35%) and showed high sensitivity to random seed.
  arXiv  Detail & Related papers  (2020-10-12T05:45:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.