Re-evaluating the Need for Multimodal Signals in Unsupervised Grammar Induction

• URL: http://arxiv.org/abs/2212.10564v3
• Date: Fri, 12 Apr 2024 14:53:30 GMT
• Title: Re-evaluating the Need for Multimodal Signals in Unsupervised Grammar Induction
• Authors: Boyi Li, Rodolfo Corona, Karttikeya Mangalam, Catherine Chen, Daniel Flaherty, Serge Belongie, Kilian Q. Weinberger, Jitendra Malik, Trevor Darrell, Dan Klein,
• Abstract summary: Recent work has shown that multimodal training inputs can improve grammar induction. We design a stronger text-only baseline, which we refer to as LC-PCFG. LC-PCFG provides an up to 17% relative improvement in Corpus-F1 compared to state-of-the-art multimodal grammar induction methods.
• Score: 119.07444667700473
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Are multimodal inputs necessary for grammar induction? Recent work has shown that multimodal training inputs can improve grammar induction. However, these improvements are based on comparisons to weak text-only baselines that were trained on relatively little textual data. To determine whether multimodal inputs are needed in regimes with large amounts of textual training data, we design a stronger text-only baseline, which we refer to as LC-PCFG. LC-PCFG is a C-PFCG that incorporates em-beddings from text-only large language models (LLMs). We use a fixed grammar family to directly compare LC-PCFG to various multi-modal grammar induction methods. We compare performance on four benchmark datasets. LC-PCFG provides an up to 17% relative improvement in Corpus-F1 compared to state-of-the-art multimodal grammar induction methods. LC-PCFG is also more computationally efficient, providing an up to 85% reduction in parameter count and 8.8x reduction in training time compared to multimodal approaches. These results suggest that multimodal inputs may not be necessary for grammar induction, and emphasize the importance of strong vision-free baselines for evaluating the benefit of multimodal approaches.

Related papers

• SelfSeg: A Self-supervised Sub-word Segmentation Method for Neural Machine Translation [51.881877192924414]
  Sub-word segmentation is an essential pre-processing step for Neural Machine Translation (NMT) This paper introduces SelfSeg, a self-supervised neural sub-word segmentation method. SelfSeg is much faster to train/decode and requires only monolingual dictionaries instead of parallel corpora.
  arXiv  Detail & Related papers  (2023-07-31T04:38:47Z)
• Prompt-Augmented Linear Probing: Scaling beyond the Limit of Few-shot In-Context Learners [25.262774179224945]
  This paper proposes prompt-augmented linear probing (PALP), a hybrid of linear probing and in-context learning (ICL) PALP significantly enhances the input representations closing the gap between ICL in the data-hungry scenario and fine-tuning in the data-abundant scenario with little training overhead.
  arXiv  Detail & Related papers  (2022-12-21T09:37:05Z)
• MURMUR: Modular Multi-Step Reasoning for Semi-Structured Data-to-Text Generation [102.20036684996248]
  We propose MURMUR, a neuro-symbolic modular approach to text generation from semi-structured data with multi-step reasoning. We conduct experiments on two data-to-text generation tasks like WebNLG and LogicNLG.
  arXiv  Detail & Related papers  (2022-12-16T17:36:23Z)
• A Multi-level Supervised Contrastive Learning Framework for Low-Resource Natural Language Inference [54.678516076366506]
  Natural Language Inference (NLI) is a growingly essential task in natural language understanding. Here we propose a multi-level supervised contrastive learning framework named MultiSCL for low-resource natural language inference.
  arXiv  Detail & Related papers  (2022-05-31T05:54:18Z)
• ALP: Data Augmentation using Lexicalized PCFGs for Few-Shot Text Classification [11.742065170002162]
  We present the data augmentation using Lexicalized Probabilistic context-free grammars (ALP) Experiments on few-shot text classification tasks demonstrate that ALP enhances many state-of-the-art classification methods. We argue empirically that the traditional splitting of training and validation sets is sub-optimal compared to our novel augmentation-based splitting strategies.
  arXiv  Detail & Related papers  (2021-12-16T09:56:35Z)
• Dependency Induction Through the Lens of Visual Perception [81.91502968815746]
  We propose an unsupervised grammar induction model that leverages word concreteness and a structural vision-based to jointly learn constituency-structure and dependency-structure grammars. Our experiments show that the proposed extension outperforms the current state-of-the-art visually grounded models in constituency parsing even with a smaller grammar size.
  arXiv  Detail & Related papers  (2021-09-20T18:40:37Z)
• Multi-view Subword Regularization [111.04350390045705]
  Multi-view Subword Regularization (MVR) is a method that enforces the consistency between predictions of using inputs tokenized by the standard and probabilistic segmentations. Results on the XTREME multilingual benchmark show that MVR brings consistent improvements of up to 2.5 points over using standard segmentation algorithms.
  arXiv  Detail & Related papers  (2021-03-15T16:07:42Z)
• Learning synchronous context-free grammars with multiple specialised non-terminals for hierarchical phrase-based translation [0.0]
  This paper presents a method to learn synchronous context-free grammars with a huge number of initial non-terminals. Experiments show that the resulting smaller set of non-terminals correctly capture the contextual information.
  arXiv  Detail & Related papers  (2020-04-03T08:09:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.