Inferring symmetry in natural language

• URL: http://arxiv.org/abs/2010.08090v1
• Date: Fri, 16 Oct 2020 01:25:01 GMT
• Title: Inferring symmetry in natural language
• Abstract summary: We present a methodological framework for inferring symmetry of verb predicates in natural language. Our work integrates existing approaches to symmetry in natural language and suggests how symmetry inference can improve systematicity in state-of-the-art language models.
• Score: 6.962103403356227
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present a methodological framework for inferring symmetry of verb predicates in natural language. Empirical work on predicate symmetry has taken two main approaches. The feature-based approach focuses on linguistic features pertaining to symmetry. The context-based approach denies the existence of absolute symmetry but instead argues that such inference is context dependent. We develop methods that formalize these approaches and evaluate them against a novel symmetry inference sentence (SIS) dataset comprised of 400 naturalistic usages of literature-informed verbs spanning the spectrum of symmetry-asymmetry. Our results show that a hybrid transfer learning model that integrates linguistic features with contextualized language models most faithfully predicts the empirical data. Our work integrates existing approaches to symmetry in natural language and suggests how symmetry inference can improve systematicity in state-of-the-art language models.

Related papers

• The Wittgensteinian Representation Hypothesis: Is Language the Attractor of Multimodal Convergence? [37.415683554349194]
  We introduce directional convergence analysis using cycle-kNN, an asymmetric alignment measure.<n>Non-language modalities move toward the neighborhood structure of language significantly more than the reverse.<n>We formalize this as the Wittgensteinian Representation Hypothesis: the semantic structure of language is the attractor of multimodal representation convergence.
  arXiv  Detail & Related papers  (2026-05-10T06:05:04Z)
• Developmental Symmetry-Loss: A Free-Energy Perspective on Brain-Inspired Invariance Learning [0.0]
  We propose Symmetry-Loss, a brain-inspired algorithmic principle.<n>We show how Symmetry-Loss operationalizes a Free-Energy-like objective for representation learning.<n>The result is a general computational mechanism linking developmental learning in the brain with principled representation learning in artificial systems.
  arXiv  Detail & Related papers  (2025-12-04T22:12:15Z)
• Large Language Models as Neurolinguistic Subjects: Discrepancy in Performance and Competence for Form and Meaning [49.60849499134362]
  This study investigates the linguistic understanding of Large Language Models (LLMs) regarding signifier (form) and signified (meaning) We introduce a neurolinguistic approach, utilizing a novel method that combines minimal pair and diagnostic probing to analyze activation patterns across model layers. We found: (1) Psycholinguistic and neurolinguistic methods reveal that language performance and competence are distinct; (2) Direct probability measurement may not accurately assess linguistic competence; and (3) Instruction tuning won't change much competence but improve performance.
  arXiv  Detail & Related papers  (2024-11-12T04:16:44Z)
• SymmetryLens: A new candidate paradigm for unsupervised symmetry learning via locality and equivariance [0.0]
  We develop a new, unsupervised symmetry learning method that starts with raw data. We demonstrate that this coupling between symmetry and locality, together with a special optimization technique developed for entropy estimation, results in a highly stable system. The symmetry actions we consider are group representations, however, we believe the approach has the potential to be generalized to more general, nonlinear actions of non-commutative Lie groups.
  arXiv  Detail & Related papers  (2024-10-07T17:40:51Z)
• Spontaneous Style Text-to-Speech Synthesis with Controllable Spontaneous Behaviors Based on Language Models [55.898594710420326]
  We propose a novel spontaneous speech synthesis system based on language models. Fine-grained prosody modeling is introduced to enhance the model's ability to capture subtle prosody variations in spontaneous speech.
  arXiv  Detail & Related papers  (2024-07-18T13:42:38Z)
• Regularized Conventions: Equilibrium Computation as a Model of Pragmatic Reasoning [72.21876989058858]
  We present a model of pragmatic language understanding, where utterances are produced and understood by searching for regularized equilibria of signaling games. In this model speakers and listeners search for contextually appropriate utterance--meaning mappings that are both close to game-theoretically optimal conventions and close to a shared, ''default'' semantics.
  arXiv  Detail & Related papers  (2023-11-16T09:42:36Z)
• Robustness of the Random Language Model [0.0]
  The model suggests a simple picture of first language learning as a type of annealing in the vast space of potential languages. It implies a single continuous transition to grammatical syntax, at which the symmetry among potential words and categories is spontaneously broken. Results are discussed in light of theory of first-language acquisition in linguistics, and recent successes in machine learning.
  arXiv  Detail & Related papers  (2023-09-26T13:14:35Z)
• Addressing Imperfect Symmetry: a Novel Symmetry-Learning Actor-Critic Extension [0.46040036610482665]
  We introduce Adaptive Symmetry (ASL) $x2013$ a model-minimization actor-critic extension that addresses incomplete symmetry. ASL consists of symmetry fitting component and modular loss function that enforces a common relation across all states while adapting to the learned policy. The results demonstrate that ASL is capable of recovering from large perturbations and generalizing to hidden symmetric states.
  arXiv  Detail & Related papers  (2023-09-06T04:47:46Z)
• Regularizing Towards Soft Equivariance Under Mixed Symmetries [23.603875905608565]
  We present a regularizer-based method for building a model for a dataset with mixed approximate symmetries. We show that our method achieves better accuracy than prior approaches while discovering the approximate symmetry levels correctly.
  arXiv  Detail & Related papers  (2023-06-01T05:33:41Z)
• Evaluating the Robustness of Interpretability Methods through Explanation Invariance and Equivariance [72.50214227616728]
  Interpretability methods are valuable only if their explanations faithfully describe the explained model. We consider neural networks whose predictions are invariant under a specific symmetry group.
  arXiv  Detail & Related papers  (2023-04-13T17:59:03Z)
• On the Intrinsic and Extrinsic Fairness Evaluation Metrics for Contextualized Language Representations [74.70957445600936]
  Multiple metrics have been introduced to measure fairness in various natural language processing tasks. These metrics can be roughly categorized into two categories: 1) emphextrinsic metrics for evaluating fairness in downstream applications and 2) emphintrinsic metrics for estimating fairness in upstream language representation models.
  arXiv  Detail & Related papers  (2022-03-25T22:17:43Z)
• Linking Emergent and Natural Languages via Corpus Transfer [98.98724497178247]
  We propose a novel way to establish a link by corpus transfer between emergent languages and natural languages. Our approach showcases non-trivial transfer benefits for two different tasks -- language modeling and image captioning. We also introduce a novel metric to predict the transferability of an emergent language by translating emergent messages to natural language captions grounded on the same images.
  arXiv  Detail & Related papers  (2022-03-24T21:24:54Z)
• A Latent-Variable Model for Intrinsic Probing [93.62808331764072]
  We propose a novel latent-variable formulation for constructing intrinsic probes. We find empirical evidence that pre-trained representations develop a cross-lingually entangled notion of morphosyntax.
  arXiv  Detail & Related papers  (2022-01-20T15:01:12Z)
• Language Models for Lexical Inference in Context [4.581468205348204]
  Lexical inference in context (LIiC) is the task of recognizing textual entailment between two very similar sentences. We formulate and evaluate the first approaches based on pretrained language models (LMs) for this task. All our approaches outperform the previous state of the art, showing the potential of pretrained LMs for LIiC.
  arXiv  Detail & Related papers  (2021-02-10T09:08:22Z)
• How Far are We from Effective Context Modeling? An Exploratory Study on Semantic Parsing in Context [59.13515950353125]
  We present a grammar-based decoding semantic parsing and adapt typical context modeling methods on top of it. We evaluate 13 context modeling methods on two large cross-domain datasets, and our best model achieves state-of-the-art performances.
  arXiv  Detail & Related papers  (2020-02-03T11:28:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.