Language Models Can Learn Exceptions to Syntactic Rules

• URL: http://arxiv.org/abs/2306.05969v1
• Date: Fri, 9 Jun 2023 15:35:11 GMT
• Title: Language Models Can Learn Exceptions to Syntactic Rules
• Authors: Cara Su-Yi Leong and Tal Linzen
• Abstract summary: We show that artificial neural networks can generalize productively to novel contexts. We also show that the relative acceptability of a verb in the active vs. passive voice is positively correlated with the relative frequency of its occurrence in those voices.
• Score: 22.810889064523167
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Artificial neural networks can generalize productively to novel contexts. Can they also learn exceptions to those productive rules? We explore this question using the case of restrictions on English passivization (e.g., the fact that "The vacation lasted five days" is grammatical, but "*Five days was lasted by the vacation" is not). We collect human acceptability judgments for passive sentences with a range of verbs, and show that the probability distribution defined by GPT-2, a language model, matches the human judgments with high correlation. We also show that the relative acceptability of a verb in the active vs. passive voice is positively correlated with the relative frequency of its occurrence in those voices. These results provide preliminary support for the entrenchment hypothesis, according to which learners track and uses the distributional properties of their input to learn negative exceptions to rules. At the same time, this hypothesis fails to explain the magnitude of unpassivizability demonstrated by certain individual verbs, suggesting that other cues to exceptionality are available in the linguistic input.

Related papers

• Testing learning hypotheses using neural networks by manipulating learning data [20.525923251193472]
  We show that a neural network language model can learn restrictions to the passive that are similar to those displayed by humans. We find that while the frequency with which a verb appears in the passive significantly affects its passivizability, the semantics of the verb does not.
  arXiv  Detail & Related papers  (2024-07-05T15:41:30Z)
• Assessing the influence of attractor-verb distance on grammatical agreement in humans and language models [0.2934352211707039]
  Subject-verb agreement in the presence of an attractor noun located between the main noun and the verb elicits complex behavior. We modulate the distance between the attractor and the verb while keeping the length of the sentence equal. We report a linear effect of attractor distance on reaction times.
  arXiv  Detail & Related papers  (2023-11-28T17:25:34Z)
• UNcommonsense Reasoning: Abductive Reasoning about Uncommon Situations [62.71847873326847]
  We investigate the ability to model unusual, unexpected, and unlikely situations. Given a piece of context with an unexpected outcome, this task requires reasoning abductively to generate an explanation. We release a new English language corpus called UNcommonsense.
  arXiv  Detail & Related papers  (2023-11-14T19:00:55Z)
• Transparency Helps Reveal When Language Models Learn Meaning [71.96920839263457]
  Our systematic experiments with synthetic data reveal that, with languages where all expressions have context-independent denotations, both autoregressive and masked language models learn to emulate semantic relations between expressions. Turning to natural language, our experiments with a specific phenomenon -- referential opacity -- add to the growing body of evidence that current language models do not well-represent natural language semantics.
  arXiv  Detail & Related papers  (2022-10-14T02:35:19Z)
• Montague semantics and modifier consistency measurement in neural language models [1.6799377888527685]
  This work proposes a methodology for measuring compositional behavior in contemporary language models. Specifically, we focus on adjectival modifier phenomena in adjective-noun phrases. Our experimental results indicate that current neural language models behave according to the expected linguistic theories to a limited extent only.
  arXiv  Detail & Related papers  (2022-10-10T18:43:16Z)
• Naturalistic Causal Probing for Morpho-Syntax [76.83735391276547]
  We suggest a naturalistic strategy for input-level intervention on real world data in Spanish. Using our approach, we isolate morpho-syntactic features from counfounders in sentences. We apply this methodology to analyze causal effects of gender and number on contextualized representations extracted from pre-trained models.
  arXiv  Detail & Related papers  (2022-05-14T11:47:58Z)
• Provable Limitations of Acquiring Meaning from Ungrounded Form: What will Future Language Models Understand? [87.20342701232869]
  We investigate the abilities of ungrounded systems to acquire meaning. We study whether assertions enable a system to emulate representations preserving semantic relations like equivalence. We find that assertions enable semantic emulation if all expressions in the language are referentially transparent. However, if the language uses non-transparent patterns like variable binding, we show that emulation can become an uncomputable problem.
  arXiv  Detail & Related papers  (2021-04-22T01:00:17Z)
• Refining Targeted Syntactic Evaluation of Language Models [6.991281327290524]
  Targeted syntactic evaluation of subject-verb number agreement in English (TSE) Method evaluates whether language models rate each grammatical sentence as more likely than its ungrammatical counterpart. We find that TSE overestimates systematicity of language models, but that models score up to 40% better on verbs that they predict are likely in context.
  arXiv  Detail & Related papers  (2021-04-19T20:55:13Z)
• ConjNLI: Natural Language Inference Over Conjunctive Sentences [89.50542552451368]
  Reasoning about conjuncts in conjunctive sentences is important for a deeper understanding of conjunctions. Existing NLI stress tests do not consider non-boolean usages of conjunctions. We introduce ConjNLI, a challenge stress-test for natural language inference over conjunctive sentences.
  arXiv  Detail & Related papers  (2020-10-20T16:29:13Z)
• Investigating Cross-Linguistic Adjective Ordering Tendencies with a Latent-Variable Model [66.84264870118723]
  We present the first purely corpus-driven model of multi-lingual adjective ordering in the form of a latent-variable model. We provide strong converging evidence for the existence of universal, cross-linguistic, hierarchical adjective ordering tendencies.
  arXiv  Detail & Related papers  (2020-10-09T18:27:55Z)
• Recurrent Neural Network Language Models Always Learn English-Like Relative Clause Attachment [17.995905582226463]
  We compare model performance in English and Spanish to show that non-linguistic biases in RNN LMs advantageously overlap with syntactic structure in English but not Spanish. English models may appear to acquire human-like syntactic preferences, while models trained on Spanish fail to acquire comparable human-like preferences.
  arXiv  Detail & Related papers  (2020-05-01T01:21:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.