Formal Semantic Control over Language Models

• URL: http://arxiv.org/abs/2602.00638v1
• Date: Sat, 31 Jan 2026 10:12:53 GMT
• Title: Formal Semantic Control over Language Models
• Authors: Yingji Zhang,
• Abstract summary: This thesis advances semantic representation learning to render language representations more semantically and geometrically interpretable.<n>We pursue this goal within a VAE framework, exploring two complementary research directions.<n>The overarching objective is to move toward language models whose internal semantic representations can be systematically interpreted.
• Score: 2.7708787391533463
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This thesis advances semantic representation learning to render language representations or models more semantically and geometrically interpretable, and to enable localised, quasi-symbolic, compositional control through deliberate shaping of their latent space geometry. We pursue this goal within a VAE framework, exploring two complementary research directions: (i) Sentence-level learning and control: disentangling and manipulating specific semantic features in the latent space to guide sentence generation, with explanatory text serving as the testbed; and (ii) Reasoning-level learning and control: isolating and steering inference behaviours in the latent space to control NLI. In this direction, we focus on Explanatory NLI tasks, in which two premises (explanations) are provided to infer a conclusion. The overarching objective is to move toward language models whose internal semantic representations can be systematically interpreted, precisely structured, and reliably directed. We introduce a set of novel theoretical frameworks and practical methodologies, together with corresponding experiments, to demonstrate that our approaches enhance both the interpretability and controllability of latent spaces for natural language across the thesis.

Related papers

• Emergent Structured Representations Support Flexible In-Context Inference in Large Language Models [77.98801218316505]
  Large language models (LLMs) exhibit emergent behaviors suggestive of human-like reasoning.<n>We investigate the internal processing of LLMs during in-context concept inference.
  arXiv  Detail & Related papers  (2026-02-08T03:14:39Z)
• Language as Mathematical Structure: Examining Semantic Field Theory Against Language Games [0.0]
  Large language models (LLMs) offer a new empirical setting in which long-standing theories of linguistic meaning can be examined.<n>We formalize the notions of lexical fields (Lexfelder) and linguistic fields (Lingofelder) as interacting structures in a continuous semantic space.<n>We argue that the success of LLMs in capturing semantic regularities supports the view that language exhibits an underlying mathematical structure.
  arXiv  Detail & Related papers  (2026-01-01T19:15:17Z)
• On the Entity-Level Alignment in Crosslingual Consistency [62.33186691736433]
  SubSub and SubInj integrate English translations of subjects into prompts across languages, leading to substantial gains in factual recall accuracy and consistency.<n>These interventions reinforce the entity representation alignment in the conceptual space through model's internal pivot-language processing.
  arXiv  Detail & Related papers  (2025-10-11T16:26:50Z)
• $I^2G$: Generating Instructional Illustrations via Text-Conditioned Diffusion [31.2362624526101]
  We propose a language-driven framework that decomposing procedural text into coherent visual instructions.<n>Our approach models the linguistic structure of instructional content by coherence it into goal statements and sequential steps, then conditioning visual generation on these linguistic elements.<n>This work contributes to the growing body of research on grounding procedural language in visual content, with applications spanning education, task guidance, and multimodal language understanding.
  arXiv  Detail & Related papers  (2025-05-22T09:10:09Z)
• Constructive Approach to Bidirectional Influence between Qualia Structure and Language Emergence [5.906966694759679]
  This perspective paper explores the bidirectional influence between language emergence and the structure of subjective experiences.<n>We hypothesize that the emergence of languages with distributional semantics is linked to the coordination of internal representations shaped by experience.
  arXiv  Detail & Related papers  (2024-09-14T11:03:12Z)
• The Empty Signifier Problem: Towards Clearer Paradigms for Operationalising "Alignment" in Large Language Models [18.16062736448993]
  We address the concept of "alignment" in large language models (LLMs) through the lens of post-structuralist socio-political theory. We propose a framework that demarcates: 1) which dimensions of model behaviour are considered important, then 2) how meanings and definitions are ascribed to these dimensions. We aim to foster a culture of transparency and critical evaluation, aiding the community in navigating the complexities of aligning LLMs with human populations.
  arXiv  Detail & Related papers  (2023-10-03T22:02:17Z)
• Feature Interactions Reveal Linguistic Structure in Language Models [2.0178765779788495]
  We study feature interactions in the context of feature attribution methods for post-hoc interpretability. We work out a grey box methodology, in which we train models to perfection on a formal language classification task. We show that under specific configurations, some methods are indeed able to uncover the grammatical rules acquired by a model.
  arXiv  Detail & Related papers  (2023-06-21T11:24:41Z)
• BabySLM: language-acquisition-friendly benchmark of self-supervised spoken language models [56.93604813379634]
  Self-supervised techniques for learning speech representations have been shown to develop linguistic competence from exposure to speech without the need for human labels. We propose a language-acquisition-friendly benchmark to probe spoken language models at the lexical and syntactic levels. We highlight two exciting challenges that need to be addressed for further progress: bridging the gap between text and speech and between clean speech and in-the-wild speech.
  arXiv  Detail & Related papers  (2023-06-02T12:54:38Z)
• Multi-Relational Hyperbolic Word Embeddings from Natural Language Definitions [5.763375492057694]
  This paper presents a multi-relational model that explicitly leverages such a structure to derive word embeddings from definitions. An empirical analysis demonstrates that the framework can help imposing the desired structural constraints. Experiments reveal the superiority of the Hyperbolic word embeddings over the Euclidean counterparts.
  arXiv  Detail & Related papers  (2023-05-12T08:16:06Z)
• An Inclusive Notion of Text [69.36678873492373]
  We argue that clarity on the notion of text is crucial for reproducible and generalizable NLP. We introduce a two-tier taxonomy of linguistic and non-linguistic elements that are available in textual sources and can be used in NLP modeling.
  arXiv  Detail & Related papers  (2022-11-10T14:26:43Z)
• A Knowledge-Enhanced Adversarial Model for Cross-lingual Structured Sentiment Analysis [31.05169054736711]
  Cross-lingual structured sentiment analysis task aims to transfer the knowledge from source language to target one. We propose a Knowledge-Enhanced Adversarial Model (textttKEAM) with both implicit distributed and explicit structural knowledge. We conduct experiments on five datasets and compare textttKEAM with both the supervised and unsupervised methods.
  arXiv  Detail & Related papers  (2022-05-31T03:07:51Z)
• Contrastive Instruction-Trajectory Learning for Vision-Language Navigation [66.16980504844233]
  A vision-language navigation (VLN) task requires an agent to reach a target with the guidance of natural language instruction. Previous works fail to discriminate the similarities and discrepancies across instruction-trajectory pairs and ignore the temporal continuity of sub-instructions. We propose a Contrastive Instruction-Trajectory Learning framework that explores invariance across similar data samples and variance across different ones to learn distinctive representations for robust navigation.
  arXiv  Detail & Related papers  (2021-12-08T06:32:52Z)
• SLM: Learning a Discourse Language Representation with Sentence Unshuffling [53.42814722621715]
  We introduce Sentence-level Language Modeling, a new pre-training objective for learning a discourse language representation. We show that this feature of our model improves the performance of the original BERT by large margins.
  arXiv  Detail & Related papers  (2020-10-30T13:33:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.