LLM-Augmented Semantic Steering of Text Embedding Projection Spaces

• URL: http://arxiv.org/abs/2605.01957v1
• Date: Sun, 03 May 2026 16:41:11 GMT
• Title: LLM-Augmented Semantic Steering of Text Embedding Projection Spaces
• Abstract summary: We introduce LLM-augmented semantic steering, which enables analysts to express semantic intent by grouping a small set of example documents within the projection.<n>A large language model externalizes this intent as natural-language representations and selectively extends it to related documents.<n>The resulting semantic information is then incorporated into document representations via text augmentation or embedding-level blending.
• Score: 7.516736502809871
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Low-dimensional projections of text embeddings support visual analysis of document collections, but their spatial organization may not reflect the relationships an analyst intends to examine. Existing semantic interaction approaches encode semantic intent indirectly through geometric constraints or model updates, limiting interpretability and flexibility. We introduce LLM-augmented semantic steering, which enables analysts to express semantic intent by grouping a small set of example documents within the projection. A large language model externalizes this intent as natural-language representations and selectively extends it to related documents; the resulting semantic information is then incorporated into document representations via text augmentation or embedding-level blending, without retraining the underlying models. A case study illustrates how the same corpus can be reorganized from different semantic perspectives, while simulation-based evaluation shows that semantic steering improves global and local alignment with target semantic structures using only minimal interaction. Embedding-level blending further enables continuous and controllable steering of projection layouts. These results position projection spaces as intent-dependent semantic workspaces that can be reshaped through explicit, interpretable, language-mediated interaction.

Related papers

• Scalable Semantic Steering of Embedding Projections [7.516736502809871]
  Low-dimensional projections support interactive visual analysis of high-dimensional data embeddings, but their structure often does not align with analyst-defined semantic relationships.<n>Recent LLM-augmented semantic steering methods address this gap by externalizing analyst intent from user-defined groups of seed examples.<n>We propose a scalable semantic steering method that shifts semantic computation from individual items to user-defined groups.
  arXiv  Detail & Related papers  (2026-07-04T18:24:00Z)
• Text-as-Signal: Quantitative Semantic Scoring with Embeddings, Logprobs, and Noise Reduction [0.0]
  This paper presents a practical pipeline for turning text corpora into quantitative semantic signals.<n>We show how Qwen embeddings, UMAP, semantic indicators derived directly from the model output space, and a three-stage anomaly-detection procedure combine into an operational text-as-signal workflow.
  arXiv  Detail & Related papers  (2026-03-17T19:56:29Z)
• AffordGrasp: Cross-Modal Diffusion for Affordance-Aware Grasp Synthesis [50.793806818677716]
  AffordGrasp produces physically stable and semantically faithful human grasps with high precision.<n>AffordGrasp integrates an affordance-aware latent representation of hand poses with a dual-conditioning diffusion process.<n>We evaluate AffordGrasp across four instruction-augmented benchmarks derived from HO-3D, OakInk, GRAB, and AffordPose.
  arXiv  Detail & Related papers  (2026-03-09T06:56:35Z)
• Formal Semantic Control over Language Models [2.7708787391533463]
  This thesis advances semantic representation learning to render language representations more semantically and geometrically interpretable.<n>We pursue this goal within a VAE framework, exploring two complementary research directions.<n>The overarching objective is to move toward language models whose internal semantic representations can be systematically interpreted.
  arXiv  Detail & Related papers  (2026-01-31T10:12:53Z)
• Mind-the-Glitch: Visual Correspondence for Detecting Inconsistencies in Subject-Driven Generation [120.23172120151821]
  We propose a novel approach for disentangling visual and semantic features from the backbones of pre-trained diffusion models.<n>We introduce an automated pipeline that constructs image pairs with annotated semantic and visual correspondences.<n>We propose a new metric, Visual Semantic Matching, that quantifies visual inconsistencies in subject-driven image generation.
  arXiv  Detail & Related papers  (2025-09-26T07:11:55Z)
• Ambiguity in LLMs is a concept missing problem [12.039364223570479]
  Ambiguity in natural language is a significant obstacle for achieving accurate text to structured data mapping.<n>We adopt a different approach that characterizes representation differences of ambiguous text in the latent space.<n>We introduce a new distance measure based on a path kernel over concepts to distinguish ambiguous from unambiguous questions.
  arXiv  Detail & Related papers  (2025-05-16T20:39:30Z)
• Hypergraph based Understanding for Document Semantic Entity Recognition [65.84258776834524]
  We build a novel hypergraph attention document semantic entity recognition framework, HGA, which uses hypergraph attention to focus on entity boundaries and entity categories at the same time. Our results on FUNSD, CORD, XFUNDIE show that our method can effectively improve the performance of semantic entity recognition tasks.
  arXiv  Detail & Related papers  (2024-07-09T14:35:49Z)
• Open-Vocabulary Camouflaged Object Segmentation [66.94945066779988]
  We introduce a new task, open-vocabulary camouflaged object segmentation (OVCOS) We construct a large-scale complex scene dataset (textbfOVCamo) containing 11,483 hand-selected images with fine annotations and corresponding object classes. By integrating the guidance of class semantic knowledge and the supplement of visual structure cues from the edge and depth information, the proposed method can efficiently capture camouflaged objects.
  arXiv  Detail & Related papers  (2023-11-19T06:00:39Z)
• LAW-Diffusion: Complex Scene Generation by Diffusion with Layouts [107.11267074981905]
  We propose a semantically controllable layout-AWare diffusion model, termed LAW-Diffusion. We show that LAW-Diffusion yields the state-of-the-art generative performance, especially with coherent object relations.
  arXiv  Detail & Related papers  (2023-08-13T08:06:18Z)
• SAP-sLDA: An Interpretable Interface for Exploring Unstructured Text [28.36260646471421]
  We propose a semi-supervised human-in-the-loop LDA-based method for learning topics that preserve semantically meaningful relationships between documents in low-dimensional projections. On synthetic corpora, our method yields more interpretable projections than baseline methods with only a fraction of labels provided.
  arXiv  Detail & Related papers  (2023-07-28T05:43:39Z)
• Syntax and Semantics Meet in the "Middle": Probing the Syntax-Semantics Interface of LMs Through Agentivity [68.8204255655161]
  We present the semantic notion of agentivity as a case study for probing such interactions. This suggests LMs may potentially serve as more useful tools for linguistic annotation, theory testing, and discovery.
  arXiv  Detail & Related papers  (2023-05-29T16:24:01Z)
• Variational Cross-Graph Reasoning and Adaptive Structured Semantics Learning for Compositional Temporal Grounding [143.5927158318524]
  Temporal grounding is the task of locating a specific segment from an untrimmed video according to a query sentence. We introduce a new Compositional Temporal Grounding task and construct two new dataset splits. We argue that the inherent structured semantics inside the videos and language is the crucial factor to achieve compositional generalization.
  arXiv  Detail & Related papers  (2023-01-22T08:02:23Z)
• Unsupervised Distillation of Syntactic Information from Contextualized Word Representations [62.230491683411536]
  We tackle the task of unsupervised disentanglement between semantics and structure in neural language representations. To this end, we automatically generate groups of sentences which are structurally similar but semantically different. We demonstrate that our transformation clusters vectors in space by structural properties, rather than by lexical semantics.
  arXiv  Detail & Related papers  (2020-10-11T15:13:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.