Scalable Semantic Steering of Embedding Projections

• URL: http://arxiv.org/abs/2607.03978v1
• Date: Sat, 04 Jul 2026 18:24:00 GMT
• Title: Scalable Semantic Steering of Embedding Projections
• Abstract summary: Low-dimensional projections support interactive visual analysis of high-dimensional data embeddings, but their structure often does not align with analyst-defined semantic relationships.<n>Recent LLM-augmented semantic steering methods address this gap by externalizing analyst intent from user-defined groups of seed examples.<n>We propose a scalable semantic steering method that shifts semantic computation from individual items to user-defined groups.
• Score: 7.516736502809871
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Low-dimensional projections support interactive visual analysis of high-dimensional data embeddings, but their structure often does not align with analyst-defined semantic relationships. Recent LLM-augmented semantic steering methods address this gap by externalizing analyst intent from user-defined groups of seed examples, but they propagate intent through per-item LLM reasoning, causing LLM calls and cost to grow linearly with collection size. We propose a scalable semantic steering method that shifts semantic computation from individual items to user-defined groups. A single LLM call generates structured profiles for all groups, which are embedded and combined with seed centroids to form hybrid semantic prototypes. The method then propagates intent without retraining, using embedding-space soft assignment, abstention, and alignment-scaled updates before reprojection. On a 5K-document LitCovid corpus, our method achieves global alignment comparable to per-item LLM steering while reducing LLM calls by over three orders of magnitude. An image case study shows that the same prototype-based mechanism extends to multimodal embeddings. These results suggest that group-level representations can make semantic steering more practical for larger embedding collections.

Related papers

• GRACE: LLM-Grounded Semantic Metric Spaces for Scalable Mixed-Data Clustering [19.352298911016668]
  GRACE is a framework for scalable mixed-data clustering.<n>It maps heterogeneous values into knowledge-informed descriptions.<n>It achieves superior clustering accuracy and conceptual interpretability over 11 competing methods.
  arXiv  Detail & Related papers  (2026-08-08T03:14:50Z)
• DySem: Uncovering Dynamic Semantic Components of Large Language Models for Calculating Semantic Textual Similarity [12.354777054071379]
  Calculating semantic textual similarity is a foundational task in natural language processing.<n>We propose DySem, a novel training-free framework that investigates more semantic-related internal components of large language models.
  arXiv  Detail & Related papers  (2026-05-28T10:47:32Z)
• LLM-Augmented Semantic Steering of Text Embedding Projection Spaces [7.516736502809871]
  We introduce LLM-augmented semantic steering, which enables analysts to express semantic intent by grouping a small set of example documents within the projection.<n>A large language model externalizes this intent as natural-language representations and selectively extends it to related documents.<n>The resulting semantic information is then incorporated into document representations via text augmentation or embedding-level blending.
  arXiv  Detail & Related papers  (2026-05-03T16:41:11Z)
• HCRE: LLM-based Hierarchical Classification for Cross-Document Relation Extraction with a Prediction-then-Verification Strategy [54.91468501159335]
  Cross-document relation extraction (RE) aims to identify relations between the head tail entities located in different documents.<n>We propose an inference strategy that improves reliability through multi-view verification at each level.
  arXiv  Detail & Related papers  (2026-04-09T07:55:27Z)
• ClusterFusion: Hybrid Clustering with Embedding Guidance and LLM Adaptation [52.794544682493814]
  Large language models (LLMs) provide strong contextual reasoning, yet prior work mainly uses them as auxiliary modules to refine embeddings or adjust cluster boundaries.<n>We propose ClusterFusion, a hybrid framework that treats the LLM as the clustering core, guided by lightweight embedding methods.<n> Experiments on three public benchmarks and two new domain-specific datasets demonstrate that ClusterFusion achieves state-of-the-art performance on standard tasks.
  arXiv  Detail & Related papers  (2025-12-04T00:49:43Z)
• LLM-MemCluster: Empowering Large Language Models with Dynamic Memory for Text Clustering [52.41664454251679]
  Large Language Models (LLMs) are reshaping unsupervised learning by offering an unprecedented ability to perform text clustering.<n>Existing methods often rely on complex pipelines with external modules, sacrificing a truly end-to-end approach.<n>We introduce LLM-MemCluster, a novel framework that reconceptualizes clustering as a fully LLM-native task.
  arXiv  Detail & Related papers  (2025-11-19T13:22:08Z)
• LLM-guided Hierarchical Retrieval [54.73080745446999]
  LATTICE is a hierarchical retrieval framework that enables an LLM to reason over and navigate large corpora with logarithmic search complexity.<n>A central challenge in such LLM-guided search is that the model's relevance judgments are noisy, context-dependent, and unaware of the hierarchy.<n>Our framework achieves state-of-the-art zero-shot performance on the reasoning-intensive BRIGHT benchmark.
  arXiv  Detail & Related papers  (2025-10-15T07:05:17Z)
• TWIST: Training-free and Label-free Short Text Clustering through Iterative Vector Updating with LLMs [18.746184073913813]
  We propose a training-free and label-free method for short text clustering.<n>Our method achieves comparable or superior results to state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-10-08T08:05:39Z)
• RecBase: Generative Foundation Model Pretraining for Zero-Shot Recommendation [78.01030342481246]
  RecBase is a domain-agnostic foundational model pretrained with a recommendation-oriented objective.<n>We introduce a unified item tokenizer that encodes items into hierarchical concept identifiers.<n>Our model matches or surpasses the performance of LLM baselines up to 7B parameters in zero-shot and cross-domain recommendation tasks.
  arXiv  Detail & Related papers  (2025-09-03T08:33:43Z)
• Latent Factor Models Meets Instructions: Goal-conditioned Latent Factor Discovery without Task Supervision [50.45597801390757]
  Instruct-LF is a goal-oriented latent factor discovery system.<n>It integrates instruction-following ability with statistical models to handle noisy datasets.
  arXiv  Detail & Related papers  (2025-02-21T02:03:08Z)
• Align$^2$LLaVA: Cascaded Human and Large Language Model Preference Alignment for Multi-modal Instruction Curation [56.75665429851673]
  This paper introduces a novel instruction curation algorithm, derived from two unique perspectives, human and LLM preference alignment.<n>Experiments demonstrate that we can maintain or even improve model performance by compressing synthetic multimodal instructions by up to 90%.
  arXiv  Detail & Related papers  (2024-09-27T08:20:59Z)
• LLM with Relation Classifier for Document-Level Relation Extraction [25.587850398830252]
  Large language models (LLMs) have created a new paradigm for natural language processing.<n>This paper investigates the causes of this performance gap, identifying the dispersion of attention by LLMs due to entity pairs without relations as a key factor.
  arXiv  Detail & Related papers  (2024-08-25T16:43:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.