SpecSteer: Synergizing Local Context and Global Reasoning for Efficient Personalized Generation

• URL: http://arxiv.org/abs/2603.16219v1
• Date: Tue, 17 Mar 2026 07:51:29 GMT
• Title: SpecSteer: Synergizing Local Context and Global Reasoning for Efficient Personalized Generation
• Abstract summary: SpecSteer is an asymmetric collaborative inference framework that synergizes private on-device context with cloud-scale reasoning.<n>Experiments demonstrate that SpecSteer successfully closes the reasoning gap and achieves superior personalized generation performance.
• Score: 79.75755802397312
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Realizing personalized intelligence faces a core dilemma: sending user history to centralized large language models raises privacy concerns, while on-device small language models lack the reasoning capacity required for high-quality generation. Our pilot study shows that purely local enhancements remain insufficient to reliably bridge this gap. We therefore propose SpecSteer, an asymmetric collaborative inference framework that synergizes private on-device context with cloud-scale reasoning. SpecSteer casts collaboration as Bayesian knowledge fusion and repurposes speculative decoding as a distributed alignment protocol, yielding a Draft--Verify--Recover pipeline: the on-device model drafts personalized sequences; the cloud validates via a ratio-based mechanism that decouples reasoning verification from private context, filtering logical flaws without accessing raw user context; upon rejection, a steering recovery injects local intent during correction. Experiments demonstrate that SpecSteer successfully closes the reasoning gap and achieves superior personalized generation performance, while delivering a 2.36x speedup over standard baselines.

Related papers

• Phantoms and Disclosures: a Causal Framework for Auditing Synthetic Data [56.86147283213652]
  We present a customizable empirical auditing framework designed to detect and explain data disclosures.<n>Our framework introduces a mechanism to distinguish between "true disclosures"-where the system directly reproduces a user's information-and "phantom disclosures"-where the system incidentally generates a user's data.
  arXiv  Detail & Related papers  (2026-06-15T16:54:02Z)
• Standing on the Shoulders of Giants: Stabilized Knowledge Distillation for Cross--Language Code Clone Detection [3.2116198597240846]
  Cross-language code clone detection (X-CCD) is challenging because semantically equivalent programs written in different languages often share little surface similarity.<n>We propose a knowledge distillation framework that transfers reasoning capabilities from DeepSeek-R1 into compact open-source student models for X-CCD detection.
  arXiv  Detail & Related papers  (2026-05-04T17:37:16Z)
• Sparse Personalized Text Generation with Multi-Trajectory Reasoning [95.19317487799292]
  PAT (Personalization with Aligned Trajectories) is a reasoning framework for cold-start LLM personalization.<n>It first retrieves information along two complementary trajectories: writing-style cues from stylistically similar users and topic-specific context from preference-aligned users.<n>It then employs a reinforcement learning-based, iterative dual-reasoning mechanism that enables the LLM to jointly refine and integrate these signals.
  arXiv  Detail & Related papers  (2026-04-27T20:58:49Z)
• Differentially Private and Communication Efficient Large Language Model Split Inference via Stochastic Quantization and Soft Prompt [33.701746954914135]
  Large Language Models (LLMs) have achieved remarkable performance and received significant research interest.<n>Existing approaches propose to allow the users to obfuscate the token embeddings before transmission and utilize local models for denoising.<n>We propose textbfDEL, a framework for textbfDifferentially private and communication textbfEfficient textbfLLM split inference.
  arXiv  Detail & Related papers  (2026-02-12T03:13:16Z)
• ConfSpec: Efficient Step-Level Speculative Reasoning via Confidence-Gated Verification [0.2578242050187029]
  Step-level speculative reasoning aims to mitigate this cost, yet existing approaches face a long-standing trade-off.<n>We propose ConfSpec, a confidence-gated cascaded verification framework that resolves this trade-off.
  arXiv  Detail & Related papers  (2026-01-28T05:58:05Z)
• CHORD: Customizing Hybrid-precision On-device Model for Sequential Recommendation with Device-cloud Collaboration [28.97362695603172]
  We propose a framework for underlinetextbfCustomizing underlinetextbfHybrid-precision underlinetextbfOn-device model for sequential underlinetextbfRecommendation with underlinetextbfDevice-cloud collaboration (textbfCHORD)<n>CHORD delivers dynamic model adaptation and accelerated inference without backpropagation, eliminating costly retraining cycles.
  arXiv  Detail & Related papers  (2025-10-03T14:20:45Z)
• SelfAug: Mitigating Catastrophic Forgetting in Retrieval-Augmented Generation via Distribution Self-Alignment [49.86376148975563]
  Large language models (LLMs) have revolutionized natural language processing through their capabilities in understanding and executing diverse tasks.<n> supervised fine-tuning, particularly in Retrieval-Augmented Generation (RAG) scenarios, often leads to catastrophic forgetting.<n>We propose SelfAug, a self-distribution alignment method that aligns input sequence logits to preserve the model's semantic distribution.
  arXiv  Detail & Related papers  (2025-09-04T06:50:47Z)
• CoSteer: Collaborative Decoding-Time Personalization via Local Delta Steering [80.54309860395763]
  CoSteer is a novel collaborative framework that enables decoding-time personalization through localized delta steering.<n>We formulate token-level optimization as an online learning problem, where local delta vectors dynamically adjust the remote LLM's logits.<n>This approach preserves privacy by transmitting only the final steered tokens rather than raw data or intermediate vectors.
  arXiv  Detail & Related papers  (2025-07-07T08:32:29Z)
• Constrained Auto-Regressive Decoding Constrains Generative Retrieval [71.71161220261655]
  Generative retrieval seeks to replace traditional search index data structures with a single large-scale neural network.<n>In this paper, we examine the inherent limitations of constrained auto-regressive generation from two essential perspectives: constraints and beam search.
  arXiv  Detail & Related papers  (2025-04-14T06:54:49Z)
• Toward Certified Robustness Against Real-World Distribution Shifts [65.66374339500025]
  We train a generative model to learn perturbations from data and define specifications with respect to the output of the learned model. A unique challenge arising from this setting is that existing verifiers cannot tightly approximate sigmoid activations. We propose a general meta-algorithm for handling sigmoid activations which leverages classical notions of counter-example-guided abstraction refinement.
  arXiv  Detail & Related papers  (2022-06-08T04:09:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.