SoftSkill: Behavioral Compression for Contextual Adaptation

• URL: http://arxiv.org/abs/2606.20333v1
• Date: Thu, 18 Jun 2026 15:04:47 GMT
• Title: SoftSkill: Behavioral Compression for Contextual Adaptation
• Abstract summary: This paper asks whether a natural-language skill can instead initialize a compact continuous context object.<n>We propose SoftSkill, a frozen-backbone method that tunes such soft skills with next-token prediction.<n>Relative to Skill, SoftSkill improves accuracy by 5.2 points on SearchQA and 12.5 points on LiveMath.
• Score: 45.10778084241114
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Agent skills are commonly deployed as natural-language Markdown files that encode answer policies, evidence-use habits, and task procedures. These files are readable and portable, but they are consumed indirectly: for each task instance, a frozen language model must translate a long textual artifact into generation-time behavior. This paper asks whether a natural-language skill can instead initialize a compact continuous context object, refined by a trainable soft delta while the base model remains frozen. We propose SoftSkill, a frozen-backbone method that tunes such soft skills with next-token prediction and deploys them as latent behavioral priors at inference time. In our main single-round setting, a length-32 SoftSkill prefix on Qwen3.5-4B improves over no-skill prompting by 8.3 points on SearchQA, 42.1 points on LiveMath, and 1.3 points on DocVQA. Relative to SkillOpt, SoftSkill improves accuracy by 5.2 points on SearchQA and 12.5 points on LiveMath, while replacing hundreds to thousands of Markdown skill tokens with a few virtual tokens. We further study agentic execution as a harder boundary case, where sparse trajectory imitation provides useful signal but does not yet robustly compress long-horizon procedural behavior. More broadly, the results suggest that some task skills are better treated not as additional Markdown to be reinterpreted at inference time, but as compact latent controls over how a frozen model enters the task.

Related papers

• SCX Router: Streaming Zero-Shot Model Selection with a Decoder-KV Classifier and a Real-World Task Ontology [47.16406021922537]
  We introduce router, a lightweight GLiClass-based router that assigns a suitability score to each inference-time model label without autore generation.<n>For task generation, we construct a task structure with 23 families, 115 task types, 345 routable subtypes, 1,173 synthetic examples, and an axis of 30 domains.<n>We then train the Qwen3 decoder on these tasks, while explicitly separating learned request prediction from per-task policies for attributes such as eligibility, cost, cache reuse, safety, and sovereignty.
  arXiv  Detail & Related papers  (2026-09-02T08:40:07Z)
• Transcription Policy as a Latent Variable: Activating Controllable Verbatim ASR with Word-Level Timing [8.921166277011347]
  We show that models already encode both styles; the challenge is controlled activation.<n>Using coverage-aware decoder task tokens trained on parallel verbatim/intended transcript pairs, we raise German disfluency F1 from 10% to 79% zero-shot.<n>Full English-only fine-tuning surpasses all baselines in verbatim accuracy, disfluency detection, and intended-mode quality across both languages.
  arXiv  Detail & Related papers  (2026-07-21T10:19:17Z)
• VISUALSKILL: Multimodal Skills for Computer-Use Agents [63.79539541125141]
  We propose VISUALSKILL: a hierarchical multimodal skill tailored to each target application.<n>We construct each skill with a two-stage pipeline that combines authored documentation with live-application UI exploration.<n>On two CUA benchmarks, a Claude Code CLI agent backed by Claude Opus 4.6 reaches an average score of 0.456 with VISUALSKILL.
  arXiv  Detail & Related papers  (2026-06-16T19:57:07Z)
• AIP: A Graph Representation for Learning and Governing Agent Skills [1.1458853556386797]
  Agent Skills today consist largely of free-form prose requiring the agent to read, interpret, and re-derive how to act in every session.<n>This imposes two compounding costs: reduced reliability on implementation-heavy tasks, and difficulty in skill creation and improvement.<n>The Agent Instruction Protocol (AIP) addresses both by modeling a skill as a directed execution graph.<n>A compiler meta-skill translates existing human-written skills into this form.
  arXiv  Detail & Related papers  (2026-06-03T12:02:49Z)
• CORE: Context-Robust Remasking for Diffusion Language Models [51.59514489363897]
  We propose Context-Robust Remasking (CORE), a training-free framework for inference-time revision.<n>Rather than trusting static token probabilities, CORE identifies context-brittle tokens by probing their sensitivity to targeted masked-context perturbations.<n>On LLaDA-8B-Base, CORE delivers consistent improvements across reasoning and code benchmarks, outperforming compute-matched baselines and improving MBPP by up to 9.2 percentage points.
  arXiv  Detail & Related papers  (2026-02-04T00:12:30Z)
• Universal Adversarial Suffixes Using Calibrated Gumbel-Softmax Relaxation [9.099589602551573]
  We study universal adversarial suffixes that, when appended to any input, broadly reduce accuracy across tasks and models.<n>Our approach learns the suffix in a differentiable "soft" form using Gumbel-Softmax relaxation and then discretizes it for inference.<n>A single suffix trained on one model transfers effectively to others, consistently lowering both accuracy and calibrated confidence.
  arXiv  Detail & Related papers  (2025-12-09T00:03:39Z)
• MrT5: Dynamic Token Merging for Efficient Byte-level Language Models [50.46453950887946]
  This work introduces MrT5 (MergeT5), a more efficient variant of ByT5 that integrates a token deletion mechanism in its encoder to dynamically shorten the input sequence length.<n>MrT5 effectively "merges" critical information from deleted tokens into a more compact sequence, leveraging contextual information from the remaining tokens.<n>MrT5 shows comparable accuracy to ByT5 on downstream evaluations such as XNLI, TyDi QA, and character-level tasks while reducing sequence lengths by up to 75%.
  arXiv  Detail & Related papers  (2024-10-28T06:14:12Z)
• Syntax-Aware On-the-Fly Code Completion [13.268277642411974]
  We propose PyCoder to leverage token types, a kind of lightweight syntactic information. Our PyCoder achieves the first rank on the CodeXGLUE leaderboard with an accuracy of 77.12% for the token-level predictions.
  arXiv  Detail & Related papers  (2022-11-09T04:24:18Z)
• Politeness Transfer: A Tag and Generate Approach [167.9924201435888]
  This paper introduces a new task of politeness transfer. It involves converting non-polite sentences to polite sentences while preserving the meaning. We design a tag and generate pipeline that identifies stylistic attributes and subsequently generates a sentence in the target style.
  arXiv  Detail & Related papers  (2020-04-29T15:08:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.