ContractSkill: Repairable Contract-Based Skills for Multimodal Web Agents

• URL: http://arxiv.org/abs/2603.20340v1
• Date: Fri, 20 Mar 2026 09:25:50 GMT
• Title: ContractSkill: Repairable Contract-Based Skills for Multimodal Web Agents
• Abstract summary: We present ContractSkill, a framework that converts a draft skill into a contracted executable artifact.<n>This representation enables deterministic verification, step-level fault localization, and minimal patch-based repair.
• Score: 13.2889725132666
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Despite rapid progress in multimodal GUI agents, reusable skill acquisition remains difficult because on-demand generated skills often leave action semantics, state assumptions, and success criteria implicit. This makes them brittle to execution errors, hard to verify, and difficult to repair. We present ContractSkill, a framework that converts a draft skill into a contracted executable artifact with explicit preconditions, step specifications, postconditions, recovery rules, and termination checks. This representation enables deterministic verification, step-level fault localization, and minimal patch-based repair, turning skill refinement into localized editing rather than full regeneration. Experiments on VisualWebArena and MiniWoB with GLM-4.6V and Qwen3.5-Plus show that ContractSkill improves self-generated skills from 9.4% and 10.9% to 28.1% and 37.5% on VisualWebArena, and from 66.5% and 60.5% to 77.5% and 81.0% on MiniWoB. Repaired artifacts also transfer across models, improving the target model's self-generated-skill baseline by up to 47.8 points and 12.8 points on the two benchmarks, respectively. These results suggest that agent skills are better treated as explicit procedural artifacts that can be verified, repaired, and shared across models.

Related papers

• Signal or Noise? A Benchmark Study of Agent Skills in Web Development [6.7220476997481375]
  We use WebDev-Skills-Bench to study 31 public WebDev Skills on 50 Web-Bench projects and 1,000 ordered tasks.<n>Across four models, target Skill injection reduces mean Pass@2 by 1.3% to 4.2%, lowers task completion depth, and increases token cost by 72% to 394%.<n>Losses concentrate on easy early tasks, Skill rankings transfer weakly across models, and anti-pattern rules outperform example-heavy content within helpful Skills.
  arXiv  Detail & Related papers  (2026-08-24T10:13:07Z)
• When Not to Imitate: Boundary-Aware Skill Memory for Reliable Tool-Use LLM Agents [62.10623192614306]
  Extracting skills from past successes is critical for the efficient evolution of Large Language Model (LLM) agents.<n>We propose textbfBoundary-Aware Skill Memory (BASM), which augments each skill with explicit boundary fields.<n>BASM consistently outperforms success-distilled skill-memory baselines.
  arXiv  Detail & Related papers  (2026-08-23T10:16:06Z)
• Demystifying Agent Skills: Why They Work-Until They Don't [58.92875630998418]
  Existing evaluations largely measure whether skills improve aggregated task success, leaving a more fundamental question underexplored: emphtextbfWhen do skills help, why do they work, and where do they fail?<n>We isolate the effects of representation, outcome annotation, retrieval difficulty, and cross-framework of skills.<n>We consolidate these observations into a taxonomy of three high-level categories and twelve skill-use modes.
  arXiv  Detail & Related papers  (2026-08-14T07:26:38Z)
• Toward Skill-Native LLMs: Skill Entropy for Benchmarking and Training Long-Horizon Reasoning [72.09826781730662]
  Long-horizon reasoning demands that the model switch between distinct skills inside a reasoning chain.<n>We introduce Skill Entropy, a measure of the difficulty of switching from one skill to another.<n>We propose Skill-Entropy RL, an RL framework where the model predicts not only the answer at each step but also the skill used to produce it.
  arXiv  Detail & Related papers  (2026-08-05T17:57:16Z)
• Not All Skills Help: Measuring and Repairing Agent Knowledge [47.498075849595374]
  ASSAY is a framework that separates generation from curation.<n>It computes a per-skill causal attribution on a small development set.<n>It restructures the library offline, and suppresses skills with negative predicted effect for each test task.
  arXiv  Detail & Related papers  (2026-06-13T16:37:38Z)
• Converted, Not Equivalent: Benchmarking Codebase Conversion via Observational Equivalence [56.25095230687242]
  Coding agents often over-trust their own local validation routines and declare success on artifacts that satisfy surface checks.<n>This problem is especially acute in conversion, where prior evaluation is outcome-driven.<n>Across blind conversion attempts, the best system reaches 26.7--28.9% overall pass rate despite Spec pass rates up to 91.1%.<n>This suggests that failures stem more from contract-misaligned self-validation than from limited budget or backbone strength.
  arXiv  Detail & Related papers  (2026-05-27T19:57:15Z)
• BadSkill: Backdoor Attacks on Agent Skills via Model-in-Skill Poisoning [34.60596020541521]
  We present BadSkill, a backdoor attack formulation that targets model-in-skill threat surface.<n>In BadSkill, an adversary publishes a seemingly benign skill whose embedded model is backdoor-fine-tuned to activate a hidden payload.<n>Our benchmark spans 13 skills, including 8 triggered tasks and 5 non-trigger control skills, with a combined main evaluation set of 571 negative-class queries and 396 trigger-aligned queries.<n>BadSkill achieves up to 99.5% average attack success rate (ASR) across the eight triggered skills while maintaining strong benign-side accuracy on negative-class queries.
  arXiv  Detail & Related papers  (2026-04-10T14:48:29Z)
• EvoSkill: Automated Skill Discovery for Multi-Agent Systems [6.319876096746374]
  We introduce textbfEvoSkill, a self-evolving framework that automatically discovers and refines agent skills.<n>EvoSkill analyzes execution failures, proposes new skills or edits to existing ones, and materializes them into structured, reusable skill folders.<n>We evaluate EvoSkill on two benchmarks: OfficeQA, a grounded reasoning benchmark over U.S. Treasury data, and SealQA, a noisy retrieval benchmark.
  arXiv  Detail & Related papers  (2026-03-03T09:07:22Z)
• SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks [61.89812116484928]
  Agent Skills are structured packages of procedural knowledge that augment LLM agents at inference time.<n>We present SkillsBench, a benchmark of 86 tasks across 11 domains paired with curated Skills and deterministic verifiers.<n>We show curated Skills raise average pass rate by 16.2 percentage points(pp), but effects vary widely by domain.<n>Self-generated Skills provide no benefit on average, showing that models cannot reliably author the procedural knowledge they benefit from consuming.
  arXiv  Detail & Related papers  (2026-02-13T07:06:06Z)
• Beyond Accuracy: Behavioral Dynamics of Agentic Multi-Hunk Repair [6.60715519922201]
  Repairing multi-hunk bugs requires coordinated edits across multiple, disjoint code regions.<n>We evaluate coding agents (Claude Code, Codex, Gemini-cli, and Qwen Code) on 372 multi-hunk bugs from the Hunk4J dataset.
  arXiv  Detail & Related papers  (2025-11-14T07:00:47Z)
• Exploring Expert Failures Improves LLM Agent Tuning [74.0772570556016]
  We propose Exploring Expert Failures (EEF), which identifies beneficial actions from failed expert trajectories.<n>EEF successfully solves some previously unsolvable subtasks and improves agent tuning performance.
  arXiv  Detail & Related papers  (2025-04-17T17:53:54Z)
• Exposing Limitations of Language Model Agents in Sequential-Task Compositions on the Web [69.6913064185993]
  Language model agents (LMA) emerged as a promising paradigm on muti-step decision making tasks.<n>Despite the promise, their performance on real-world applications is still underexplored.<n>We show that while existing LMAs achieve 94.0% average success rate on base tasks, their performance degrades to 24.9% success rate on compositional tasks.
  arXiv  Detail & Related papers  (2023-11-30T17:50:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.