Efficient Test-Time Adaptation through Human-AI Interaction
Abstract Overview
This paper introduces TAHI (Test-time Adaptation through Human-agent Interaction), a framework enabling interactive AI agents to adapt to individual human expertise on open-ended tasks. TAHI updates either agent context (memory and skills) or model weights via Direct Preference Optimization using multi-turn interaction signals, including messages, plan adjustments, direct deliverable edits, and rubric adjustments, alongside an evolving verifier module that turns interaction histories into evaluation rubrics. The authors evaluate the framework by adapting agents across 30 human participants on 600 tasks in paper abstract writing and data visualization. The results demonstrate that agents efficiently adapt to individual user preferences within 20 task sessions while also acquiring shared domain standards.
Novelty
TAHI frames multi-channel, cross-session human-agent interaction as a streaming supervision signal for efficient test-time agent adaptation across both context and weight parameters. It also couples personalization with an evolving verifier module that dynamically extracts and refines evaluation rubrics directly from user interaction traces.
Results
During online adaptation, context- and weight-adaptive agents improved solo task success rates by 4.5–12.9% and 4.5–20.9%, respectively, demonstrating statistically significant gains over unadapted baselines and generalizing to held-out tasks. Evolved rubrics captured 16.0–22.3% more failure modes on imperfect agent solutions than rubrics generated by LLMs or humans alone. In addition, integrating non-message interaction actions outperformed message-only feedback, while weight adaptation achieved higher inference efficiency by requiring 62.6–88.3% fewer input tokens than context adaptation.
Key Points
- TAHI adapts agents to individual human experts at test time using multi-channel interaction signals across abstract writing and data visualization tasks.
- The evolving rubric module derives comprehensive evaluation criteria directly from interaction traces, capturing substantially more failure modes than static LLM or human rubrics.
- Both context and weight adaptation yield significant solo performance gains, with weight adaptation providing much greater token efficiency at inference time.