NOVA: A Verification-Aware Agent Harness for Architecture Evolution in Industrial Recommender Systems

• URL: http://arxiv.org/abs/2606.27243v2
• Date: Fri, 26 Jun 2026 11:32:22 GMT
• Title: NOVA: A Verification-Aware Agent Harness for Architecture Evolution in Industrial Recommender Systems
• Abstract summary: NOVA is a level-aware agent harness for verification-aware architecture evolution.<n>It achieves the highest effective pass rate on L2 ScaleUp and L3 Literature-to-Production tasks.<n>It shortens one literature-to-production cycle by over 13x in human-attended time.
• Score: 24.27056236910489
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Industrial advertising recommender models are continuously improved through architecture evolution. Upgrades such as RankMixer, TokenMixer-Large, and MixFormer show that better structures remain a key source of quality and business gains. Yet developing such upgrades in production is expert-intensive and difficult to scale. Existing automation is insufficient: AutoML mainly tunes hyper-parameters, while effective gains often require cross-module changes under strict constraints; generic LLM coding agents optimize for runnable code, but runnable code does not imply a valid recommender architecture. Candidates may pass local tests while causing silent failures that degrade performance. We present NOVA, a level-aware agent harness for verification-aware architecture evolution. NOVA uses an architecture gradient, an SGD-inspired, non-differentiable update signal that aggregates prior modifications, verification diagnostics, metric feedback, and trajectory memory to guide the next modification. A verification cascade checks structure semantics, local executability, offline effectiveness, and online impact; invalid candidates are blocked early, with failure patterns recorded as forbidden directions. L1--L4 task-level control matches automation to task complexity and risk, routing high-risk tasks to Copilot for human oversight. Deployed in an industrial advertising system, NOVA achieves the highest effective pass rate on L2 ScaleUp and L3 Literature-to-Production tasks (54.5% and 60.0%), reduces silent failures compared with coding-agent baselines, and shortens one literature-to-production cycle by over 13x in human-attended time. In online A/B testing, the selected L3 candidate improves GMV on three pCVR objectives by +1.25%, +1.70%, and +2.02%, while reducing pCVR bias by 58.8%, 66.7%, and 37.3%.

Related papers

• AutoSaddler: Automatic Harness Optimization with Durable Updates from Agent Execution Traces [61.51487329124135]
  AutoSaddler is a framework that formulates harness improvement as an offline learning problem.<n>It iteratively updates the harness using failure signals from mini-batches.<n>Experiments on GAIA2, SWE-Bench Pro, and Terminal-Bench 2.0 show that AutoSaddler substantially improves agent performance.
  arXiv  Detail & Related papers  (2026-08-24T09:45:08Z)
• REFINE: A Multi-Agent LLM Approach for Evidence-Guided Code Refactoring [3.294884078309524]
  REFINE is a tool-agnostic, evidence-aware multi-agent approach for generating Java file-level candidates.<n>We evaluate REFINE on 450 Java files from 15 open-source systems.
  arXiv  Detail & Related papers  (2026-08-21T17:44:40Z)
• LEGO-RL: Harness-Native Reinforcement Learning for Coding Agents [56.902747731541695]
  We present LEGO-RL, a framework that bridges native coding-agent harnesses with scalable policy-gradient optimization.<n>We evaluate LEGO-RL by training the sparse MoE model Qwen3.5-35B-A3B with GSPO across three native coding-agent harnesses.
  arXiv  Detail & Related papers  (2026-08-18T05:34:35Z)
• Perception, Verdict, and Evolution: Hindsight-Driven Self-Refining Forensics Agent for AI-Generated Image Detection [80.15317858033534]
  ForeAgent is an agentic forensics framework for AI-generated image detection with iterative self-evolution.<n>ForeAgent achieves state-of-the-art performance on the Chameleon benchmark, reaching 82.18% accuracy.<n>ForeAgent produces more consistent and causally grounded reasoning compared to GPT-5 and GPT-5-mini.
  arXiv  Detail & Related papers  (2026-06-25T02:59:33Z)
• BLAgent: Agentic RAG for File-Level Bug Localization [2.2917707112773593]
  BLAgent is a novel agentic RAG framework for file-level bug localization.<n> BLAgent attains over 78% Top-1 accuracy with open-source models.<n> BLAgent improves end-to-end repair success by over 20%.
  arXiv  Detail & Related papers  (2026-05-18T07:20:13Z)
• AdaExplore: Failure-Driven Adaptation and Diversity-Preserving Search for Efficient Kernel Generation [59.964989458924585]
  AdaExplore is an agent framework that enables self-improvement via accumulated execution feedback for performance-critical kernel code generation.<n>In the adaptation stage, the agent synthesizes tasks and converts recurring failures into a reusable memory of validity rules.<n>In the search stage, the agent organizes candidate kernels as a tree and alternates between small local refinements and larger structural regeneration.
  arXiv  Detail & Related papers  (2026-04-17T18:25:03Z)
• AVO: Agentic Variation Operators for Autonomous Evolutionary Search [38.64857334672555]
  Agentic Variation Operators (AVO) are evolutionary variation operators that replace the fixed mutation, crossover, and hand-designeds of classical evolutionary search.<n>We evaluate AVO on attention, among the most aggressively optimized kernel targets in AI, on NVIDIA Blackwell (B200)<n>AVO instantiates variation as a self-directed agent loop that can consult the current lineage, a domain-specific knowledge base, and execution feedback.
  arXiv  Detail & Related papers  (2026-03-25T16:55:04Z)
• Architecture-Aware Multi-Design Generation for Repository-Level Feature Addition [53.50448142467294]
  RAIM is a multi-design and architecture-aware framework for repository-level feature addition.<n>It shifts away from linear patching by generating multiple diverse implementation designs.<n>Experiments on the NoCode-bench Verified dataset demonstrate that RAIM establishes a new state-of-the-art performance.
  arXiv  Detail & Related papers  (2026-03-02T12:50:40Z)
• AWE: Adaptive Agents for Dynamic Web Penetration Testing [0.0]
  AWE is a memory-augmented multi-agent framework for autonomous web penetration testing.<n>It embeds structured, vulnerability-specific analysis pipelines within a lightweight LLM orchestration layer.<n>AWE achieves substantial gains on injection-class vulnerabilities.
  arXiv  Detail & Related papers  (2026-03-01T07:32:42Z)
• ROAD: Reflective Optimization via Automated Debugging for Zero-Shot Agent Alignment [1.6968020497268546]
  ROAD is a novel framework that treats optimization as a dynamic debug investigation rather than a search.<n>Road is highly sample-efficient, achieving a 5.6 percent increase in success rate and a 3.8 percent increase in search accuracy.<n>These findings suggest that mimicking the human engineering loop of failure analysis and patching offers a viable, data-efficient alternative to resource-intensive training.
  arXiv  Detail & Related papers  (2025-12-30T07:31:34Z)
• Towards Engineering Multi-Agent LLMs: A Protocol-Driven Approach [13.760107452858044]
  This paper introduces Software Engineering Multi-Agent Protocol (SEMAP), a protocol-layer methodology that instantiates three core SE design principles for multi-agents.<n>In code development, it achieves up to a 69.6% reduction in total failures function-level development and 56.7% for deployment-level development.
  arXiv  Detail & Related papers  (2025-10-14T03:49:30Z)
• AegisLLM: Scaling Agentic Systems for Self-Reflective Defense in LLM Security [74.22452069013289]
  AegisLLM is a cooperative multi-agent defense against adversarial attacks and information leakage.<n>We show that scaling agentic reasoning system at test-time substantially enhances robustness without compromising model utility.<n> Comprehensive evaluations across key threat scenarios, including unlearning and jailbreaking, demonstrate the effectiveness of AegisLLM.
  arXiv  Detail & Related papers  (2025-04-29T17:36:05Z)
• DS-Agent: Automated Data Science by Empowering Large Language Models with Case-Based Reasoning [56.887047551101574]
  We present DS-Agent, a novel framework that harnesses large language models (LLMs) agent and case-based reasoning (CBR) In the development stage, DS-Agent follows the CBR framework to structure an automatic iteration pipeline, which can flexibly capitalize on the expert knowledge from Kaggle. In the deployment stage, DS-Agent implements a low-resource deployment stage with a simplified CBR paradigm, significantly reducing the demand on foundational capabilities of LLMs.
  arXiv  Detail & Related papers  (2024-02-27T12:26:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.