EvolveScaler: Synthesizing Information-Evolution Contexts via Executable State Machines and Natural-Language Rendering

• URL: http://arxiv.org/abs/2609.08435v2
• Date: Wed, 09 Sep 2026 04:21:40 GMT
• Title: EvolveScaler: Synthesizing Information-Evolution Contexts via Executable State Machines and Natural-Language Rendering
• Abstract summary: We introduce EvolveScaler, a code-driven framework that defines information evolution before rendering it as natural language.<n>We instantiate EvolveScaler with 117 task prototypes and 159 final-question operators across five difficulty levels spanning approximately 7 to 1,200 events per instance.<n>On the very_long tier, the strongest model reaches 59.3% avg@5, while six models score below 10%.
• Score: 57.87517047585447
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In persistent interactions, long contexts may encode an evolving process rather than a fixed record: later events can revise or revoke earlier information, changing what remains valid and what conclusions follow. We call this setting information evolution (IE). Solving IE requires identifying valid records, applying updates in order, and reconstructing the query-relevant state from the event history. Existing text-first synthesis pipelines make such data difficult to verify because state transitions and answer logic remain implicit. We introduce EvolveScaler, a code-driven framework that defines information evolution before rendering it as natural language. Human-authored operational specifications define state transitions, record validity, difficulty controls, and executable answer logic; a strong LLM then synthesizes a self-contained simulator from each specification. Executing validated simulators produces natural-language multi-turn event histories, while deterministic replay computes reference answers and atomic checklists. We instantiate EvolveScaler with 117 task prototypes and 159 final-question operators across five difficulty levels spanning approximately 7 to 1,200 events per instance, yielding about 35,100 training examples and 585 validated evaluation instances. On the very_long tier, the strongest model reaches 59.3% avg@5, while six models score below 10%. Training an internal A3B model on 6,000 EvolveScaler examples improves performance over its base checkpoint on all eight independently constructed out-of-distribution benchmarks, with a 5.25-point average gain. These results show that code-driven IE synthesis provides both challenging evaluation and transferable training supervision.

Related papers

• EvoGenUI-Bench: Evaluating LLMs as Multi-Turn Generative UI Assistants [10.510136510866099]
  Large language models can generate interactive web interfaces, but reliable generative UI requires maintaining an executable artifact as user requests evolve.<n>We introduce EvoGenUI-Bench, a benchmark for multi-turn interface maintenance comprising 150 five-turn tasks and 750 turns across three scenarios.<n>We execute generated artifacts in a browser and evaluate them using screenshots, source and DOM evidence, actor traces, and runtime logs.
  arXiv  Detail & Related papers  (2026-08-29T17:54:02Z)
• VeriEvol: Scaling Multimodal Mathematical Reasoning via Verifiable Evol-Instruct [71.22358682581215]
  Scaling reinforcement learning for visual mathematical reasoning requires more than generating harder questions.<n>We instantiate this as VeriEvol, an iterative framework with two components.<n>On a five-benchmark visual-math suite, scaling evolved SFT data from 10K to 250K samples raises the mean accuracy from 35.42 to 54.73.
  arXiv  Detail & Related papers  (2026-06-22T16:17:30Z)
• Agentic Insight Generation in VSM Simulations [4.626393563516877]
  We propose a decoupled, two-step agentic architecture for value stream map simulations.<n>By separating orchestration from data analysis, the system leverages progressive data discovery infused with domain expert knowledge.<n>Results from multiple state-of-the-art large language models demonstrate the framework's viability.
  arXiv  Detail & Related papers  (2026-04-14T08:11:53Z)
• Pioneer Agent: Continual Improvement of Small Language Models in Production [6.452260317191361]
  Small language models are attractive for production deployment due to their low cost, fast inference, and ease of specialization.<n>We present Pioneer Agent, a closed-loop system that automates the process of adapting small language models to a specific task.
  arXiv  Detail & Related papers  (2026-04-10T18:13:09Z)
• ClawArena: Benchmarking AI Agents in Evolving Information Environments [61.664633997138004]
  ClawArena is a benchmark for evaluating AI agents in evolving information environments.<n>Each scenario maintains a complete hidden ground truth while exposing the agent only to noisy, partial, and sometimes contradictory traces.<n> evaluation is organized around three coupled challenges: multi-source conflict reasoning, dynamic belief revision, and implicit personalization.
  arXiv  Detail & Related papers  (2026-04-05T17:55:23Z)
• SimuScene: Training and Benchmarking Code Generation to Simulate Physical Scenarios [71.65387146697319]
  Large language models (LLMs) have been extensively studied for tasks like math competitions, complex coding, and scientific reasoning.<n>We propose SimuScene, the first systematic study that trains and evaluates LLMs on simulating physical scenarios.<n>We build an automatic pipeline to collect data, with human verification to ensure quality.
  arXiv  Detail & Related papers  (2026-02-11T13:26:02Z)
• Scaling the Scaling Logic: Agentic Meta-Synthesis of Logic Reasoning [18.75349680577575]
  SSLogic is a framework for continuous family evolution with controllable difficulty.<n>Training on SSLogic-evolved data yields consistent gains over the seed baseline at matched steps.
  arXiv  Detail & Related papers  (2026-01-23T13:26:01Z)
• One Battle After Another: Probing LLMs' Limits on Multi-Turn Instruction Following with a Benchmark Evolving Framework [51.50565654314582]
  Large language models can follow users' instructions throughout a dialogue spanning multiple topics.<n>Existing benchmarks are often limited to a fixed number of turns, making them susceptible to saturation and failing to account for the user's interactive experience.<n>We propose a framework for assessing multi-turn instruction-following ability.
  arXiv  Detail & Related papers  (2025-11-05T14:39:59Z)
• AutoEvoEval: An Automated Framework for Evolving Close-Ended LLM Evaluation Data [0.6278186810520364]
  Large language models (LLMs) have shown remarkable performance on various tasks.<n>Existing evaluation benchmarks are often static and insufficient to fully assess their robustness and generalization.<n>We propose AutoEvoEval, an evolution-based evaluation framework for close-ended tasks such as question answering.
  arXiv  Detail & Related papers  (2025-06-30T11:18:56Z)
• Logical Reasoning for Task Oriented Dialogue Systems [57.440956636333325]
  We propose a novel method to fine-tune transformer models such as Roberta and T5 to reason over a set of facts in a given dialogue context. Our method includes a synthetic data generation mechanism which helps the model learn logical relations. We show that the transformer based model can perform logical reasoning to answer questions when the dialogue context contains all the required information.
  arXiv  Detail & Related papers  (2022-02-08T21:46:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.