Agentic Commerce World: An Auditable and Verifiable Environment for Vibe Commerce

• URL: http://arxiv.org/abs/2608.02441v1
• Date: Mon, 03 Aug 2026 16:20:17 GMT
• Title: Agentic Commerce World: An Auditable and Verifiable Environment for Vibe Commerce
• Abstract summary: Vibe commerce allows people to express buying or selling goals in natural language and delegate tasks to AI agents.<n>We introduce Agentic Commerce World (ACWorld), an environment for evaluating such agents across ongoing transactions.<n>ACWorld validates agent actions before updating shared transaction state and records the resulting interactions, making agent behavior auditable and evaluation reproducible.
• Score: 83.90449137361595
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In vibe coding, people describe software in natural language and delegate implementation to AI agents. By analogy, vibe commerce allows people to express buying or selling goals in natural language and delegate the corresponding tasks to agents. Commerce, however, requires independently controlled Buyer and Merchant agents to interact in a shared market while preserving their private objectives and distinct authority. We introduce Agentic Commerce World (ACWorld), an environment for evaluating such agents across ongoing transactions. Through its Vibe Commerce Protocol (VCP), ACWorld validates agent actions before updating shared transaction state and records the resulting interactions, making agent behavior auditable and evaluation reproducible. The ACWorld Benchmark contains a 200-task capability-coverage track and a 60-task large-catalog track that searches 785,022 transactable listings. Across ten models, mean scores range from 65.9% to 85.6% and from 56.1% to 91.4%, respectively. Our analysis shows that process-level evidence is necessary: final state alone can miss evaluated errors, incomplete trajectories still retain useful process signals, and large-catalog tasks expose bottlenecks across stages.

Related papers

• Efficient Test-Time Adaptation through Human-AI Interaction [112.5131603022122]
  We propose test-time adaptation through human-agent interaction (TAHI)<n>We adapt agents to 30 individuals in two high-utility domains, writing and visual creation, on a total of 600 tasks.<n>Our agents improve solo task success by 4.5-20.9% within only tens of tasks.
  arXiv  Detail & Related papers  (2026-09-03T17:33:18Z)
• Messier: A High-Resolution Corpus for Cross-Benchmark Agent Evaluation [37.16322609017877]
  Messier is a unified corpus of 957,253 records that span 30 benchmarks, 714 agents, 11,891 tasks, and 74,205 verifiers.<n>M Messier consolidates public benchmark scores and supplements them with five-agent runs across six underrepresented professional and scientific domains.
  arXiv  Detail & Related papers  (2026-07-28T15:50:19Z)
• EComAgentBench: Benchmarking Shopping Agents on Long-Horizon Tasks with Distributed Hidden Intent [8.1223119066448]
  EComAgentBench is a benchmark of 662 tasks grounded in real Amazon products and reviews.<n>Each task scatters requirements across a visible query, a tool-gated profile, and scripted clarification.<n> typed, source-tagged rubrics grade every task, attributing each failure to a requirement and its source.
  arXiv  Detail & Related papers  (2026-06-16T09:10:48Z)
• CHI-Bench: Can AI Agents Automate End-to-End, Long-Horizon, Policy-Rich Healthcare Workflows? [150.84850629123287]
  End-to-end automation of realistic healthcare operations stresses three capabilities underrepresented in current benchmarks.<n>$$-Bench is a benchmark of long-horizon healthcare across three domains: provider prior authorization, payer utilization management, and care management.<n>Across 30 agent harness/models configurations, the best agent resolves only 28.0% of tasks, no agent clears 20% on strict pass3, and executing all tasks in a single session slumps the performance to 3.8%.
  arXiv  Detail & Related papers  (2026-05-15T22:34:31Z)
• AgentIF-OneDay: A Task-level Instruction-Following Benchmark for General AI Agents in Daily Scenarios [49.90735676070039]
  The capacity of AI agents to effectively handle tasks of increasing duration and complexity continues to grow.<n>We argue that current evaluations prioritize increasing task difficulty without sufficiently addressing the diversity of agentic tasks.<n>We propose AgentIF-OneDay, aimed at determining whether general users can utilize natural language instructions and AI agents to complete a diverse array of daily tasks.
  arXiv  Detail & Related papers  (2026-01-28T13:49:18Z)
• Holistic Agent Leaderboard: The Missing Infrastructure for AI Agent Evaluation [87.47155146067962]
  We provide a standardized evaluation harness that orchestrates parallel evaluations across hundreds of tasks.<n>We conduct three-dimensional analysis spanning models, scaffolds, and benchmarks.<n>Our analysis reveals surprising insights, such as higher reasoning effort reducing accuracy in the majority of runs.
  arXiv  Detail & Related papers  (2025-10-13T22:22:28Z)
• ManiAgent: An Agentic Framework for General Robotic Manipulation [30.154478145473792]
  We introduce ManiAgent, an agentic architecture for general manipulation tasks.<n>Multiple agents involve inter-agent communication to perform environmental perception, sub-task decomposition and action generation.<n>ManiAgent achieves an 86.8% success rate on the SimplerEnv benchmark and 95.8% on real-world pick-and-place tasks.
  arXiv  Detail & Related papers  (2025-10-13T17:34:48Z)
• Evaluating Multi-Turn Bargain Skills in LLM-Based Seller Agent [20.0134260493017]
  We introduce a multi-turn evaluation framework for measuring the bargaining ability of seller agents in e-commerce dialogues.<n>Our contributions are: (1) a large-scale e-commerce bargaining benchmark spanning 622 categories, 9,892 products, and 3,014 tasks; (2) a turn-level evaluation framework grounded in Theory of Mind with annotated buyer intents, moving beyond outcome-only metrics; and (3) an automated pipeline that extracts reliable intent from massive dialogue data.
  arXiv  Detail & Related papers  (2025-09-08T05:12:03Z)
• OmniEAR: Benchmarking Agent Reasoning in Embodied Tasks [52.87238755666243]
  We present OmniEAR, a framework for evaluating how language models reason about physical interactions, tool usage, and multi-agent coordination in embodied tasks.<n>We model continuous physical properties and complex spatial relationships across 1,500 scenarios spanning household and industrial domains.<n>Our systematic evaluation reveals severe performance degradation when models must reason from constraints.
  arXiv  Detail & Related papers  (2025-08-07T17:54:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.