Trip+: Benchmarking Agents in Personalized Interactive Travel Planning

• URL: http://arxiv.org/abs/2606.21169v1
• Date: Fri, 19 Jun 2026 07:17:40 GMT
• Title: Trip+: Benchmarking Agents in Personalized Interactive Travel Planning
• Abstract summary: We introduce Trip+ to measure the ability of agents to plan travel holistically.<n>Given traveler profiles and dynamic interactions, agents must generate and revise minute-level itineraries.<n>End-to-end traveler experiences are evaluated via an LLM-based simulator.
• Score: 12.167681198611744
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Interactive travel planning has become a popular use case for language models. Agents are deployed to manage evolving preferences and unexpected disruptions over multiple turns. Such settings require models to make complex, profile-conditioned planning decisions. However, existing benchmarks often evaluate feasibility, personalization, or interaction in relatively isolated settings. We therefore introduce Trip+ to measure the ability of agents to plan travel holistically. In Trip+, given traveler profiles and dynamic interactions, agents must generate and revise minute-level itineraries. End-to-end traveler experiences are evaluated via an LLM-based simulator, enabling the assessment of subjective metrics like fatigue. Our scenarios range from simple request resolutions to complex environment-driven replanning. We evaluate 18 LMs and find a consistent gap in experiential quality. Models favor technically feasible but exhausting itineraries that diverge sharply from profiled traveler preferences.

Related papers

• TRIPPULSE: Multi-Agent Travel Planning with Review-Grounded Reasoning [8.15237621583531]
  We propose TRIPPULSE, a multi- agent framework for review-grounded travel planning.<n>We augment TRIPCRAFT with 100K+ real-world reviews and introduce Review-Grounded Per- sona Alignment (RGPA), an LLM-as-a-Judge metric for evaluating alignment with human-centric travel experiences.
  arXiv  Detail & Related papers  (2026-08-31T15:02:37Z)
• GroupTravelBench: Benchmarking LLM Agents on Multi-Person Travel Planning [21.321198339300334]
  We introduce textbfGroupTravelBench, the first benchmark for textbfmulti-user, multi-turn travel planning.<n>Based on real user profiles, POI data, and ticket price data, we synthesize 650 tasks and divide them into three difficulty levels.<n>We evaluate a wide range of LLMs and find that even frontier models still show substantial weaknesses in preference coverage and group fairness.
  arXiv  Detail & Related papers  (2026-05-24T17:59:58Z)
• OdysseyArena: Benchmarking Large Language Models For Long-Horizon, Active and Inductive Interactions [66.84396313837765]
  We introduce OdysseyArena, which re-centers agent evaluation on long-horizon, active, and inductive interactions.<n>We provide a set of 120 tasks to measure an agent's inductive efficiency and long-horizon discovery.<n>We also introduce OdysseyArena-Challenge to stress-test agent stability across extreme interaction horizons.
  arXiv  Detail & Related papers  (2026-02-05T16:31:43Z)
• TravelBench: A Broader Real-World Benchmark for Multi-Turn and Tool-Using Travel Planning [22.3041021610283]
  Travel planning is a natural real-world task to test large language models (LLMs) planning and tool-use abilities.<n>TravelBench is a benchmark for fully real-world travel planning.
  arXiv  Detail & Related papers  (2025-12-27T18:25:14Z)
• COMPASS: A Multi-Turn Benchmark for Tool-Mediated Planning & Preference Optimization [47.26757420020116]
  We introduce a benchmark that evaluates agents on realistic travel-planning scenarios.<n>We build a realistic travel database covering transportation, accommodation, and ticketing for 20 U.S. National Parks.<n>We uncover two critical gaps: (i) an acceptable-optimal gap, where agents reliably meet constraints but fail to optimize preferences, and (ii) a plan-coordination gap, where performance collapses on multi-service (flight and hotel) coordination tasks.
  arXiv  Detail & Related papers  (2025-10-08T14:09:46Z)
• ATLAS: Constraints-Aware Multi-Agent Collaboration for Real-World Travel Planning [53.065247112514534]
  ATLAS is a general multi-agent framework designed to handle complex nature of constraints awareness in real-world travel planning tasks.<n>We demonstrate state-of-the-art performance on the TravelPlanner benchmark, improving the final pass rate from 23.3% to 44.4% over its best alternative.
  arXiv  Detail & Related papers  (2025-09-29T23:23:52Z)
• TripTailor: A Real-World Benchmark for Personalized Travel Planning [28.965273870656446]
  TripTailor is a benchmark for personalized travel planning in real-world scenarios.<n>This dataset features over 500,000 real-world points of interest (POIs) and nearly 4,000 diverse travel itineraries.<n>We identify several critical challenges in travel planning, including the feasibility, rationality, and personalized customization.
  arXiv  Detail & Related papers  (2025-08-02T16:44:02Z)
• Wide-Horizon Thinking and Simulation-Based Evaluation for Real-World LLM Planning with Multifaceted Constraints [39.01715254437105]
  This paper introduces the Multiple Aspects of Planning (MAoP) to solve planning problems with multifaceted constraints.<n>Instead of direct planning, MAoP leverages the strategist to conduct pre-planning from various aspects and provide the planning blueprint for planners.
  arXiv  Detail & Related papers  (2025-06-14T09:37:59Z)
• TripCraft: A Benchmark for Spatio-Temporally Fine Grained Travel Planning [7.841787597078323]
  TripCraft establishes a new benchmark for LLM driven personalized travel planning, offering a more realistic, constraint aware framework for itinerary generation.<n>Our parameter informed setting significantly enhances meal scheduling, improving the Temporal Meal Score from 61% to 80% in a 7 day scenario.
  arXiv  Detail & Related papers  (2025-02-27T20:33:28Z)
• ChinaTravel: An Open-Ended Benchmark for Language Agents in Chinese Travel Planning [38.44879526364259]
  We introduce emphChinaTravel, the first open-ended benchmark grounded in authentic Chinese travel requirements.<n>We design a compositionally generalizable domain-specific language for scalable evaluation, covering feasibility, constraint satisfaction, and preference comparison.<n> Empirical studies reveal the potential of neuro-symbolic agents in travel planning, achieving a 37.0% constraint satisfaction rate on human queries.
  arXiv  Detail & Related papers  (2024-12-18T10:10:12Z)
• Planning with Adaptive World Models for Autonomous Driving [50.4439896514353]
  We present nuPlan, a real-world motion planning benchmark that captures multi-agent interactions.<n>We learn to model such unique behaviors with BehaviorNet, a graph convolutional neural network (GCNN)<n>We also present AdaptiveDriver, a model-predictive control (MPC) based planner that unrolls different world models conditioned on BehaviorNet's predictions.
  arXiv  Detail & Related papers  (2024-06-15T18:53:45Z)
• TravelPlanner: A Benchmark for Real-World Planning with Language Agents [63.199454024966506]
  We propose TravelPlanner, a new planning benchmark that focuses on travel planning, a common real-world planning scenario. It provides a rich sandbox environment, various tools for accessing nearly four million data records, and 1,225 meticulously curated planning intents and reference plans. Comprehensive evaluations show that the current language agents are not yet capable of handling such complex planning tasks-even GPT-4 only achieves a success rate of 0.6%.
  arXiv  Detail & Related papers  (2024-02-02T18:39:51Z)
• Is Ego Status All You Need for Open-Loop End-to-End Autonomous Driving? [84.17711168595311]
  End-to-end autonomous driving has emerged as a promising research direction to target autonomy from a full-stack perspective. nuScenes dataset, characterized by relatively simple driving scenarios, leads to an under-utilization of perception information in end-to-end models. We introduce a new metric to evaluate whether the predicted trajectories adhere to the road.
  arXiv  Detail & Related papers  (2023-12-05T11:32:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.