Structured Preference Optimization for Vision-Language Long-Horizon Task Planning

• URL: http://arxiv.org/abs/2502.20742v2
• Date: Thu, 06 Mar 2025 12:50:44 GMT
• Title: Structured Preference Optimization for Vision-Language Long-Horizon Task Planning
• Authors: Xiwen Liang, Min Lin, Weiqi Ruan, Rongtao Xu, Yuecheng Liu, Jiaqi Chen, Bingqian Lin, Yuzheng Zhuang, Xiaodan Liang,
• Abstract summary: Existing methods for vision-language task planning excel in short-horizon tasks but often fall short in complex, long-horizon planning within dynamic environments.<n>These challenges arise from the difficulty of effectively training models to produce high-quality reasoning processes for long-horizon tasks.<n>We propose Structured Preference Optimization (SPO), which aims to enhance reasoning and action selection in long-horizon task planning.
• Score: 60.26885165189447
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Existing methods for vision-language task planning excel in short-horizon tasks but often fall short in complex, long-horizon planning within dynamic environments. These challenges primarily arise from the difficulty of effectively training models to produce high-quality reasoning processes for long-horizon tasks. To address this, we propose Structured Preference Optimization (SPO), which aims to enhance reasoning and action selection in long-horizon task planning through structured preference evaluation and optimized training strategies. Specifically, SPO introduces: 1) Preference-Based Scoring and Optimization, which systematically evaluates reasoning chains based on task relevance, visual grounding, and historical consistency; and 2) Curriculum-Guided Training, where the model progressively adapts from simple to complex tasks, improving its generalization ability in long-horizon scenarios and enhancing reasoning robustness. To advance research in vision-language long-horizon task planning, we introduce ExtendaBench, a comprehensive benchmark covering 1,509 tasks across VirtualHome and Habitat 2.0, categorized into ultra-short, short, medium, and long tasks. Experimental results demonstrate that SPO significantly improves reasoning quality and final decision accuracy, outperforming prior methods on long-horizon tasks and underscoring the effectiveness of preference-driven optimization in vision-language task planning. Specifically, SPO achieves a +5.98% GCR and +4.68% SR improvement in VirtualHome and a +3.30% GCR and +2.11% SR improvement in Habitat over the best-performing baselines.

Related papers

• World Modeling Makes a Better Planner: Dual Preference Optimization for Embodied Task Planning [60.100794160682646]
  We propose a new learning framework that jointly optimize state prediction and action selection through preference learning. To automatically collect trajectories and stepwise preference data without human annotation, we introduce a tree search mechanism for extensive exploration via trial-and-error. Our method significantly outperforms existing methods and GPT-4o when applied to Qwen2-VL (7B), LLaVA-1.6 (7B), and LLaMA-3.2 (11B)
  arXiv  Detail & Related papers  (2025-03-13T15:49:56Z)
• MPO: Boosting LLM Agents with Meta Plan Optimization [37.35230659116656]
  Large language models (LLMs) have enabled agents to successfully tackle interactive planning tasks. Existing approaches often suffer from planning hallucinations and require retraining for each new agent. We propose the Meta Plan Optimization framework, which enhances agent planning capabilities by directly incorporating explicit guidance.
  arXiv  Detail & Related papers  (2025-03-04T14:54:45Z)
• Generalization of Compositional Tasks with Logical Specification via Implicit Planning [14.46490764849977]
  We introduce a new hierarchical RL framework that enhances the efficiency and optimality of task generalization. At the high level, we present an implicit planner specifically designed for generalizing compositional tasks. It learns a latent transition model and performs planning in the latent space by using a graph neural network (GNN)
  arXiv  Detail & Related papers  (2024-10-13T00:57:10Z)
• Propose, Assess, Search: Harnessing LLMs for Goal-Oriented Planning in Instructional Videos [48.15438373870542]
  VidAssist is an integrated framework designed for zero/few-shot goal-oriented planning in instructional videos. It employs a breadth-first search algorithm for optimal plan generation. Experiments demonstrate that VidAssist offers a unified framework for different goal-oriented planning setups.
  arXiv  Detail & Related papers  (2024-09-30T17:57:28Z)
• On The Planning Abilities of OpenAI's o1 Models: Feasibility, Optimality, and Generalizability [59.72892401927283]
  We evaluate the planning capabilities of OpenAI's o1 models across a variety of benchmark tasks. Our results reveal that o1-preview outperforms GPT-4 in adhering to task constraints.
  arXiv  Detail & Related papers  (2024-09-30T03:58:43Z)
• Exploring and Benchmarking the Planning Capabilities of Large Language Models [57.23454975238014]
  This work lays the foundations for improving planning capabilities of large language models (LLMs) We construct a comprehensive benchmark suite encompassing both classical planning benchmarks and natural language scenarios. We investigate the use of many-shot in-context learning to enhance LLM planning, exploring the relationship between increased context length and improved planning performance.
  arXiv  Detail & Related papers  (2024-06-18T22:57:06Z)
• A Survey of Optimization-based Task and Motion Planning: From Classical To Learning Approaches [15.136760934936381]
  Task and Motion Planning (TAMP) integrates high-level task planning and low-level motion planning to equip robots with the autonomy to reason over long-horizon, dynamic tasks. This survey provides a comprehensive review on optimization-based TAMP, covering (i) planning domain representations, (ii) individual solution strategies for components, including AI planning and trajectory optimization (TO), and (iii) the dynamic interplay between logic-based task planning and model-based TO.
  arXiv  Detail & Related papers  (2024-04-03T15:38:36Z)
• EmbodiedGPT: Vision-Language Pre-Training via Embodied Chain of Thought [95.37585041654535]
  Embodied AI is capable of planning and executing action sequences for robots to accomplish long-horizon tasks in physical environments. In this work, we introduce EmbodiedGPT, an end-to-end multi-modal foundation model for embodied AI. Experiments show the effectiveness of EmbodiedGPT on embodied tasks, including embodied planning, embodied control, visual captioning, and visual question answering.
  arXiv  Detail & Related papers  (2023-05-24T11:04:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.