PlanViz: Evaluating Planning-Oriented Image Generation and Editing for Computer-Use Tasks

• URL: http://arxiv.org/abs/2602.06663v1
• Date: Fri, 06 Feb 2026 12:47:16 GMT
• Title: PlanViz: Evaluating Planning-Oriented Image Generation and Editing for Computer-Use Tasks
• Authors: Junxian Li, Kai Liu, Leyang Chen, Weida Wang, Zhixin Wang, Jiaqi Xu, Fan Li, Renjing Pei, Linghe Kong, Yulun Zhang,
• Abstract summary: We propose PlanViz, a new benchmark designed to evaluate image generation and editing for computer-use tasks.<n>Three new sub-tasks are designed: route planning, work diagramming, and web&UI displaying.<n>For challenges of comprehensive and exact evaluation, a task-adaptive score, PlanScore, is proposed.
• Score: 52.5195594960371
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Unified multimodal models (UMMs) have shown impressive capabilities in generating natural images and supporting multimodal reasoning. However, their potential in supporting computer-use planning tasks, which are closely related to our lives, remain underexplored. Image generation and editing in computer-use tasks require capabilities like spatial reasoning and procedural understanding, and it is still unknown whether UMMs have these capabilities to finish these tasks or not. Therefore, we propose PlanViz, a new benchmark designed to evaluate image generation and editing for computer-use tasks. To achieve the goal of our evaluation, we focus on sub-tasks which frequently involve in daily life and require planning steps. Specifically, three new sub-tasks are designed: route planning, work diagramming, and web&UI displaying. We address challenges in data quality ensuring by curating human-annotated questions and reference images, and a quality control process. For challenges of comprehensive and exact evaluation, a task-adaptive score, PlanScore, is proposed. The score helps understanding the correctness, visual quality and efficiency of generated images. Through experiments, we highlight key limitations and opportunities for future research on this topic.

Related papers

• Ranking-aware adapter for text-driven image ordering with CLIP [76.80965830448781]
  We propose an effective yet efficient approach that reframes the CLIP model into a learning-to-rank task.<n>Our approach incorporates learnable prompts to adapt to new instructions for ranking purposes.<n>Our ranking-aware adapter consistently outperforms fine-tuned CLIPs on various tasks.
  arXiv  Detail & Related papers  (2024-12-09T18:51:05Z)
• VeriGraph: Scene Graphs for Execution Verifiable Robot Planning [33.8868315479384]
  We propose VeriGraph, a framework that integrates vision-language models (VLMs) for robotic planning while verifying action feasibility. VeriGraph employs scene graphs as an intermediate representation, capturing key objects and spatial relationships to improve plan verification and refinement. Our approach significantly enhances task completion rates across diverse manipulation scenarios, outperforming baseline methods by 58% for language-based tasks and 30% for image-based tasks.
  arXiv  Detail & Related papers  (2024-11-15T18:59:51Z)
• Learning A Low-Level Vision Generalist via Visual Task Prompt [43.54563263106761]
  We propose a Visual task Prompt-based Image Processing (VPIP) framework to overcome these challenges. VPIP employs visual task prompts to manage tasks with different input-target domains and allows flexible selection of backbone network. Based on the VPIP framework, we train a low-level vision generalist model, namely GenLV, on 30 diverse tasks.
  arXiv  Detail & Related papers  (2024-08-16T08:37:56Z)
• VSP: Assessing the dual challenges of perception and reasoning in spatial planning tasks for VLMs [102.36953558562436]
  Vision language models (VLMs) are an exciting emerging class of language models (LMs) One understudied capability inVLMs is visual spatial planning. Our study introduces a benchmark that evaluates the spatial planning capability in these models in general.
  arXiv  Detail & Related papers  (2024-07-02T00:24:01Z)
• Unifying Image Processing as Visual Prompting Question Answering [62.84955983910612]
  Image processing is a fundamental task in computer vision, which aims at enhancing image quality and extracting essential features for subsequent vision applications. Traditionally, task-specific models are developed for individual tasks and designing such models requires distinct expertise. We propose a universal model for general image processing that covers image restoration, image enhancement, image feature extraction tasks.
  arXiv  Detail & Related papers  (2023-10-16T15:32:57Z)
• InstructCV: Instruction-Tuned Text-to-Image Diffusion Models as Vision Generalists [66.85125112199898]
  We develop a unified language interface for computer vision tasks that abstracts away task-specific design choices. Our model, dubbed InstructCV, performs competitively compared to other generalist and task-specific vision models.
  arXiv  Detail & Related papers  (2023-09-30T14:26:43Z)
• Images Speak in Images: A Generalist Painter for In-Context Visual Learning [98.78475432114595]
  In-context learning allows the model to rapidly adapt to various tasks with only a handful of prompts and examples. It is unclear how to define the general-purpose task prompts that the vision model can understand and transfer to out-of-domain tasks. We present Painter, a generalist model which redefines the output of core vision tasks as images, and specify task prompts as also images.
  arXiv  Detail & Related papers  (2022-12-05T18:59:50Z)
• Task Scoping: Generating Task-Specific Abstractions for Planning [19.411900372400183]
  Planning to solve any specific task using an open-scope world model is computationally intractable. We propose task scoping: a method that exploits knowledge of the initial condition, goal condition, and transition-dynamics structure of a task. We prove that task scoping never deletes relevant factors or actions, characterize its computational complexity, and characterize the planning problems for which it is especially useful.
  arXiv  Detail & Related papers  (2020-10-17T21:19:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.