GraphicBench: A Planning Benchmark for Graphic Design with Language Agents

• URL: http://arxiv.org/abs/2504.11571v1
• Date: Tue, 15 Apr 2025 19:26:59 GMT
• Title: GraphicBench: A Planning Benchmark for Graphic Design with Language Agents
• Authors: Dayeon Ki, Tianyi Zhou, Marine Carpuat, Gang Wu, Puneet Mathur, Viswanathan Swaminathan,
• Abstract summary: GraphicBench is a new planning benchmark for graphic design that covers 1,079 user queries and input images across four types.<n>We present GraphicTown, an LLM agent framework with three design experts and 46 actions (tools) to choose from for executing each step of the planned in web environments.
• Score: 36.854520928838745
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large Language Model (LLM)-powered agents have unlocked new possibilities for automating human tasks. While prior work has focused on well-defined tasks with specified goals, the capabilities of agents in creative design tasks with open-ended goals remain underexplored. We introduce GraphicBench, a new planning benchmark for graphic design that covers 1,079 user queries and input images across four design types. We further present GraphicTown, an LLM agent framework with three design experts and 46 actions (tools) to choose from for executing each step of the planned workflows in web environments. Experiments with six LLMs demonstrate their ability to generate workflows that integrate both explicit design constraints from user queries and implicit commonsense constraints. However, these workflows often do not lead to successful execution outcomes, primarily due to challenges in: (1) reasoning about spatial relationships, (2) coordinating global dependencies across experts, and (3) retrieving the most appropriate action per step. We envision GraphicBench as a challenging yet valuable testbed for advancing LLM-agent planning and execution in creative design tasks.

Related papers

• BIMgent: Towards Autonomous Building Modeling via Computer-use Agents [0.7499722271664147]
  We propose BIMgent, an agentic framework powered by multimodal large language models (LLMs)<n>We evaluate BIMgent on real-world building modeling tasks, including both text-based conceptual design generation and reconstruction from existing building design.<n>Results demonstrate that BIMgent effectively reduces manual workload while preserving design intent, highlighting its potential for practical deployment in real-world architectural modeling scenarios.
  arXiv  Detail & Related papers  (2025-06-08T16:45:31Z)
• Plan-over-Graph: Towards Parallelable LLM Agent Schedule [53.834646147919436]
  Large Language Models (LLMs) have demonstrated exceptional abilities in reasoning for task planning.<n>This paper introduces a novel paradigm, plan-over-graph, in which the model first decomposes a real-life textual task into executable subtasks and constructs an abstract task graph.<n>The model then understands this task graph as input and generates a plan for parallel execution.
  arXiv  Detail & Related papers  (2025-02-20T13:47:51Z)
• VeriGraph: Scene Graphs for Execution Verifiable Robot Planning [33.8868315479384]
  We propose VeriGraph, a framework that integrates vision-language models (VLMs) for robotic planning while verifying action feasibility. VeriGraph employs scene graphs as an intermediate representation, capturing key objects and spatial relationships to improve plan verification and refinement. Our approach significantly enhances task completion rates across diverse manipulation scenarios, outperforming baseline methods by 58% for language-based tasks and 30% for image-based tasks.
  arXiv  Detail & Related papers  (2024-11-15T18:59:51Z)
• Benchmarking Agentic Workflow Generation [80.74757493266057]
  We introduce WorfBench, a unified workflow generation benchmark with multi-faceted scenarios and intricate graph workflow structures.<n>We also present WorfEval, a systemic evaluation protocol utilizing subsequence and subgraph matching algorithms.<n>We observe that the generated can enhance downstream tasks, enabling them to achieve superior performance with less time during inference.
  arXiv  Detail & Related papers  (2024-10-10T12:41:19Z)
• Automatic Layout Planning for Visually-Rich Documents with Instruction-Following Models [81.6240188672294]
  In graphic design, non-professional users often struggle to create visually appealing layouts due to limited skills and resources. We introduce a novel multimodal instruction-following framework for layout planning, allowing users to easily arrange visual elements into tailored layouts. Our method not only simplifies the design process for non-professionals but also surpasses the performance of few-shot GPT-4V models, with mIoU higher by 12% on Crello.
  arXiv  Detail & Related papers  (2024-04-23T17:58:33Z)
• Embodied Task Planning with Large Language Models [86.63533340293361]
  We propose a TAsk Planing Agent (TaPA) in embodied tasks for grounded planning with physical scene constraint. During inference, we discover the objects in the scene by extending open-vocabulary object detectors to multi-view RGB images collected in different achievable locations. Experimental results show that the generated plan from our TaPA framework can achieve higher success rate than LLaVA and GPT-3.5 by a sizable margin.
  arXiv  Detail & Related papers  (2023-07-04T17:58:25Z)
• Egocentric Planning for Scalable Embodied Task Achievement [6.870094263016224]
  Egocentric Planning is an innovative approach that combines symbolic planning and Object-oriented POMDPs to solve tasks in complex environments. We evaluated our approach in ALFRED, a simulated environment designed for domestic tasks, and demonstrated its high scalability. Our method requires reliable perception and the specification or learning of a symbolic description of the preconditions and effects of the agent's actions.
  arXiv  Detail & Related papers  (2023-06-02T06:41:24Z)
• Efficient Automatic Machine Learning via Design Graphs [72.85976749396745]
  We propose FALCON, an efficient sample-based method to search for the optimal model design. FALCON features 1) a task-agnostic module, which performs message passing on the design graph via a Graph Neural Network (GNN), and 2) a task-specific module, which conducts label propagation of the known model performance information. We empirically show that FALCON can efficiently obtain the well-performing designs for each task using only 30 explored nodes.
  arXiv  Detail & Related papers  (2022-10-21T21:25:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.