JarvisArt: Liberating Human Artistic Creativity via an Intelligent Photo Retouching Agent

• URL: http://arxiv.org/abs/2506.17612v1
• Date: Sat, 21 Jun 2025 06:36:00 GMT
• Title: JarvisArt: Liberating Human Artistic Creativity via an Intelligent Photo Retouching Agent
• Abstract summary: Photo retouching has become integral to contemporary visual storytelling, enabling users to capture aesthetics and express creativity.<n>We introduce JarvisArt, a multi-modal language model (MLLM)-driven agent that understands user intent, mimics the reasoning process of professional artists, and intelligently coordinates over 200 retouching tools within Lightroom.<n>To evaluate performance, we develop MMArt-Bench, a novel benchmark constructed from real-world user edits.<n>JarvisArt outperforms GPT-4o with a 60% improvement in average pixel-level metrics on MMArt-Bench for content fidelity, while maintaining comparable instruction-following capabilities.
• Score: 74.64342043677975
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Photo retouching has become integral to contemporary visual storytelling, enabling users to capture aesthetics and express creativity. While professional tools such as Adobe Lightroom offer powerful capabilities, they demand substantial expertise and manual effort. In contrast, existing AI-based solutions provide automation but often suffer from limited adjustability and poor generalization, failing to meet diverse and personalized editing needs. To bridge this gap, we introduce JarvisArt, a multi-modal large language model (MLLM)-driven agent that understands user intent, mimics the reasoning process of professional artists, and intelligently coordinates over 200 retouching tools within Lightroom. JarvisArt undergoes a two-stage training process: an initial Chain-of-Thought supervised fine-tuning to establish basic reasoning and tool-use skills, followed by Group Relative Policy Optimization for Retouching (GRPO-R) to further enhance its decision-making and tool proficiency. We also propose the Agent-to-Lightroom Protocol to facilitate seamless integration with Lightroom. To evaluate performance, we develop MMArt-Bench, a novel benchmark constructed from real-world user edits. JarvisArt demonstrates user-friendly interaction, superior generalization, and fine-grained control over both global and local adjustments, paving a new avenue for intelligent photo retouching. Notably, it outperforms GPT-4o with a 60% improvement in average pixel-level metrics on MMArt-Bench for content fidelity, while maintaining comparable instruction-following capabilities. Project Page: https://jarvisart.vercel.app/.

Related papers

• ToolArtist: Tool-Using Unified Multimodal Models for Agentic Image Generation [57.673228074553414]
  ToolArtist is a fully agentic image generation model obtained by post-training a Unified Multimodal Model (UMM)<n>We show that ToolArtist orchestrates reasoning, external tool use, and native image generation within one unified policy.<n> Experiments show that placing the entire open-world image-generation process under an agent policy consistently outperforms approaches with fixed pipelines.
  arXiv  Detail & Related papers  (2026-08-05T04:26:19Z)
• VeraRetouch: A Lightweight Fully Differentiable Framework for Multi-Task Reasoning Photo Retouching [27.18046653314556]
  VeraRetouch is a lightweight and fully differentiable framework for multi-task photo retouching.<n>We employ a 0.5B Vision-Language Model (VLM) as the central intelligence to formulate retouching plans.<n>We develop a fully differentiable Retouch Renderer that replaces external tools.
  arXiv  Detail & Related papers  (2026-04-30T03:39:32Z)
• SmartPhotoCrafter: Unified Reasoning, Generation and Optimization for Automatic Photographic Image Editing [29.50529252612344]
  We propose SmartPhotoCrafter, an automatic photographic image editing method which formulates image editing as a tightly coupled reasoning-to-generation process.<n>Experiments demonstrate that SmartPhotoCrafter outperforms existing generative models on the task of automatic photographic enhancement.
  arXiv  Detail & Related papers  (2026-04-21T15:38:49Z)
• ImageEdit-R1: Boosting Multi-Agent Image Editing via Reinforcement Learning [20.61925053714293]
  Image editing systems often struggle with complex, indirect, or multi-step user instructions.<n>We propose ImageEdit-R1, a multi-agent framework for intelligent image editing.<n>Our method treats image editing as a sequential decision-making problem, enabling dynamic and context-aware editing strategies.
  arXiv  Detail & Related papers  (2026-03-09T07:50:14Z)
• RetouchIQ: MLLM Agents for Instruction-Based Image Retouching with Generalist Reward [64.78078130943489]
  We introduce RetouchIQ, a framework that performs instruction-based executable image editing through MLLM agents guided by a reward model.<n>We show that RetouchIQ substantially improves both semantic consistency and perceptual quality over previous MLLM-based and diffusion-based editing systems.
  arXiv  Detail & Related papers  (2026-02-19T17:11:59Z)
• Training Multi-Image Vision Agents via End2End Reinforcement Learning [51.81337984526068]
  We propose IMAgent, an open-source vision agent trained via end-to-end reinforcement learning.<n>By leveraging a multi-agent system, we generate challenging and visually-rich multi-image QA pairs.<n>We develop two specialized tools for visual reflection and confirmation, allowing the model to proactively reallocate its attention to image content.
  arXiv  Detail & Related papers  (2025-12-05T10:02:38Z)
• JarvisEvo: Towards a Self-Evolving Photo Editing Agent with Synergistic Editor-Evaluator Optimization [39.107071937105395]
  JarvisEvo is a unified image editing agent that emulates an expert human designer by iteratively editing, selecting appropriate tools, evaluating results, and reflecting on its own decisions to refine outcomes.<n>On ArtEdit-Bench, JarvisEvo outperforms Nano-Banana by an average of 18.95% on preservative editing metrics, including a substantial 44.96% improvement in pixel-level content fidelity.
  arXiv  Detail & Related papers  (2025-11-28T09:04:51Z)
• AI Co-Artist: A LLM-Powered Framework for Interactive GLSL Shader Animation Evolution [2.7010154811483162]
  We present AI Co-Artist, a novel interactive system that harnesses the capabilities of large language models (LLMs), specifically GPT-4.<n>Our system empowers users to evolve shader art using intuitive interactions, without needing to write or understand code.
  arXiv  Detail & Related papers  (2025-11-27T18:55:32Z)
• MIRA: Empowering One-Touch AI Services on Smartphones with MLLM-based Instruction Recommendation [61.19099947706954]
  This paper introduces MIRA, a pioneering framework for task instruction recommendation.<n>With MIRA, users can long-press on images or text objects to receive contextually relevant instruction recommendations for executing AI tasks.<n> MIRA has demonstrated substantial improvements in the accuracy of instruction recommendation.
  arXiv  Detail & Related papers  (2025-09-17T07:43:14Z)
• LensCraft: Your Professional Virtual Cinematographer [12.512681517449868]
  Digital creators, from indie filmmakers to animation studios, face a persistent bottleneck: translating their creative vision into precise camera movements.<n>LensCraft solves this problem by mimicking the expertise of a professional cinematographer, using a data-driven approach.<n>LensCraft achieves markedly lower computational complexity and faster inference while maintaining high output quality.
  arXiv  Detail & Related papers  (2025-06-01T12:43:55Z)
• PhotoArtAgent: Intelligent Photo Retouching with Language Model-Based Artist Agents [28.44728600512551]
  PhotoArtAgent is an intelligent interpretative system that emulates the creative process of a professional artist.<n>PhotoArtAgent provides transparent, text-based explanations of its creative rationale, fostering meaningful interaction and user control.<n> Experimental results show that PhotoArtAgent not only surpasses existing automated tools in user studies but also achieves results comparable to those of professional human artists.
  arXiv  Detail & Related papers  (2025-05-29T06:00:51Z)
• MonetGPT: Solving Puzzles Enhances MLLMs' Image Retouching Skills [37.48977077142813]
  We show that a multimodal large language model (MLLM) can be taught to critique raw photographs.<n>We demonstrate that MLLMs can be first made aware of the underlying image processing operations.<n>We then synthesize a reasoning dataset by procedurally manipulating expert-edited photos.
  arXiv  Detail & Related papers  (2025-05-09T16:38:27Z)
• Marmot: Multi-Agent Reasoning for Multi-Object Self-Correcting in Improving Image-Text Alignment [55.74860093731475]
  Marmot is a novel framework that employs Multi-Agent Reasoning for Multi-Object Self-Correcting.<n>We construct a multi-agent self-correcting system featuring a decision-execution-verification mechanism.<n>Experiments demonstrate that Marmot significantly improves accuracy in object counting, attribute assignment, and spatial relationships.
  arXiv  Detail & Related papers  (2025-04-10T16:54:28Z)
• WorldCraft: Photo-Realistic 3D World Creation and Customization via LLM Agents [67.31920821192323]
  We introduce WorldCraft, a system where large language model (LLM) agents leverage procedural generation to create scenes populated with objects.<n>In our framework, a coordinator agent manages the overall process and works with two specialized LLM agents to complete the scene creation.<n>Our pipeline incorporates a trajectory control agent, allowing users to animate the scene and operate the camera through natural language interactions.
  arXiv  Detail & Related papers  (2025-02-21T17:18:30Z)
• INRetouch: Context Aware Implicit Neural Representation for Photography Retouching [54.17599183365242]
  We propose a novel retouch transfer approach that learns from professional edits through before-after image pairs.<n>We develop a context-aware Implicit Neural Representation that learns to apply edits adaptively based on image content and context.<n>Our method extracts implicit transformations from reference edits and adaptively applies them to new images.
  arXiv  Detail & Related papers  (2024-12-05T03:31:48Z)
• SPIRE: Semantic Prompt-Driven Image Restoration [66.26165625929747]
  We develop SPIRE, a Semantic and restoration Prompt-driven Image Restoration framework. Our approach is the first framework that supports fine-level instruction through language-based quantitative specification of the restoration strength. Our experiments demonstrate the superior restoration performance of SPIRE compared to the state of the arts.
  arXiv  Detail & Related papers  (2023-12-18T17:02:30Z)
• LightPainter: Interactive Portrait Relighting with Freehand Scribble [79.95574780974103]
  We introduce LightPainter, a scribble-based relighting system that allows users to interactively manipulate portrait lighting effect with ease. To train the relighting module, we propose a novel scribble simulation procedure to mimic real user scribbles. We demonstrate high-quality and flexible portrait lighting editing capability with both quantitative and qualitative experiments.
  arXiv  Detail & Related papers  (2023-03-22T23:17:11Z)
• NICER: Aesthetic Image Enhancement with Humans in the Loop [0.7756211500979312]
  This work proposes a neural network based approach to no-reference image enhancement in a fully-, semi-automatic or fully manual process. We show that NICER can improve image aesthetics without user interaction and that allowing user interaction leads to diverse enhancement outcomes.
  arXiv  Detail & Related papers  (2020-12-03T09:14:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.