PaAgent: Portrait-Aware Image Restoration Agent via Subjective-Objective Reinforcement Learning

• URL: http://arxiv.org/abs/2603.17055v1
• Date: Tue, 17 Mar 2026 18:41:36 GMT
• Title: PaAgent: Portrait-Aware Image Restoration Agent via Subjective-Objective Reinforcement Learning
• Abstract summary: Image Restoration (IR) agents, leveraging multimodal large language models to perceive degradation and invoke restoration tools, have shown promise in automating IR tasks.<n>We propose a portrait-aware IR agent, dubbed PaAgent, which incorporates a self-evolving portrait bank for IR tools and Retrieval-Augmented Generation (RAG) to select a suitable IR tool for input.<n>To enhance PaAgent's ability to perceive degradation in complex scenes, we propose a subjective-objective reinforcement learning strategy.
• Score: 26.45794333055125
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Image Restoration (IR) agents, leveraging multimodal large language models to perceive degradation and invoke restoration tools, have shown promise in automating IR tasks. However, existing IR agents typically lack an insight summarization mechanism for past interactions, which results in an exhaustive search for the optimal IR tool. To address this limitation, we propose a portrait-aware IR agent, dubbed PaAgent, which incorporates a self-evolving portrait bank for IR tools and Retrieval-Augmented Generation (RAG) to select a suitable IR tool for input. Specifically, to construct and evolve the portrait bank, the PaAgent continuously enriches it by summarizing the characteristics of various IR tools with restored images, selected IR tools, and degraded images. In addition, the RAG is employed to select the optimal IR tool for the input image by retrieving relevant insights from the portrait bank. Furthermore, to enhance PaAgent's ability to perceive degradation in complex scenes, we propose a subjective-objective reinforcement learning strategy that considers both image quality scores and semantic insights in reward generation, which accurately provides the degradation information even under partial and non-uniform degradation. Extensive experiments across 8 IR benchmarks, covering six single-degradation and eight mixed-degradation scenarios, validate PaAgent's superiority in addressing complex IR tasks. Our project page is \href{https://wyjgr.github.io/PaAgent.html}{PaAgent}.

Related papers

• WeAgent-MMSearch: Native Text-Vision Interaction for Multimodal Search Agents [53.426679066400844]
  We introduce WeAgent-Harness, a multimodal agentic harness that supports native text-vision interaction and runtime recovery.<n>We develop WeAgent-MMSearch, an integrated system spanning data construction, agentic post-training, and multimodal rollout.<n>We also introduce VisTarget-Bench, a 150-task human-verified benchmark that pairs each question with a held-out target image.
  arXiv  Detail & Related papers  (2026-08-28T08:28:43Z)
• ToolArtist: Tool-Using Unified Multimodal Models for Agentic Image Generation [57.673228074553414]
  ToolArtist is a fully agentic image generation model obtained by post-training a Unified Multimodal Model (UMM)<n>We show that ToolArtist orchestrates reasoning, external tool use, and native image generation within one unified policy.<n> Experiments show that placing the entire open-world image-generation process under an agent policy consistently outperforms approaches with fixed pipelines.
  arXiv  Detail & Related papers  (2026-08-05T04:26:19Z)
• A Multimodal RAG Framework for Housing Damage Assessment: Collaborative Optimization of Image Encoding and Policy Vector Retrieval [0.0]
  After natural disasters, accurate evaluations of damage to housing are important for insurance claims response and planning of resources.<n>In this work, we introduce a novel multimodal retrieval-augmented generation (MM-RAG) framework.<n>The framework takes end-to-end training, and combines the comparison loss, the retrieval loss and the generation loss to form multi-task optimization objectives.
  arXiv  Detail & Related papers  (2025-09-10T01:58:07Z)
• GenIR: Generative Visual Feedback for Mental Image Retrieval [8.753622774569774]
  We study the task of Mental Image Retrieval (MIR)<n>MIR targets the realistic yet underexplored setting where users refine their search for a mentally envisioned image through multi-round interactions with an image search engine.<n>We propose GenIR, a generative multi-round retrieval paradigm leveraging diffusion-based image generation to explicitly reify the AI system's understanding at each round.
  arXiv  Detail & Related papers  (2025-06-06T16:28:03Z)
• Manifold-aware Representation Learning for Degradation-agnostic Image Restoration [135.90908995927194]
  Image Restoration (IR) aims to recover high quality images from degraded inputs affected by various corruptions such as noise, blur, haze, rain, and low light conditions.<n>We present MIRAGE, a unified framework for all in one IR that explicitly decomposes the input feature space into three semantically aligned parallel branches.<n>This modular decomposition significantly improves generalization and efficiency across diverse degradations.
  arXiv  Detail & Related papers  (2025-05-24T12:52:10Z)
• Multi-Agent Image Restoration [9.614197636859435]
  We propose MAIR, a novel Multi-Agent approach for complex IR problems.<n>Built upon a three-stage restoration framework, MAIR emulates a team of collaborative human specialists.<n>MAIR achieves competitive performance and improved efficiency over the previous agentic IR system.
  arXiv  Detail & Related papers  (2025-03-12T13:53:57Z)
• UniRestore: Unified Perceptual and Task-Oriented Image Restoration Model Using Diffusion Prior [56.35236964617809]
  Image restoration aims to recover content from inputs degraded by various factors, such as adverse weather, blur, and noise.<n>This paper introduces UniRestore, a unified image restoration model that bridges the gap between PIR and TIR.<n>We propose a Complementary Feature Restoration Module (CFRM) to reconstruct degraded encoder features and a Task Feature Adapter (TFA) module to facilitate adaptive feature fusion in the decoder.
  arXiv  Detail & Related papers  (2025-01-22T08:06:48Z)
• Hierarchical Information Flow for Generalized Efficient Image Restoration [108.83750852785582]
  We propose a hierarchical information flow mechanism for image restoration, dubbed Hi-IR.<n>Hi-IR constructs a hierarchical information tree representing the degraded image across three levels.<n>In seven common image restoration tasks, Hi-IR achieves its effectiveness and generalizability.
  arXiv  Detail & Related papers  (2024-11-27T18:30:08Z)
• An Intelligent Agentic System for Complex Image Restoration Problems [39.93819777300997]
  AgenticIR mimics the human approach to image processing by following five key stages: Perception, Scheduling, Execution, Reflection, and Rescheduling.<n>We employ large language models (LLMs) and vision-language models (VLMs) that interact via text generation to operate a toolbox of IR models.<n> Experiments demonstrate AgenticIR's potential in handling complex IR tasks, representing a promising path toward achieving general intelligence in visual processing.
  arXiv  Detail & Related papers  (2024-10-23T12:11:26Z)
• LoRA-IR: Taming Low-Rank Experts for Efficient All-in-One Image Restoration [62.3751291442432]
  We propose LoRA-IR, a flexible framework that dynamically leverages compact low-rank experts to facilitate efficient all-in-one image restoration. LoRA-IR consists of two training stages: degradation-guided pre-training and parameter-efficient fine-tuning. Experiments demonstrate that LoRA-IR achieves SOTA performance across 14 IR tasks and 29 benchmarks, while maintaining computational efficiency.
  arXiv  Detail & Related papers  (2024-10-20T13:00:24Z)
• DRM-IR: Task-Adaptive Deep Unfolding Network for All-In-One Image Restoration [5.573836220587265]
  This work proposes an efficient Dynamic Reference Modeling paradigm (DRM-IR) DRM-IR consists of task-adaptive degradation modeling and model-based image restoring. Experiments on multiple benchmark datasets show that our DRM-IR achieves state-of-the-art in All-In-One IR.
  arXiv  Detail & Related papers  (2023-07-15T02:42:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.