AIM-Bench: Benchmarking and Improving Affective Image Manipulation via Fine-Grained Hierarchical Control

• URL: http://arxiv.org/abs/2604.10454v1
• Date: Sun, 12 Apr 2026 04:32:30 GMT
• Title: AIM-Bench: Benchmarking and Improving Affective Image Manipulation via Fine-Grained Hierarchical Control
• Abstract summary: Affective Image Manipulation (AIM) aims to evoke specific emotions through targeted editing.<n>We introduce the first benchmark designed for AIM termed AIM-Bench.<n>This benchmark is built upon a dual-path affective modeling scheme that integrates the Mikels emotion taxonomy with the Valence-Arousal-Dominance framework.
• Score: 30.172679310027533
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Affective Image Manipulation (AIM) aims to evoke specific emotions through targeted editing. Current image editing benchmarks primarily focus on object-level modifications in general scenarios, lacking the fine-grained granularity to capture affective dimensions. To bridge this gap, we introduce the first benchmark designed for AIM termed AIM-Bench. This benchmark is built upon a dual-path affective modeling scheme that integrates the Mikels emotion taxonomy with the Valence-Arousal-Dominance framework, enabling high-level semantic and fine-grained continuous manipulation. Through a hierarchical human-in-the-loop workflow, we finally curate 800 high-quality samples covering 8 emotional categories and 5 editing types. To effectively assess performance, we also design a composite evaluation suite combining rule-based and model-based metrics to holistically assess instruction consistency, aesthetics, and emotional expressiveness. Extensive evaluations reveal that current editing models face significant challenges, most notably a prevalent positivity bias, which stemming from inherent imbalances in training data distribution. To tackle this, we propose a scalable data engine utilizing an inverse repainting strategy to construct AIM-40k, a balanced instruction-tuning dataset comprising 40k samples. Concretely, we enhance raw affective images via generative redrawing to establish high-fidelity ground truths, and synthesize input images with divergent emotions and paired precise instructions. Fine-tuning a baseline model on AIM-40k yields a 9.15% relative improvement in overall performance, demonstrating the effectiveness of our AIM-40k. Our data and related code will be made open soon.

Related papers

• GIFT: Goal-Injected Fine-Tuning for Efficient Manipulation Policy Adaptation [24.751598161321933]
  Goal-Injected Fine-Tuning (GIFT) is a lightweight and efficient fine-tuning framework that seamlessly integrates generated goal images into pretrained VLA models.<n>Our approach introduces goal image features into observations via a zero-d convolution which progressively grows parameters from zero.<n>As training proceeds, goal information is gradually incorporated, enabling efficient goal understanding without disrupting model stability.
  arXiv  Detail & Related papers  (2026-09-07T03:56:49Z)
• MooD: Perception-Enhanced Efficient Affective Image Editing via Continuous Valence-Arousal Modeling [6.7122126781526745]
  Affective Image Editing (AIE) aims to modify visual content to evoke targeted emotions.<n>We propose MooD, the first framework that directly leverages continuous Valence-Arousal (VA) values as editing instruction.<n>We show MooD achieves superior performance in both affective controllability and visual fidelity while maintaining high efficiency.
  arXiv  Detail & Related papers  (2026-05-04T12:22:43Z)
• AIM: Intent-Aware Unified world action Modeling with Spatial Value Maps [7.710034405765985]
  AIM is an intent-aware unified world action model that bridges this gap via an explicit spatial interface.<n>Built on a pretrained video generation model, AIM jointly models future observations and value maps within a shared mixture-of-transformers architecture.<n>Experiments on RoboTwin 2.0 benchmark show that AIM achieves a 94.0% average success rate, significantly outperforming prior unified world action baselines.
  arXiv  Detail & Related papers  (2026-04-13T07:48:58Z)
• HY-Embodied-0.5: Embodied Foundation Models for Real-World Agents [71.09493646932046]
  We introduce HY-Embodied-0.5, a family of foundation models specifically designed for real-world embodied agents.<n>The HY-Embodied-0.5 suite comprises two primary variants: an efficient model with 2B activated parameters designed for edge deployment, and a powerful model with 32B activated parameters targeted for complex reasoning.<n>Our MoT-2B model outperforms similarly sized state-of-the-art models on 16 benchmarks, while the 32B variant achieves performance comparable to frontier models such as Gemini 3.0 Pro.
  arXiv  Detail & Related papers  (2026-04-08T17:59:48Z)
• EchoGen: Generating Visual Echoes in Any Scene via Feed-Forward Subject-Driven Auto-Regressive Model [56.53617289548353]
  EchoGen is a pioneering framework that empowers Visual Auto-Regressive ( VAR) models with subject-driven generation capabilities.<n>We employ a semantic encoder to extract the subject's abstract identity, which is injected through decoupled cross-attention to guide the overall composition.<n>To the best of our knowledge, EchoGen is the first feed-forward subject-driven framework built upon VAR models.
  arXiv  Detail & Related papers  (2025-09-30T11:45:48Z)
• AIM: Amending Inherent Interpretability via Self-Supervised Masking [57.17600766859953]
  We propose "Amending Inherent Interpretability via Self-Supervised Masking" (AIM)<n>AIM promotes the network's utilization of genuine features over spurious alternatives without requiring additional annotations.<n>We validate AIM across a range of challenging datasets that test both out-of-distribution generalization and fine-grained visual understanding.
  arXiv  Detail & Related papers  (2025-08-15T14:29:59Z)
• Resilience of Vision Transformers for Domain Generalisation in the Presence of Out-of-Distribution Noisy Images [2.2124795371148616]
  We evaluate vision tramsformers pre-trained with masked image modelling (MIM) against synthetic out-of-distribution (OOD) benchmarks.<n>Experiments demonstrate BEIT's known robustness while maintaining 94% accuracy on PACS and 87% on Office-Home, despite significant occlusions.<n>These insights bridge the gap between lab-trained models and real-world deployment that offer a blueprint for building AI systems that generalise reliably under uncertainty.
  arXiv  Detail & Related papers  (2025-04-05T16:25:34Z)
• Benchmark Granularity and Model Robustness for Image-Text Retrieval [44.045767657945895]
  We show how dataset granularity and query perturbations affect retrieval performance and robustness.<n>We show that richer captions consistently enhance retrieval, especially in text-to-image tasks.<n>Our results highlight variation in model robustness and a dataset-dependent relationship between caption granularity and sensitivity perturbation.
  arXiv  Detail & Related papers  (2024-07-21T18:08:44Z)
• Model Inversion Attacks Through Target-Specific Conditional Diffusion Models [54.69008212790426]
  Model inversion attacks (MIAs) aim to reconstruct private images from a target classifier's training set, thereby raising privacy concerns in AI applications. Previous GAN-based MIAs tend to suffer from inferior generative fidelity due to GAN's inherent flaws and biased optimization within latent space. We propose Diffusion-based Model Inversion (Diff-MI) attacks to alleviate these issues.
  arXiv  Detail & Related papers  (2024-07-16T06:38:49Z)
• Scalable Pre-training of Large Autoregressive Image Models [65.824197847617]
  This paper introduces AIM, a collection of vision models pre-trained with an autoregressive objective. We highlight two key findings: (1) the performance of the visual features scale with both the model capacity and the quantity of data, and (2) the value of the objective function correlates with the performance of the model on downstream tasks.
  arXiv  Detail & Related papers  (2024-01-16T18:03:37Z)
• Semantic-Aware Autoregressive Image Modeling for Visual Representation Learning [9.960835000101115]
  We present a semantic-aware autoregressive image modeling (SemAIM) method to tackle this challenge. The key insight of SemAIM is to autoregressive model images from the semantic patches to the less semantic patches. With ViT-B, SemAIM achieves 84.1% top-1 accuracy for fine-tuning on ImageNet, 51.3% AP and 45.4% AP for object detection and instance/semantic segmentation.
  arXiv  Detail & Related papers  (2023-12-16T14:03:10Z)
• QualEval: Qualitative Evaluation for Model Improvement [82.73561470966658]
  We propose QualEval, which augments quantitative scalar metrics with automated qualitative evaluation as a vehicle for model improvement. QualEval uses a powerful LLM reasoner and our novel flexible linear programming solver to generate human-readable insights. We demonstrate that leveraging its insights, for example, improves the absolute performance of the Llama 2 model by up to 15% points relative.
  arXiv  Detail & Related papers  (2023-11-06T00:21:44Z)
• Improving Visual Grounding by Encouraging Consistent Gradient-based Explanations [58.442103936918805]
  We show that Attention Mask Consistency produces superior visual grounding results than previous methods. AMC is effective, easy to implement, and is general as it can be adopted by any vision-language model.
  arXiv  Detail & Related papers  (2022-06-30T17:55:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.