RePIC: Reinforced Post-Training for Personalizing Multi-Modal Language Models

• URL: http://arxiv.org/abs/2506.18369v1
• Date: Mon, 23 Jun 2025 07:55:52 GMT
• Title: RePIC: Reinforced Post-Training for Personalizing Multi-Modal Language Models
• Authors: Yeongtak Oh, Jisoo Mok, Dohyun Chung, Juhyeon Shin, Sangha Park, Johan Barthelemy, Sungroh Yoon,
• Abstract summary: We propose a reinforcement learning-based post-training framework for personalized image captioning.<n>Our method significantly enhances both visual recognition and personalized generation capabilities of MLLMs.
• Score: 29.471762181109018
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent multi-modal large language models (MLLMs) often struggle to generate personalized image captions, even when trained on high-quality captions. In this work, we observe that such limitations persist in existing post-training-based MLLM personalization methods. Specifically, despite being post-tuned with large-scale caption data through supervised fine-tuning (SFT), these models frequently fail to produce faithful descriptions in real-world scenarios, such as multi-concept image captioning. However, acquiring large-scale, high-quality captions for such complex settings is both costly and difficult. To address the data-centric nature of SFT, we propose a reinforcement learning (RL)-based post-training framework. To the best of our knowledge, this is the first RL-based approach to post-train MLLMs for personalized image captioning. Our method significantly enhances both visual recognition and personalized generation capabilities of MLLMs, and consistently outperforms existing SFT-based baselines, especially in the challenging multi-concept image captioning task.

Related papers

• Generative Multimodal Pretraining with Discrete Diffusion Timestep Tokens [66.02261367232256]
  Multimodal Large Language Models (MLLMs) aim to unify visual comprehension and generation.<n>Existing approaches rely on spatial visual tokens, where image patches are encoded and arranged according to a spatial order.<n>In this paper, we build a proper visual language by reconstructing diffusion timesteps to learn discrete visual tokens.
  arXiv  Detail & Related papers  (2025-04-20T16:14:28Z)
• ViLa-MIL: Dual-scale Vision-Language Multiple Instance Learning for Whole Slide Image Classification [52.405499816861635]
  Multiple instance learning (MIL)-based framework has become the mainstream for processing the whole slide image (WSI)<n>We propose a dual-scale vision-language multiple instance learning (ViLa-MIL) framework for whole slide image classification.
  arXiv  Detail & Related papers  (2025-02-12T13:28:46Z)
• Multi-LLM Collaborative Caption Generation in Scientific Documents [30.856381292477177]
  We introduce a framework called Multi-LLM Collaborative Figure Caption Generation (MLBCAP)<n>Our approach unfolds in three key modules.<n>Human evaluations demonstrate that informative captions produced by our approach rank better than human-written captions.
  arXiv  Detail & Related papers  (2025-01-05T14:09:12Z)
• Personalizing Multimodal Large Language Models for Image Captioning: An Experimental Analysis [44.008094698200026]
  This paper investigates whether Multimodal LLMs can supplant traditional image captioning networks by evaluating their performance on various image description benchmarks.<n>We explore both the zero-shot capabilities of these models and their adaptability to different semantic domains through fine-tuning methods.<n>Our results demonstrate that while Multimodal LLMs achieve impressive zero-shot performance, fine-tuning for specific domains while maintaining their generalization capabilities intact remains challenging.
  arXiv  Detail & Related papers  (2024-12-04T19:01:06Z)
• LLM2CLIP: Powerful Language Model Unlocks Richer Visual Representation [72.02635550088546]
  This work explores how large language models (LLMs) can enhance CLIP's capability, especially for processing longer and more complex image captions.<n>We introduce a caption-to-caption contrastive fine-tuning framework, significantly enhancing the discriminative quality of LLM outputs.<n>Our approach outperforms LoRA-based methods, achieving nearly fourfold faster training with superior performance.
  arXiv  Detail & Related papers  (2024-11-07T18:59:16Z)
• TWIST & SCOUT: Grounding Multimodal LLM-Experts by Forget-Free Tuning [54.033346088090674]
  We introduce TWIST & SCOUT, a framework that equips pre-trained MLLMs with visual grounding ability.<n>To fine-tune the model effectively, we generate a high-quality synthetic dataset we call SCOUT.<n>This dataset provides rich supervision signals, describing a step-by-step multimodal reasoning process.
  arXiv  Detail & Related papers  (2024-10-14T13:35:47Z)
• Browse and Concentrate: Comprehending Multimodal Content via prior-LLM Context Fusion [70.9767518332692]
  Multimodal Large Language Models (MLLMs) that incorporate LLMs with pre-trained vision models have recently demonstrated impressive performance across diverse vision-language tasks. However, they fall short to comprehend context involving multiple images. We propose a two phase paradigm, browse-and-concentrate, to enable in-depth multimodal context fusion.
  arXiv  Detail & Related papers  (2024-02-19T14:59:07Z)
• MLLMs-Augmented Visual-Language Representation Learning [70.5293060238008]
  We demonstrate that Multi-modal Large Language Models (MLLMs) can enhance visual-language representation learning. Our approach is simple, utilizing MLLMs to extend multiple diverse captions for each image. We propose "text shearing" to maintain the quality and availability of extended captions.
  arXiv  Detail & Related papers  (2023-11-30T18:05:52Z)
• Linear Alignment of Vision-language Models for Image Captioning [8.921774238325566]
  We propose a lightweight captioning method called ReCap, which can be trained up to 1000 times faster than existing lightweight methods.<n>We also propose two new learning-based image-captioning metrics built on CLIP score along with our proposed alignment.
  arXiv  Detail & Related papers  (2023-07-10T17:59:21Z)
• Fusion Models for Improved Visual Captioning [18.016295296424413]
  This paper proposes a generic multimodal model fusion framework for caption generation and emendation. We employ the same fusion strategies to integrate a pretrained Masked Language Model (MLM) with a visual captioning model, viz. Show, Attend, and Tell. Our caption emendation experiments on three benchmark image captioning datasets, viz. Flickr8k, Flickr30k, and MSCOCO, show improvements over the baseline.
  arXiv  Detail & Related papers  (2020-10-28T21:55:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.