SSL-R1: Self-Supervised Visual Reinforcement Post-Training for Multimodal Large Language Models

• URL: http://arxiv.org/abs/2604.20705v1
• Date: Wed, 22 Apr 2026 15:46:42 GMT
• Title: SSL-R1: Self-Supervised Visual Reinforcement Post-Training for Multimodal Large Language Models
• Abstract summary: SSL-R1 is a generic self-supervised RL framework that derives verifiable rewards directly from images.<n>We reformulate widely-used SSL tasks into a set of verifiable visual puzzles for RL post-training, requiring neither human nor external model supervision.
• Score: 88.62912181680413
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reinforcement learning (RL) with verifiable rewards (RLVR) has demonstrated the great potential of enhancing the reasoning abilities in multimodal large language models (MLLMs). However, the reliance on language-centric priors and expensive manual annotations prevents MLLMs' intrinsic visual understanding and scalable reward designs. In this work, we introduce SSL-R1, a generic self-supervised RL framework that derives verifiable rewards directly from images. To this end, we revisit self-supervised learning (SSL) in visual domains and reformulate widely-used SSL tasks into a set of verifiable visual puzzles for RL post-training, requiring neither human nor external model supervision. Training MLLMs on these tasks substantially improves their performance on multimodal understanding and reasoning benchmarks, highlighting the potential of leveraging vision-centric self-supervised tasks for MLLM post-training. We think this work will provide useful experience in devising effective self-supervised verifiable rewards to enable RL at scale. Project page: https://github.com/Jiahao000/SSL-R1.

Related papers

• Learn to Think: Improving Multimodal Reasoning through Vision-Aware Self-Improvement Training [82.17582358979884]
  Post-training with explicit reasoning traces is common to improve the reasoning capabilities of Multimodal Large Language Models (MLLMs)<n>We propose VISTA, a vision-aware self-improvement training framework for enhancing the multimodal reasoning of MLLMs.
  arXiv  Detail & Related papers  (2026-05-12T10:44:35Z)
• Seeing Beyond Words: Self-Supervised Visual Learning for Multimodal Large Language Models [53.06230963851451]
  JARVIS is a JEPA-inspired framework for self-supervised visual enhancement in MLLMs.<n>We introduce JARVIS, a JEPA-inspired framework for self-supervised visual enhancement in MLLMs.
  arXiv  Detail & Related papers  (2025-12-17T19:01:34Z)
• SSL4RL: Revisiting Self-supervised Learning as Intrinsic Reward for Visual-Language Reasoning [88.9014727048442]
  SSL4RL is a novel framework that leverages self-supervised learning tasks as a source of verifiable rewards for RL-based fine-tuning.<n>Our approach reformulates SSL objectives-such as predicting image rotation or reconstructing masked patches-into dense, automatic reward signals.<n>Experiments show that SSL4RL substantially improves performance on both vision-centric and vision-language reasoning benchmarks.
  arXiv  Detail & Related papers  (2025-10-18T09:22:40Z)
• Perception-R1: Advancing Multimodal Reasoning Capabilities of MLLMs via Visual Perception Reward [77.34936657745578]
  We propose Perception-R1, which introduces a novel visual perception reward that explicitly encourages MLLMs to perceive the visual content accurately.<n>We show that Perception-R1 achieves state-of-the-art performance on most benchmarks using only 1,442 training data.
  arXiv  Detail & Related papers  (2025-06-08T16:48:42Z)
• VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Model [29.524164786422368]
  Recently, DeepSeek R1 has shown that reinforcement learning can substantially improve the reasoning capabilities of Large Language Models (LLMs)<n>We investigate the extension of R1-style reinforcement learning to Vision-Language Models (VLMs)<n>We develop VLM-R1, a dedicated framework designed to harness RL for improving VLMs' performance on general vision-language tasks.
  arXiv  Detail & Related papers  (2025-04-10T10:05:15Z)
• Satori: Reinforcement Learning with Chain-of-Action-Thought Enhances LLM Reasoning via Autoregressive Search [57.28671084993782]
  Large language models (LLMs) have demonstrated remarkable reasoning capabilities across diverse domains.<n>Recent studies have shown that increasing test-time computation enhances LLMs' reasoning capabilities.<n>We propose a two-stage training paradigm: 1) a small-scale format tuning stage to internalize the COAT reasoning format and 2) a large-scale self-improvement stage leveraging reinforcement learning.
  arXiv  Detail & Related papers  (2025-02-04T17:26:58Z)
• FuRL: Visual-Language Models as Fuzzy Rewards for Reinforcement Learning [18.60627708199452]
  We investigate how to leverage pre-trained visual-language models (VLM) for online Reinforcement Learning (RL) We first identify the problem of reward misalignment when applying VLM as a reward in RL tasks. We introduce a lightweight fine-tuning method, named Fuzzy VLM reward-aided RL (FuRL)
  arXiv  Detail & Related papers  (2024-06-02T07:20:08Z)
• RLSF: Fine-tuning LLMs via Symbolic Feedback [11.407319705797242]
  Large Language Models (LLMs) have transformed AI but often struggle with tasks that require domain-specific reasoning and logical alignment.<n>Traditional fine-tuning methods do not leverage the vast amount of symbolic domain-knowledge available to us.<n>We introduce Reinforcement Learning via Symbolic Feedback (RLSF), a novel fine-tuning paradigm.
  arXiv  Detail & Related papers  (2024-05-26T18:49:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.