C^2GR: Coupled Comprehensive Generative Replay for a Continually Learnable Universal Segmentation Model

• URL: http://arxiv.org/abs/2606.23473v1
• Date: Mon, 22 Jun 2026 15:19:03 GMT
• Title: C^2GR: Coupled Comprehensive Generative Replay for a Continually Learnable Universal Segmentation Model
• Abstract summary: Universal segmentation models exhibit significant potential for diverse tasks involving different imaging modalities and segmentation objectives.<n> Task-Incremental Learning provides a privacy-preserving approach to continually evolve a universal model on tasks from sequentially-arriving medical departments.<n>We propose a novel Coupled Comprehensive Generative (C2GR) framework that simultaneously synthesizes image-mask pairs of previous tasks.
• Score: 17.749377881392828
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Universal segmentation models exhibit significant potential for diverse tasks involving different imaging modalities and segmentation objectives. Task-Incremental Learning provides a privacy-preserving approach to continually evolve a universal model on tasks from sequentially-arriving medical departments. However, training the model solely on the incoming task induces forgetting on past tasks, since consecutive tasks exhibit concurrent shifts in image appearance and segmentation objective. To address this problem, we propose a novel Coupled Comprehensive Generative Replay (C^2GR) framework that simultaneously synthesizes image-mask pairs of previous tasks to mitigate forgetting under concurrent appearance and objective shifts. This requires preserving image-mask correspondence for structure-realistic generation and bridging asynchronous optimization of the generator and segmentor for segmentation-oriented generation. Specifically, we propose a Bayesian Joint Diffusion (BJD) method that formulates the correspondence as conditional distributions optimized via conditional denoising. Furthermore, we develop a Relation-aware Unified Prompt Synchronization (RUPS) scheme to simultaneously modulate the generator and segmentor via a shared task-relation-aware prompt for synchronizing their optimization. Experiments on 20 tasks spanning diverse modalities and objectives demonstrate that C^2GR exhibits only a 2.44% drop in overall performance compared to joint training with all task data, effectively alleviating forgetting from the concurrent shifts. Our code will be made publicly available at https://github.com/mar-cry/C2GR.

Related papers

• Uni-Synergy: Bridging Understanding and Generation for Personalized Reasoning via Co-operative Reinforcement Learning [56.99010101756807]
  We propose Sync-R1, an end-to-end reinforcement learning framework that jointly optimize personalized understanding and generation.<n> Sync-R1 enables personalized comprehension to guide content creation, while the resulting generation quality reciprocally refines understanding.<n>We also introduce Dynamic Group Scaling (DGS), which adaptively filters low-potential trajectories to reduce gradient variance and accelerate convergence.<n> Experimental results demonstrate that Sync-R1 achieves state-of-the-art performance, showcasing superior cross-task reasoning and robust personalization.
  arXiv  Detail & Related papers  (2026-05-11T12:18:26Z)
• GenMask: Adapting DiT for Segmentation via Direct Mask Generation [81.54526445834294]
  We argue that instead of indirect adaptation, segmentation tasks should be trained directly in a generative manner.<n>We introduce timesteps sampling strategy for binary masks that emphasizes extreme noise levels for segmentation and moderate noise for image generation.<n>We present GenMask, a DiT trains to generate black-and-white segmentation masks as well as colorful images in RGB space under the original generative objective.
  arXiv  Detail & Related papers  (2026-03-25T03:52:05Z)
• Co-GRPO: Co-Optimized Group Relative Policy Optimization for Masked Diffusion Model [74.99242687133408]
  Masked Diffusion Models (MDMs) have shown promising potential across vision, language, and cross-modal generation.<n>We introduce Co-GRPO, which reformulates MDM generation as a unified Markov Decision Process (MDP) that jointly incorporates both the model and the inference schedule.
  arXiv  Detail & Related papers  (2025-12-25T12:06:04Z)
• Comprehensive Generative Replay for Task-Incremental Segmentation with Concurrent Appearance and Semantic Forgetting [49.87694319431288]
  Generalist segmentation models are increasingly favored for diverse tasks involving various objects from different image sources. We propose a Comprehensive Generative (CGR) framework that restores appearance and semantic knowledge by synthesizing image-mask pairs. Experiments on incremental tasks (cardiac, fundus and prostate segmentation) show its clear advantage for alleviating concurrent appearance and semantic forgetting.
  arXiv  Detail & Related papers  (2024-06-28T10:05:58Z)
• Towards a Unified Transformer-based Framework for Scene Graph Generation and Human-object Interaction Detection [116.21529970404653]
  We introduce SG2HOI+, a unified one-step model based on the Transformer architecture. Our approach employs two interactive hierarchical Transformers to seamlessly unify the tasks of SGG and HOI detection. Our approach achieves competitive performance when compared to state-of-the-art HOI methods.
  arXiv  Detail & Related papers  (2023-11-03T07:25:57Z)
• Unified Segment-to-Segment Framework for Simultaneous Sequence Generation [38.27500308373048]
  We propose a unified segment-to-segment framework (Seg2Seg) for simultaneous sequence generation. Seg2Seg learns the mapping in an adaptive and unified manner. Experiments on multiple simultaneous generation tasks demonstrate that Seg2Seg achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2023-10-27T07:34:51Z)
• RGM: A Robust Generalizable Matching Model [49.60975442871967]
  We propose a deep model for sparse and dense matching, termed RGM (Robust Generalist Matching) To narrow the gap between synthetic training samples and real-world scenarios, we build a new, large-scale dataset with sparse correspondence ground truth. We are able to mix up various dense and sparse matching datasets, significantly improving the training diversity.
  arXiv  Detail & Related papers  (2023-10-18T07:30:08Z)
• UniSeg: A Prompt-driven Universal Segmentation Model as well as A Strong Representation Learner [32.698493660851035]
  We propose a prompt-driven universal model (UniSeg) for multi-task medical image segmentation. We make the model 'aware' of the ongoing task early and boost the task-specific training of the whole decoder. Our results indicate that the proposed UniSeg outperforms other universal models and single-task models on 11 upstream tasks.
  arXiv  Detail & Related papers  (2023-04-07T06:28:51Z)
• Cross-modal tumor segmentation using generative blending augmentation and self training [1.6440045168835438]
  We propose a cross-modal segmentation method based on conventional image synthesis boosted by a new data augmentation technique. Generative Blending Augmentation (GBA) learns representative generative features from a single training image to realistically diversify tumor appearances. The proposed solution ranked first for vestibular schwannoma (VS) segmentation during the validation and test phases of the MICCAI CrossMoDA 2022 challenge.
  arXiv  Detail & Related papers  (2023-04-04T11:01:46Z)
• CoADNet: Collaborative Aggregation-and-Distribution Networks for Co-Salient Object Detection [91.91911418421086]
  Co-Salient Object Detection (CoSOD) aims at discovering salient objects that repeatedly appear in a given query group containing two or more relevant images. One challenging issue is how to effectively capture co-saliency cues by modeling and exploiting inter-image relationships. We present an end-to-end collaborative aggregation-and-distribution network (CoADNet) to capture both salient and repetitive visual patterns from multiple images.
  arXiv  Detail & Related papers  (2020-11-10T04:28:11Z)
• Generalized Adversarially Learned Inference [42.40405470084505]
  We develop methods of inference of latent variables in GANs by adversarially training an image generator along with an encoder to match two joint distributions of image and latent vector pairs. We incorporate multiple layers of feedback on reconstructions, self-supervision, and other forms of supervision based on prior or learned knowledge about the desired solutions.
  arXiv  Detail & Related papers  (2020-06-15T02:18:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.