REFINE-CONTROL: A Semi-supervised Distillation Method For Conditional Image Generation

• URL: http://arxiv.org/abs/2509.22139v1
• Date: Fri, 26 Sep 2025 09:59:40 GMT
• Title: REFINE-CONTROL: A Semi-supervised Distillation Method For Conditional Image Generation
• Authors: Yicheng Jiang, Jin Yuan, Hua Yuan, Yao Zhang, Yong Rui,
• Abstract summary: Conditional image generation models have achieved remarkable results by leveraging text-based control to generate customized images.<n>However, the high resource demands of these models and the scarcity of well-annotated data have hindered their deployment on edge devices.<n>We propose Refine-Control, a semi-supervised distillation framework to overcome these challenges.
• Score: 22.926677024175763
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Conditional image generation models have achieved remarkable results by leveraging text-based control to generate customized images. However, the high resource demands of these models and the scarcity of well-annotated data have hindered their deployment on edge devices, leading to enormous costs and privacy concerns, especially when user data is sent to a third party. To overcome these challenges, we propose Refine-Control, a semi-supervised distillation framework. Specifically, we improve the performance of the student model by introducing a tri-level knowledge fusion loss to transfer different levels of knowledge. To enhance generalization and alleviate dataset scarcity, we introduce a semi-supervised distillation method utilizing both labeled and unlabeled data. Our experiments reveal that Refine-Control achieves significant reductions in computational cost and latency, while maintaining high-fidelity generation capabilities and controllability, as quantified by comparative metrics.

Related papers

• ScaleWeaver: Weaving Efficient Controllable T2I Generation with Multi-Scale Reference Attention [86.93601565563954]
  ScaleWeaver is a framework designed to achieve high-fidelity, controllable generation upon advanced visual autoregressive( VAR) models.<n>The proposed Reference Attention module discards the unnecessary attention from image$rightarrow$condition, reducing computational cost.<n>Experiments show that ScaleWeaver delivers high-quality generation and precise control while attaining superior efficiency over diffusion-based methods.
  arXiv  Detail & Related papers  (2025-10-16T17:00:59Z)
• A Practical Investigation of Spatially-Controlled Image Generation with Transformers [16.682348277650817]
  We aim to provide clear takeaways across generation paradigms for practitioners wishing to develop systems for spatially-controlled generation.<n>We perform controlled experiments on ImageNet across diffusion-based/flow-based and autoregressive (AR) models.
  arXiv  Detail & Related papers  (2025-07-21T15:33:49Z)
• PerLDiff: Controllable Street View Synthesis Using Perspective-Layout Diffusion Models [55.080748327139176]
  We introduce PerLDiff, a novel method for effective street view image generation that fully leverages perspective 3D geometric information.<n>PerLDiff employs 3D geometric priors to guide the generation of street view images with precise object-level control within the network learning process.<n> Empirical results justify that our PerLDiff markedly enhances the precision of controllable generation on the NuScenes and KITTI datasets.
  arXiv  Detail & Related papers  (2024-07-08T16:46:47Z)
• EnTruth: Enhancing the Traceability of Unauthorized Dataset Usage in Text-to-image Diffusion Models with Minimal and Robust Alterations [73.94175015918059]
  We introduce a novel approach, EnTruth, which Enhances Traceability of unauthorized dataset usage. By strategically incorporating the template memorization, EnTruth can trigger the specific behavior in unauthorized models as the evidence of infringement. Our method is the first to investigate the positive application of memorization and use it for copyright protection, which turns a curse into a blessing.
  arXiv  Detail & Related papers  (2024-06-20T02:02:44Z)
• DetDiffusion: Synergizing Generative and Perceptive Models for Enhanced Data Generation and Perception [78.26734070960886]
  Current perceptive models heavily depend on resource-intensive datasets. We introduce perception-aware loss (P.A. loss) through segmentation, improving both quality and controllability. Our method customizes data augmentation by extracting and utilizing perception-aware attribute (P.A. Attr) during generation.
  arXiv  Detail & Related papers  (2024-03-20T04:58:03Z)
• Refining the ONCE Benchmark with Hyperparameter Tuning [45.55545585587993]
  This work focuses on the evaluation of semi-supervised learning approaches for point cloud data. Data annotation is of paramount importance in the context of LiDAR applications. We show that improvements from previous semi-supervised methods may not be as profound as previously thought.
  arXiv  Detail & Related papers  (2023-11-10T13:39:07Z)
• Generative Modeling Helps Weak Supervision (and Vice Versa) [87.62271390571837]
  We propose a model fusing weak supervision and generative adversarial networks. It captures discrete variables in the data alongside the weak supervision derived label estimate. It is the first approach to enable data augmentation through weakly supervised synthetic images and pseudolabels.
  arXiv  Detail & Related papers  (2022-03-22T20:24:21Z)
• Semi-Supervised StyleGAN for Disentanglement Learning [79.01988132442064]
  Current disentanglement methods face several inherent limitations. We design new architectures and loss functions based on StyleGAN for semi-supervised high-resolution disentanglement learning.
  arXiv  Detail & Related papers  (2020-03-06T22:54:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.