Rethinking cluster-conditioned diffusion models

• URL: http://arxiv.org/abs/2403.00570v1
• Date: Fri, 1 Mar 2024 14:47:46 GMT
• Title: Rethinking cluster-conditioned diffusion models
• Authors: Nikolas Adaloglou and Tim Kaiser and Felix Michels and Markus Kollmann
• Abstract summary: We elucidate how individual components regarding image clustering impact image synthesis across three datasets. We show that, given the optimal cluster granularity with respect to image synthesis (visual groups), cluster-conditioning can achieve state-of-the-art FID. We propose a novel method to derive an upper cluster bound that reduces the search space of the visual groups using solely feature-based clustering.
• Score: 1.597617022056624
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present a comprehensive experimental study on image-level conditioning for diffusion models using cluster assignments. We elucidate how individual components regarding image clustering impact image synthesis across three datasets. By combining recent advancements from image clustering and diffusion models, we show that, given the optimal cluster granularity with respect to image synthesis (visual groups), cluster-conditioning can achieve state-of-the-art FID (i.e. 1.67, 2.17 on CIFAR10 and CIFAR100 respectively), while attaining a strong training sample efficiency. Finally, we propose a novel method to derive an upper cluster bound that reduces the search space of the visual groups using solely feature-based clustering. Unlike existing approaches, we find no significant connection between clustering and cluster-conditional image generation. The code and cluster assignments will be released.

Related papers

• Self-Enhanced Image Clustering with Cross-Modal Semantic Consistency [57.961869351897384]
  We propose a framework based on cross-modal semantic consistency for efficient image clustering.<n>Our framework first builds a strong foundation via Cross-Modal Semantic Consistency.<n>In the first stage, we train lightweight clustering heads to align with the rich semantics of the pre-trained model.<n>In the second stage, we introduce a Self-Enhanced fine-tuning strategy.
  arXiv  Detail & Related papers  (2025-08-02T08:12:57Z)
• Hierarchical Clustering for Conditional Diffusion in Image Generation [12.618079575423868]
  This paper introduces TreeDiffusion, a deep generative model that conditions Diffusion Models on hierarchical clusters to obtain high-quality, cluster-specific generations. The proposed pipeline consists of two steps: a VAE-based clustering model that learns the hierarchical structure of the data, and a conditional diffusion model that generates realistic images for each cluster.
  arXiv  Detail & Related papers  (2024-10-22T11:35:36Z)
• Image Clustering Algorithm Based on Self-Supervised Pretrained Models and Latent Feature Distribution Optimization [4.39139858370436]
  This paper introduces an image clustering algorithm based on self-supervised pretrained models and latent feature distribution optimization. Our approach outperforms the latest clustering algorithms and achieves state-of-the-art clustering results.
  arXiv  Detail & Related papers  (2024-08-04T04:08:21Z)
• A consensus-constrained parsimonious Gaussian mixture model for clustering hyperspectral images [0.0]
  Food engineers use hyperspectral images to classify the type and quality of a food sample. In order to train these methods, every pixel in each training image needs to be labelled. A consensus-constrained parsimonious Gaussian mixture model (ccPGMM) is proposed to label pixels in hyperspectral images.
  arXiv  Detail & Related papers  (2024-03-05T22:23:43Z)
• ClusterFormer: Clustering As A Universal Visual Learner [80.79669078819562]
  CLUSTERFORMER is a universal vision model based on the CLUSTERing paradigm with TransFORMER. It is capable of tackling heterogeneous vision tasks with varying levels of clustering granularity. For its efficacy, we hope our work can catalyze a paradigm shift in universal models in computer vision.
  arXiv  Detail & Related papers  (2023-09-22T22:12:30Z)
• C3: Cross-instance guided Contrastive Clustering [8.953252452851862]
  Clustering is the task of gathering similar data samples into clusters without using any predefined labels. We propose a novel contrastive clustering method, Cross-instance guided Contrastive Clustering (C3) Our proposed method can outperform state-of-the-art algorithms on benchmark computer vision datasets.
  arXiv  Detail & Related papers  (2022-11-14T06:28:07Z)
• On Mitigating Hard Clusters for Face Clustering [48.39472979642971]
  Face clustering is a promising way to scale up face recognition systems using large-scale unlabeled face images. We introduce two novel modules, Neighborhood-Diffusion-based Density (NDDe) and Transition-Probability-based Distance (TPDi) Our experiments on multiple benchmarks show that each module contributes to the final performance of our method.
  arXiv  Detail & Related papers  (2022-07-25T03:55:15Z)
• Multi-Facet Clustering Variational Autoencoders [9.150555507030083]
  High-dimensional data, such as images, typically feature multiple interesting characteristics one could cluster over. We introduce Multi-Facet Clustering Variational Autoencoders (MFCVAE) MFCVAE learns multiple clusterings simultaneously, and is trained fully unsupervised and end-to-end.
  arXiv  Detail & Related papers  (2021-06-09T17:36:38Z)
• Graph Contrastive Clustering [131.67881457114316]
  We propose a novel graph contrastive learning framework, which is then applied to the clustering task and we come up with the Graph Constrastive Clustering(GCC) method. Specifically, on the one hand, the graph Laplacian based contrastive loss is proposed to learn more discriminative and clustering-friendly features. On the other hand, a novel graph-based contrastive learning strategy is proposed to learn more compact clustering assignments.
  arXiv  Detail & Related papers  (2021-04-03T15:32:49Z)
• Scalable Hierarchical Agglomerative Clustering [65.66407726145619]
  Existing scalable hierarchical clustering methods sacrifice quality for speed. We present a scalable, agglomerative method for hierarchical clustering that does not sacrifice quality and scales to billions of data points.
  arXiv  Detail & Related papers  (2020-10-22T15:58:35Z)
• Contrastive Clustering [57.71729650297379]
  We propose Contrastive Clustering (CC) which explicitly performs the instance- and cluster-level contrastive learning. In particular, CC achieves an NMI of 0.705 (0.431) on the CIFAR-10 (CIFAR-100) dataset, which is an up to 19% (39%) performance improvement compared with the best baseline.
  arXiv  Detail & Related papers  (2020-09-21T08:54:40Z)
• LSD-C: Linearly Separable Deep Clusters [145.89790963544314]
  We present LSD-C, a novel method to identify clusters in an unlabeled dataset. Our method draws inspiration from recent semi-supervised learning practice and proposes to combine our clustering algorithm with self-supervised pretraining and strong data augmentation. We show that our approach significantly outperforms competitors on popular public image benchmarks including CIFAR 10/100, STL 10 and MNIST, as well as the document classification dataset Reuters 10K.
  arXiv  Detail & Related papers  (2020-06-17T17:58:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.