CoC-GAN: Employing Context Cluster for Unveiling a New Pathway in Image Generation

• URL: http://arxiv.org/abs/2308.11857v1
• Date: Wed, 23 Aug 2023 01:19:58 GMT
• Title: CoC-GAN: Employing Context Cluster for Unveiling a New Pathway in Image Generation
• Authors: Zihao Wang, Yiming Huang, Ziyu Zhou
• Abstract summary: We propose a unique image generation process premised on the perspective of converting images into a set of point clouds. Our methodology leverages simple clustering methods named Context Clustering (CoC) to generate images from unordered point sets. We introduce this model with the novel structure as the Context Clustering Generative Adversarial Network (CoC-GAN)
• Score: 12.211795836214112
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Image generation tasks are traditionally undertaken using Convolutional Neural Networks (CNN) or Transformer architectures for feature aggregating and dispatching. Despite the frequent application of convolution and attention structures, these structures are not fundamentally required to solve the problem of instability and the lack of interpretability in image generation. In this paper, we propose a unique image generation process premised on the perspective of converting images into a set of point clouds. In other words, we interpret an image as a set of points. As such, our methodology leverages simple clustering methods named Context Clustering (CoC) to generate images from unordered point sets, which defies the convention of using convolution or attention mechanisms. Hence, we exclusively depend on this clustering technique, combined with the multi-layer perceptron (MLP) in a generative model. Furthermore, we implement the integration of a module termed the 'Point Increaser' for the model. This module is just an MLP tasked with generating additional points for clustering, which are subsequently integrated within the paradigm of the Generative Adversarial Network (GAN). We introduce this model with the novel structure as the Context Clustering Generative Adversarial Network (CoC-GAN), which offers a distinctive viewpoint in the domain of feature aggregating and dispatching. Empirical evaluations affirm that our CoC-GAN, devoid of convolution and attention mechanisms, exhibits outstanding performance. Its interpretability, endowed by the CoC module, also allows for visualization in our experiments. The promising results underscore the feasibility of our method and thus warrant future investigations of applying Context Clustering to more novel and interpretable image generation.

Related papers

• Neural Clustering based Visual Representation Learning [61.72646814537163]
  Clustering is one of the most classic approaches in machine learning and data analysis. We propose feature extraction with clustering (FEC), which views feature extraction as a process of selecting representatives from data. FEC alternates between grouping pixels into individual clusters to abstract representatives and updating the deep features of pixels with current representatives.
  arXiv  Detail & Related papers  (2024-03-26T06:04:50Z)
• Grid Jigsaw Representation with CLIP: A New Perspective on Image Clustering [37.15595383168132]
  Jigsaw based strategy method for image clustering called Grid Jigsaw Representation (GJR) with systematic exposition from pixel to feature in discrepancy against human and computer. GJR modules are appended to a variety of deep convolutional networks and tested with significant improvements on a wide range of benchmark datasets. Experiment results show the effectiveness on the clustering task with respect to the ACC, NMI and ARI three metrics and super fast convergence speed.
  arXiv  Detail & Related papers  (2023-10-27T03:07:05Z)
• Image Clustering with External Guidance [33.664812922814754]
  The core of clustering is incorporating prior knowledge to construct supervision signals. We propose leveraging external knowledge as a new supervision signal to guide clustering, even though it seems irrelevant to the given data.
  arXiv  Detail & Related papers  (2023-10-18T14:20:55Z)
• Distance Weighted Trans Network for Image Completion [52.318730994423106]
  We propose a new architecture that relies on Distance-based Weighted Transformer (DWT) to better understand the relationships between an image's components. CNNs are used to augment the local texture information of coarse priors. DWT blocks are used to recover certain coarse textures and coherent visual structures.
  arXiv  Detail & Related papers  (2023-10-11T12:46:11Z)
• ClusterFormer: Clustering As A Universal Visual Learner [80.79669078819562]
  CLUSTERFORMER is a universal vision model based on the CLUSTERing paradigm with TransFORMER. It is capable of tackling heterogeneous vision tasks with varying levels of clustering granularity. For its efficacy, we hope our work can catalyze a paradigm shift in universal models in computer vision.
  arXiv  Detail & Related papers  (2023-09-22T22:12:30Z)
• Image as Set of Points [60.30495338399321]
  Context clusters (CoCs) view an image as a set of unorganized points and extract features via simplified clustering algorithm. Our CoCs are convolution- and attention-free, and only rely on clustering algorithm for spatial interaction.
  arXiv  Detail & Related papers  (2023-03-02T18:56:39Z)
• DeepCut: Unsupervised Segmentation using Graph Neural Networks Clustering [6.447863458841379]
  This study introduces a lightweight Graph Neural Network (GNN) to replace classical clustering methods. Unlike existing methods, our GNN takes both the pair-wise affinities between local image features and the raw features as input. We demonstrate how classical clustering objectives can be formulated as self-supervised loss functions for training an image segmentation GNN.
  arXiv  Detail & Related papers  (2022-12-12T12:31:46Z)
• Deep Image Clustering with Contrastive Learning and Multi-scale Graph Convolutional Networks [58.868899595936476]
  This paper presents a new deep clustering approach termed image clustering with contrastive learning and multi-scale graph convolutional networks (IcicleGCN) Experiments on multiple image datasets demonstrate the superior clustering performance of IcicleGCN over the state-of-the-art.
  arXiv  Detail & Related papers  (2022-07-14T19:16:56Z)
• MOGAN: Morphologic-structure-aware Generative Learning from a Single Image [59.59698650663925]
  Recently proposed generative models complete training based on only one image. We introduce a MOrphologic-structure-aware Generative Adversarial Network named MOGAN that produces random samples with diverse appearances. Our approach focuses on internal features including the maintenance of rational structures and variation on appearance.
  arXiv  Detail & Related papers  (2021-03-04T12:45:23Z)
• Deep Transformation-Invariant Clustering [24.23117820167443]
  We present an approach that does not rely on abstract features but instead learns to predict image transformations. This learning process naturally fits in the gradient-based training of K-means and Gaussian mixture model. We demonstrate that our novel approach yields competitive and highly promising results on standard image clustering benchmarks.
  arXiv  Detail & Related papers  (2020-06-19T13:43:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.