Creative divergent synthesis with generative models

• URL: http://arxiv.org/abs/2211.08861v1
• Date: Wed, 16 Nov 2022 12:12:31 GMT
• Title: Creative divergent synthesis with generative models
• Authors: Axel Chemla--Romeu-Santos, Philippe Esling
• Abstract summary: Machine learning approaches now achieve impressive generation capabilities in numerous domains such as image, audio or video. We propose various perspectives on how this complicated goal could ever be achieved, and provide preliminary results on our novel training objective called textitBounded Adversarial Divergence (BAD)
• Score: 3.655021726150369
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Machine learning approaches now achieve impressive generation capabilities in numerous domains such as image, audio or video. However, most training \& evaluation frameworks revolve around the idea of strictly modelling the original data distribution rather than trying to extrapolate from it. This precludes the ability of such models to diverge from the original distribution and, hence, exhibit some creative traits. In this paper, we propose various perspectives on how this complicated goal could ever be achieved, and provide preliminary results on our novel training objective called \textit{Bounded Adversarial Divergence} (BAD).

Related papers

• Bridging Generative and Discriminative Models for Unified Visual Perception with Diffusion Priors [56.82596340418697]
  We propose a simple yet effective framework comprising a pre-trained Stable Diffusion (SD) model containing rich generative priors, a unified head (U-head) capable of integrating hierarchical representations, and an adapted expert providing discriminative priors. Comprehensive investigations unveil potential characteristics of Vermouth, such as varying granularity of perception concealed in latent variables at distinct time steps and various U-net stages. The promising results demonstrate the potential of diffusion models as formidable learners, establishing their significance in furnishing informative and robust visual representations.
  arXiv  Detail & Related papers  (2024-01-29T10:36:57Z)
• SODA: Bottleneck Diffusion Models for Representation Learning [75.7331354734152]
  We introduce SODA, a self-supervised diffusion model, designed for representation learning. The model incorporates an image encoder, which distills a source view into a compact representation, that guides the generation of related novel views. We show that by imposing a tight bottleneck between the encoder and a denoising decoder, we can turn diffusion models into strong representation learners.
  arXiv  Detail & Related papers  (2023-11-29T18:53:34Z)
• Fantastic Gains and Where to Find Them: On the Existence and Prospect of General Knowledge Transfer between Any Pretrained Model [74.62272538148245]
  We show that for arbitrary pairings of pretrained models, one model extracts significant data context unavailable in the other. We investigate if it is possible to transfer such "complementary" knowledge from one model to another without performance degradation.
  arXiv  Detail & Related papers  (2023-10-26T17:59:46Z)
• Reinforcement Learning for Generative AI: A Survey [40.21640713844257]
  This survey aims to shed light on a high-level review that spans a range of application areas. We provide a rigorous taxonomy in this area and make sufficient coverage on various models and applications. We conclude this survey by showing the potential directions that might tackle the limit of current models and expand the frontiers for generative AI.
  arXiv  Detail & Related papers  (2023-08-28T06:15:14Z)
• UniDiff: Advancing Vision-Language Models with Generative and Discriminative Learning [86.91893533388628]
  This paper presents UniDiff, a unified multi-modal model that integrates image-text contrastive learning (ITC), text-conditioned image synthesis learning (IS), and reciprocal semantic consistency modeling (RSC) UniDiff demonstrates versatility in both multi-modal understanding and generative tasks.
  arXiv  Detail & Related papers  (2023-06-01T15:39:38Z)
• Exploring Compositional Visual Generation with Latent Classifier Guidance [19.48538300223431]
  We train latent diffusion models and auxiliary latent classifiers to facilitate non-linear navigation of latent representation generation. We show that such conditional generation achieved by latent classifier guidance provably maximizes a lower bound of the conditional log probability during training. We show that this paradigm based on latent classifier guidance is agnostic to pre-trained generative models, and present competitive results for both image generation and sequential manipulation of real and synthetic images.
  arXiv  Detail & Related papers  (2023-04-25T03:02:58Z)
• Challenges in creative generative models for music: a divergence maximization perspective [3.655021726150369]
  Development of generative Machine Learning models in creative practices is raising more interest among artists, practitioners and performers. Most models are still unable to generate content that lay outside of the domain defined by the training dataset. We propose an alternative prospective framework, starting from a new general formulation of ML objectives.
  arXiv  Detail & Related papers  (2022-11-16T12:02:43Z)
• Composing Ensembles of Pre-trained Models via Iterative Consensus [95.10641301155232]
  We propose a unified framework for composing ensembles of different pre-trained models. We use pre-trained models as "generators" or "scorers" and compose them via closed-loop iterative consensus optimization. We demonstrate that consensus achieved by an ensemble of scorers outperforms the feedback of a single scorer.
  arXiv  Detail & Related papers  (2022-10-20T18:46:31Z)
• Meta Internal Learning [88.68276505511922]
  Internal learning for single-image generation is a framework, where a generator is trained to produce novel images based on a single image. We propose a meta-learning approach that enables training over a collection of images, in order to model the internal statistics of the sample image more effectively. Our results show that the models obtained are as suitable as single-image GANs for many common image applications.
  arXiv  Detail & Related papers  (2021-10-06T16:27:38Z)
• Active Divergence with Generative Deep Learning -- A Survey and Taxonomy [0.6435984242701043]
  We present a taxonomy and comprehensive survey of the state of the art of active divergence techniques. We highlight the potential for computational creativity researchers to advance these methods and use deep generative models in truly creative systems.
  arXiv  Detail & Related papers  (2021-07-12T17:29:28Z)
• Zero-shot Neural Passage Retrieval via Domain-targeted Synthetic Question Generation [10.324402925019946]
  A major obstacle to the wide-spread adoption of neural retrieval models is that they require large supervised training sets to surpass traditional term-based techniques. In this paper, we propose an approach to zero-shot learning for passage retrieval that uses synthetic question generation to close this gap.
  arXiv  Detail & Related papers  (2020-04-29T22:21:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.