CLIP-CLOP: CLIP-Guided Collage and Photomontage

• URL: http://arxiv.org/abs/2205.03146v1
• Date: Fri, 6 May 2022 11:33:49 GMT
• Title: CLIP-CLOP: CLIP-Guided Collage and Photomontage
• Authors: Piotr Mirowski, Dylan Banarse, Mateusz Malinowski, Simon Osindero, Chrisantha Fernando
• Abstract summary: We design a gradient-based generator to produce collages. It requires the human artist to curate libraries of image patches and to describe (with prompts) the whole image composition. We explore the aesthetic potentials of high-resolution collages, and provide an open-source Google Colab as an artistic tool.
• Score: 16.460669517251084
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The unabated mystique of large-scale neural networks, such as the CLIP dual image-and-text encoder, popularized automatically generated art. Increasingly more sophisticated generators enhanced the artworks' realism and visual appearance, and creative prompt engineering enabled stylistic expression. Guided by an artist-in-the-loop ideal, we design a gradient-based generator to produce collages. It requires the human artist to curate libraries of image patches and to describe (with prompts) the whole image composition, with the option to manually adjust the patches' positions during generation, thereby allowing humans to reclaim some control of the process and achieve greater creative freedom. We explore the aesthetic potentials of high-resolution collages, and provide an open-source Google Colab as an artistic tool.

Related papers

• Neural-Polyptych: Content Controllable Painting Recreation for Diverse Genres [30.83874057768352]
  We present a unified framework, Neural-Polyptych, to facilitate the creation of expansive, high-resolution paintings. We have designed a multi-scale GAN-based architecture to decompose the generation process into two parts. We validate our approach to diverse genres of both Eastern and Western paintings.
  arXiv  Detail & Related papers  (2024-09-29T12:46:00Z)
• PixWizard: Versatile Image-to-Image Visual Assistant with Open-Language Instructions [66.92809850624118]
  PixWizard is an image-to-image visual assistant designed for image generation, manipulation, and translation based on free-from language instructions. We tackle a variety of vision tasks into a unified image-text-to-image generation framework and curate an Omni Pixel-to-Pixel Instruction-Tuning dataset. Our experiments demonstrate that PixWizard not only shows impressive generative and understanding abilities for images with diverse resolutions but also exhibits promising generalization capabilities with unseen tasks and human instructions.
  arXiv  Detail & Related papers  (2024-09-23T17:59:46Z)
• Alfie: Democratising RGBA Image Generation With No $$$ [33.334956022229846]
  We propose a fully-automated approach for obtaining RGBA illustrations by modifying the inference-time behavior of a pre-trained Diffusion Transformer model. We force the generation of entire subjects without sharp croppings, whose background is easily removed for seamless integration into design projects or artistic scenes.
  arXiv  Detail & Related papers  (2024-08-27T07:13:44Z)
• CreativeSynth: Creative Blending and Synthesis of Visual Arts based on Multimodal Diffusion [74.44273919041912]
  Large-scale text-to-image generative models have made impressive strides, showcasing their ability to synthesize a vast array of high-quality images. However, adapting these models for artistic image editing presents two significant challenges. We build the innovative unified framework Creative Synth, which is based on a diffusion model with the ability to coordinate multimodal inputs.
  arXiv  Detail & Related papers  (2024-01-25T10:42:09Z)
• Neural Collage Transfer: Artistic Reconstruction via Material Manipulation [20.72219392904935]
  Collage is a creative art form that uses diverse material scraps as a base unit to compose a single image. pixel-wise generation techniques can reproduce a target image in collage style, but it is not a suitable method due to the solid stroke-by-stroke nature of the collage form. We propose a method for learning to make collages via reinforcement learning without the need for demonstrations or collage artwork data.
  arXiv  Detail & Related papers  (2023-11-03T19:10:37Z)
• SketchDreamer: Interactive Text-Augmented Creative Sketch Ideation [111.2195741547517]
  We present a method to generate controlled sketches using a text-conditioned diffusion model trained on pixel representations of images. Our objective is to empower non-professional users to create sketches and, through a series of optimisation processes, transform a narrative into a storyboard.
  arXiv  Detail & Related papers  (2023-08-27T19:44:44Z)
• Composite Diffusion | whole >= \Sigma parts [0.0]
  This paper introduces Composite Diffusion as a means for artists to generate high-quality images by composing from the sub-scenes. We provide a comprehensive and modular method for Composite Diffusion that enables alternative ways of generating, composing, and harmonizing sub-scenes.
  arXiv  Detail & Related papers  (2023-07-25T17:58:43Z)
• QuantArt: Quantizing Image Style Transfer Towards High Visual Fidelity [94.5479418998225]
  We propose a new style transfer framework called QuantArt for high visual-fidelity stylization. Our framework achieves significantly higher visual fidelity compared with the existing style transfer methods.
  arXiv  Detail & Related papers  (2022-12-20T17:09:53Z)
• Exploring Latent Dimensions of Crowd-sourced Creativity [0.02294014185517203]
  We build our work on the largest AI-based creativity platform, Artbreeder. We explore the latent dimensions of images generated on this platform and present a novel framework for manipulating images to make them more creative.
  arXiv  Detail & Related papers  (2021-12-13T19:24:52Z)
• Generating Person Images with Appearance-aware Pose Stylizer [66.44220388377596]
  We present a novel end-to-end framework to generate realistic person images based on given person poses and appearances. The core of our framework is a novel generator called Appearance-aware Pose Stylizer (APS) which generates human images by coupling the target pose with the conditioned person appearance progressively.
  arXiv  Detail & Related papers  (2020-07-17T15:58:05Z)
• SketchyCOCO: Image Generation from Freehand Scene Sketches [71.85577739612579]
  We introduce the first method for automatic image generation from scene-level freehand sketches. Key contribution is an attribute vector bridged Geneversarative Adrial Network called EdgeGAN. We have built a large-scale composite dataset called SketchyCOCO to support and evaluate the solution.
  arXiv  Detail & Related papers  (2020-03-05T14:54:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.