Evolving Three Dimension (3D) Abstract Art: Fitting Concepts by Language

• URL: http://arxiv.org/abs/2304.12932v1
• Date: Mon, 24 Apr 2023 07:47:48 GMT
• Title: Evolving Three Dimension (3D) Abstract Art: Fitting Concepts by Language
• Authors: Yingtao Tian
• Abstract summary: We propose to explore computational creativity in making abstract 3D art by bridging evolution strategies (ES) and 3D rendering through customizable parameterization of scenes. Our approach is capable of placing semi-transparent triangles in 3D scenes that, when viewed from specified angles, render into films that look like artists' specification expressed in natural language.
• Score: 2.7336516660166295
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Computational creativity has contributed heavily to abstract art in modern era, allowing artists to create high quality, abstract two dimension (2D) arts with a high level of controllability and expressibility. However, even with computational approaches that have promising result in making concrete 3D art, computationally addressing abstract 3D art with high-quality and controllability remains an open question. To fill this gap, we propose to explore computational creativity in making abstract 3D art by bridging evolution strategies (ES) and 3D rendering through customizable parameterization of scenes. We demonstrate that our approach is capable of placing semi-transparent triangles in 3D scenes that, when viewed from specified angles, render into films that look like artists' specification expressed in natural language. This provides a new way for the artist to easily express creativity ideas for abstract 3D art. The supplementary material, which contains code, animation for all figures, and more examples, is here: https://es3dart.github.io/

Related papers

• Illusion3D: 3D Multiview Illusion with 2D Diffusion Priors [19.58299058678772]
  We present a simple yet effective approach for creating 3D multiview illusions based on user-provided text prompts or images. Our method leverages a pre-trained text-to-image diffusion model to optimize the textures and geometry of neural 3D representations. We develop several techniques to improve the quality of the generated 3D multiview illusions.
  arXiv  Detail & Related papers  (2024-12-12T18:59:59Z)
• ART3D: 3D Gaussian Splatting for Text-Guided Artistic Scenes Generation [18.699440994076003]
  ART3D is a novel framework that combines diffusion models and 3D Gaussian splatting techniques. By leveraging depth information and an initial artistic image, we generate a point cloud map. We also propose a depth consistency module to enhance 3D scene consistency.
  arXiv  Detail & Related papers  (2024-05-17T03:19:36Z)
• Agent3D-Zero: An Agent for Zero-shot 3D Understanding [79.88440434836673]
  Agent3D-Zero is an innovative 3D-aware agent framework addressing the 3D scene understanding. We propose a novel way to make use of a Large Visual Language Model (VLM) via actively selecting and analyzing a series of viewpoints for 3D understanding. A distinctive advantage of Agent3D-Zero is the introduction of novel visual prompts, which significantly unleash the VLMs' ability to identify the most informative viewpoints.
  arXiv  Detail & Related papers  (2024-03-18T14:47:03Z)
• Control3D: Towards Controllable Text-to-3D Generation [107.81136630589263]
  We present a text-to-3D generation conditioning on the additional hand-drawn sketch, namely Control3D. A 2D conditioned diffusion model (ControlNet) is remoulded to guide the learning of 3D scene parameterized as NeRF. We exploit a pre-trained differentiable photo-to-sketch model to directly estimate the sketch of the rendered image over synthetic 3D scene.
  arXiv  Detail & Related papers  (2023-11-09T15:50:32Z)
• Aladdin: Zero-Shot Hallucination of Stylized 3D Assets from Abstract Scene Descriptions [0.19116784879310023]
  We present a system to generate stylized assets for 3D scenes described by a short phrase. It is robust to open-world concepts in a way that traditional methods trained on limited data are not, more creative freedom to the 3D artist.
  arXiv  Detail & Related papers  (2023-06-09T19:24:39Z)
• Multi-CLIP: Contrastive Vision-Language Pre-training for Question Answering tasks in 3D Scenes [68.61199623705096]
  Training models to apply common-sense linguistic knowledge and visual concepts from 2D images to 3D scene understanding is a promising direction that researchers have only recently started to explore. We propose a novel 3D pre-training Vision-Language method, namely Multi-CLIP, that enables a model to learn language-grounded and transferable 3D scene point cloud representations.
  arXiv  Detail & Related papers  (2023-06-04T11:08:53Z)
• CLIP-Guided Vision-Language Pre-training for Question Answering in 3D Scenes [68.61199623705096]
  We design a novel 3D pre-training Vision-Language method that helps a model learn semantically meaningful and transferable 3D scene point cloud representations. We inject the representational power of the popular CLIP model into our 3D encoder by aligning the encoded 3D scene features with the corresponding 2D image and text embeddings. We evaluate our model's 3D world reasoning capability on the downstream task of 3D Visual Question Answering.
  arXiv  Detail & Related papers  (2023-04-12T16:52:29Z)
• Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creation [45.69270771487455]
  We propose a new method of Fantasia3D for high-quality text-to-3D content creation. Key to Fantasia3D is the disentangled modeling and learning of geometry and appearance. Our framework is more compatible with popular graphics engines, supporting relighting, editing, and physical simulation of the generated 3D assets.
  arXiv  Detail & Related papers  (2023-03-24T09:30:09Z)
• Learning Indoor Inverse Rendering with 3D Spatially-Varying Lighting [149.1673041605155]
  We address the problem of jointly estimating albedo, normals, depth and 3D spatially-varying lighting from a single image. Most existing methods formulate the task as image-to-image translation, ignoring the 3D properties of the scene. We propose a unified, learning-based inverse framework that formulates 3D spatially-varying lighting.
  arXiv  Detail & Related papers  (2021-09-13T15:29:03Z)
• Realistic Image Synthesis with Configurable 3D Scene Layouts [59.872657806747576]
  We propose a novel approach to realistic-looking image synthesis based on a 3D scene layout. Our approach takes a 3D scene with semantic class labels as input and trains a 3D scene painting network. With the trained painting network, realistic-looking images for the input 3D scene can be rendered and manipulated.
  arXiv  Detail & Related papers  (2021-08-23T09:44:56Z)
• Shadow Art Revisited: A Differentiable Rendering Based Approach [26.910401398827123]
  We revisit shadow art using differentiable rendering based optimization frameworks to obtain the 3D sculpture. Our choice of using differentiable rendering for generating shadow art sculptures can be attributed to its ability to learn the underlying 3D geometry solely from image data. We demonstrate the generation of 3D sculptures to cast shadows of faces, animated movie characters, and applicability of the framework to sketch-based 3D reconstruction of underlying shapes.
  arXiv  Detail & Related papers  (2021-07-30T10:43:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.