From Language Games to Drawing Games

• URL: http://arxiv.org/abs/2010.02820v2
• Date: Thu, 10 Dec 2020 11:09:47 GMT
• Title: From Language Games to Drawing Games
• Authors: Chrisantha Fernando, Daria Zenkova, Stanislav Nikolov, Simon Osindero
• Abstract summary: We invent a set of drawing games, analogous to the approach taken by emergent language research in inventing communication games. A critical difference is that drawing games demand much less effort from the receiver than do language games. We present some preliminary experiments which have generated images by closing the generative-critical loop.
• Score: 6.93765975252665
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We attempt to automate various artistic processes by inventing a set of drawing games, analogous to the approach taken by emergent language research in inventing communication games. A critical difference is that drawing games demand much less effort from the receiver than do language games. Artists must work with pre-trained viewers who spend little time learning artist specific representational conventions, but who instead have a pre-trained visual system optimized for behaviour in the world by understanding to varying extents the environment's visual affordances. After considering various kinds of drawing game we present some preliminary experiments which have generated images by closing the generative-critical loop.

Related papers

• ARtVista: Gateway To Empower Anyone Into Artist [14.700883382465452]
  We propose ARtVista - a novel system integrating AR and generative AI technologies. ARtVista recommends reference images aligned with users' abstract ideas and generates sketches for users to draw. We perform a pilot study and reveal positive feedback on its usability.
  arXiv  Detail & Related papers  (2024-03-13T18:00:57Z)
• Re:Draw -- Context Aware Translation as a Controllable Method for Artistic Production [8.383295377277836]
  We introduce context-aware translation, a novel method that combines the benefits of inpainting and image-to-image translation. As an use case, we apply our method to redraw any hand-drawn animated character eyes based on any design specifications.
  arXiv  Detail & Related papers  (2024-01-07T14:34:34Z)
• SketchDreamer: Interactive Text-Augmented Creative Sketch Ideation [111.2195741547517]
  We present a method to generate controlled sketches using a text-conditioned diffusion model trained on pixel representations of images. Our objective is to empower non-professional users to create sketches and, through a series of optimisation processes, transform a narrative into a storyboard.
  arXiv  Detail & Related papers  (2023-08-27T19:44:44Z)
• ImageBrush: Learning Visual In-Context Instructions for Exemplar-Based Image Manipulation [49.07254928141495]
  We propose a novel manipulation methodology, dubbed ImageBrush, that learns visual instructions for more accurate image editing. Our key idea is to employ a pair of transformation images as visual instructions, which precisely captures human intention. Our model exhibits robust generalization capabilities on various downstream tasks such as pose transfer, image translation and video inpainting.
  arXiv  Detail & Related papers  (2023-08-02T01:57:11Z)
• Iconary: A Pictionary-Based Game for Testing Multimodal Communication with Drawings and Text [70.14613727284741]
  Communicating with humans is challenging for AIs because it requires a shared understanding of the world, complex semantics, and at times multi-modal gestures. We investigate these challenges in the context of Iconary, a collaborative game of drawing and guessing based on Pictionary. We propose models to play Iconary and train them on over 55,000 games between human players.
  arXiv  Detail & Related papers  (2021-12-01T19:41:03Z)
• Emergent Graphical Conventions in a Visual Communication Game [80.79297387339614]
  Humans communicate with graphical sketches apart from symbolic languages. We take the very first step to model and simulate such an evolution process via two neural agents playing a visual communication game. We devise a novel reinforcement learning method such that agents are evolved jointly towards successful communication and abstract graphical conventions.
  arXiv  Detail & Related papers  (2021-11-28T18:59:57Z)
• Contrastive Learning of Generalized Game Representations [2.323282558557423]
  Representing games through their pixels offers a promising approach for building general-purpose and versatile game models. While games are not merely images, neural network models trained on game pixels often capture differences of the visual style of the image rather than the content of the game. In this paper we build on recent advances in contrastive learning and showcase its benefits for representation learning in games.
  arXiv  Detail & Related papers  (2021-06-18T11:17:54Z)
• The emergence of visual semantics through communication games [0.0]
  Communication systems which capture visual semantics can be learned in a completely self-supervised manner by playing the right types of game. Our work bridges a gap between emergent communication research and self-supervised feature learning.
  arXiv  Detail & Related papers  (2021-01-25T17:43:37Z)
• Deep Reinforcement Learning with Stacked Hierarchical Attention for Text-based Games [64.11746320061965]
  We study reinforcement learning for text-based games, which are interactive simulations in the context of natural language. We aim to conduct explicit reasoning with knowledge graphs for decision making, so that the actions of an agent are generated and supported by an interpretable inference procedure. We extensively evaluate our method on a number of man-made benchmark games, and the experimental results demonstrate that our method performs better than existing text-based agents.
  arXiv  Detail & Related papers  (2020-10-22T12:40:22Z)
• Exploration Based Language Learning for Text-Based Games [72.30525050367216]
  This work presents an exploration and imitation-learning-based agent capable of state-of-the-art performance in playing text-based computer games. Text-based computer games describe their world to the player through natural language and expect the player to interact with the game using text. These games are of interest as they can be seen as a testbed for language understanding, problem-solving, and language generation by artificial agents.
  arXiv  Detail & Related papers  (2020-01-24T03:03:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.