An Empirical Study on the Generalization Power of Neural Representations Learned via Visual Guessing Games

• URL: http://arxiv.org/abs/2102.00424v1
• Date: Sun, 31 Jan 2021 10:30:48 GMT
• Title: An Empirical Study on the Generalization Power of Neural Representations Learned via Visual Guessing Games
• Authors: Alessandro Suglia, Yonatan Bisk, Ioannis Konstas, Antonio Vergari, Emanuele Bastianelli, Andrea Vanzo, Oliver Lemon
• Abstract summary: This work investigates how well an artificial agent can benefit from playing guessing games when later asked to perform on novel NLP downstream tasks such as Visual Question Answering (VQA) We propose two ways to exploit playing guessing games: 1) a supervised learning scenario in which the agent learns to mimic successful guessing games and 2) a novel way for an agent to play by itself, called Self-play via Iterated Experience Learning (SPIEL)
• Score: 79.23847247132345
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Guessing games are a prototypical instance of the "learning by interacting" paradigm. This work investigates how well an artificial agent can benefit from playing guessing games when later asked to perform on novel NLP downstream tasks such as Visual Question Answering (VQA). We propose two ways to exploit playing guessing games: 1) a supervised learning scenario in which the agent learns to mimic successful guessing games and 2) a novel way for an agent to play by itself, called Self-play via Iterated Experience Learning (SPIEL). We evaluate the ability of both procedures to generalize: an in-domain evaluation shows an increased accuracy (+7.79) compared with competitors on the evaluation suite CompGuessWhat?!; a transfer evaluation shows improved performance for VQA on the TDIUC dataset in terms of harmonic average accuracy (+5.31) thanks to more fine-grained object representations learned via SPIEL.

Related papers

• Preference-conditioned Pixel-based AI Agent For Game Testing [1.5059676044537105]
  Game-testing AI agents that learn by interaction with the environment have the potential to mitigate these challenges. This paper proposes an agent design that mainly depends on pixel-based state observations while exploring the environment conditioned on a user's preference. Our agent significantly outperforms state-of-the-art pixel-based game testing agents over exploration coverage and test execution quality when evaluated on a complex open-world environment resembling many aspects of real AAA games.
  arXiv  Detail & Related papers  (2023-08-18T04:19:36Z)
• Scaling Laws for Imitation Learning in Single-Agent Games [29.941613597833133]
  We investigate whether carefully scaling up model and data size can bring similar improvements in the imitation learning setting for single-agent games. We first demonstrate our findings on a variety of Atari games, and thereafter focus on the extremely challenging game of NetHack. We find that IL loss and mean return scale smoothly with the compute budget and are strongly correlated, resulting in power laws for training compute-optimal IL agents.
  arXiv  Detail & Related papers  (2023-07-18T16:43:03Z)
• SPRING: Studying the Paper and Reasoning to Play Games [102.5587155284795]
  We propose a novel approach, SPRING, to read the game's original academic paper and use the knowledge learned to reason and play the game through a large language model (LLM) In experiments, we study the quality of in-context "reasoning" induced by different forms of prompts under the setting of the Crafter open-world environment. Our experiments suggest that LLMs, when prompted with consistent chain-of-thought, have great potential in completing sophisticated high-level trajectories.
  arXiv  Detail & Related papers  (2023-05-24T18:14:35Z)
• Learning to Play Text-based Adventure Games with Maximum Entropy Reinforcement Learning [4.698846136465861]
  We adapt the soft-actor-critic (SAC) algorithm to the text-based environment. We show that the reward shaping technique helps the agent to learn the policy faster and achieve higher scores.
  arXiv  Detail & Related papers  (2023-02-21T15:16:12Z)
• Read and Reap the Rewards: Learning to Play Atari with the Help of Instruction Manuals [69.76245723797368]
  Read and Reward speeds up RL algorithms on Atari games by reading manuals released by the Atari game developers. Various RL algorithms obtain significant improvement in performance and training speed when assisted by our design.
  arXiv  Detail & Related papers  (2023-02-09T05:47:03Z)
• PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-training [94.87393610927812]
  We present an off-policy, interactive reinforcement learning algorithm that capitalizes on the strengths of both feedback and off-policy learning. We demonstrate that our approach is capable of learning tasks of higher complexity than previously considered by human-in-the-loop methods.
  arXiv  Detail & Related papers  (2021-06-09T14:10:50Z)
• Approximate exploitability: Learning a best response in large games [31.066412349285994]
  We introduce ISMCTS-BR, a scalable search-based deep reinforcement learning algorithm for learning a best response to an agent. We demonstrate the technique in several two-player zero-sum games against a variety of agents.
  arXiv  Detail & Related papers  (2020-04-20T23:36:40Z)
• Learning from Learners: Adapting Reinforcement Learning Agents to be Competitive in a Card Game [71.24825724518847]
  We present a study on how popular reinforcement learning algorithms can be adapted to learn and to play a real-world implementation of a competitive multiplayer card game. We propose specific training and validation routines for the learning agents, in order to evaluate how the agents learn to be competitive and explain how they adapt to each others' playing style.
  arXiv  Detail & Related papers  (2020-04-08T14:11:05Z)
• Disentangling Controllable Object through Video Prediction Improves Visual Reinforcement Learning [82.25034245150582]
  In many vision-based reinforcement learning problems, the agent controls a movable object in its visual field. We propose an end-to-end learning framework to disentangle the controllable object from the observation signal. The disentangled representation is shown to be useful for RL as additional observation channels to the agent.
  arXiv  Detail & Related papers  (2020-02-21T05:43:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.