A Partially Supervised Reinforcement Learning Framework for Visual Active Search

• URL: http://arxiv.org/abs/2310.09689v2
• Date: Wed, 8 Nov 2023 03:32:10 GMT
• Title: A Partially Supervised Reinforcement Learning Framework for Visual Active Search
• Authors: Anindya Sarkar, Nathan Jacobs, Yevgeniy Vorobeychik
• Abstract summary: Visual active search (VAS) has been proposed as a modeling framework in which visual cues are used to guide exploration. We propose an approach that combines the strength of both DRL and conventional active search by decomposing the search policy into a prediction module. We develop a novel meta-learning approach for jointly learning the resulting combined policy that can make effective use of supervised information obtained both at training and decision time.
• Score: 36.966522001393734
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Visual active search (VAS) has been proposed as a modeling framework in which visual cues are used to guide exploration, with the goal of identifying regions of interest in a large geospatial area. Its potential applications include identifying hot spots of rare wildlife poaching activity, search-and-rescue scenarios, identifying illegal trafficking of weapons, drugs, or people, and many others. State of the art approaches to VAS include applications of deep reinforcement learning (DRL), which yield end-to-end search policies, and traditional active search, which combines predictions with custom algorithmic approaches. While the DRL framework has been shown to greatly outperform traditional active search in such domains, its end-to-end nature does not make full use of supervised information attained either during training, or during actual search, a significant limitation if search tasks differ significantly from those in the training distribution. We propose an approach that combines the strength of both DRL and conventional active search by decomposing the search policy into a prediction module, which produces a geospatial distribution of regions of interest based on task embedding and search history, and a search module, which takes the predictions and search history as input and outputs the search distribution. We develop a novel meta-learning approach for jointly learning the resulting combined policy that can make effective use of supervised information obtained both at training and decision time. Our extensive experiments demonstrate that the proposed representation and meta-learning frameworks significantly outperform state of the art in visual active search on several problem domains.

Related papers

• Semantic-Based Active Perception for Humanoid Visual Tasks with Foveal Sensors [49.99728312519117]
  The aim of this work is to establish how accurately a recent semantic-based active perception model is able to complete visual tasks that are regularly performed by humans. This model exploits the ability of current object detectors to localize and classify a large number of object classes and to update a semantic description of a scene across multiple fixations. In the task of scene exploration, the semantic-based method demonstrates superior performance compared to the traditional saliency-based model.
  arXiv  Detail & Related papers  (2024-04-16T18:15:57Z)
• A Survey on Deep Learning-based Spatio-temporal Action Detection [8.456482280676884]
  STAD aims to classify the actions present in a video and localize them in space and time. It has become a particularly active area of research in computer vision because of its explosively emerging real-world applications. This paper provides a comprehensive review of the state-of-the-art deep learning-based methods for STAD.
  arXiv  Detail & Related papers  (2023-08-03T08:48:14Z)
• On the Importance of Exploration for Generalization in Reinforcement Learning [89.63074327328765]
  We propose EDE: Exploration via Distributional Ensemble, a method that encourages exploration of states with high uncertainty. Our algorithm is the first value-based approach to achieve state-of-the-art on both Procgen and Crafter.
  arXiv  Detail & Related papers  (2023-06-08T18:07:02Z)
• A Visual Active Search Framework for Geospatial Exploration [36.31732056074638]
  Many problems can be viewed as forms of geospatial search aided by aerial imagery. We model this class of problems in a visual active search (VAS) framework, which has three key inputs. We propose a reinforcement learning approach for VAS that learns a meta-search policy from a collection of fully annotated search tasks.
  arXiv  Detail & Related papers  (2022-11-28T21:53:05Z)
• CrossBeam: Learning to Search in Bottom-Up Program Synthesis [51.37514793318815]
  We propose training a neural model to learn a hands-on search policy for bottom-up synthesis. Our approach, called CrossBeam, uses the neural model to choose how to combine previously-explored programs into new programs. We observe that CrossBeam learns to search efficiently, exploring much smaller portions of the program space compared to the state-of-the-art.
  arXiv  Detail & Related papers  (2022-03-20T04:41:05Z)
• Deep Reinforcement Agent for Efficient Instant Search [14.086339486783018]
  We propose to address the load issue by identifying tokens that are semantically more salient towards retrieving relevant documents. We train a reinforcement agent that interacts directly with the search engine and learns to predict the word's importance. A novel evaluation framework is presented to study the trade-off between the number of triggered searches and the system's performance.
  arXiv  Detail & Related papers  (2022-03-17T22:47:15Z)
• Embodied Learning for Lifelong Visual Perception [33.02424587900808]
  We study lifelong visual perception in an embodied setup, where we develop new models and compare various agents that navigate in buildings. The purpose of the agents is to recognize objects and other semantic classes in the whole building at the end of a process that combines exploration and active visual learning.
  arXiv  Detail & Related papers  (2021-12-28T10:47:13Z)
• Nonmyopic Multifidelity Active Search [15.689830609697685]
  We propose a model of multifidelity active search, as well as a novel, computationally efficient policy for this setting. We evaluate the performance of our solution on real-world datasets and demonstrate significantly better performance than natural benchmarks.
  arXiv  Detail & Related papers  (2021-06-11T12:55:51Z)
• Reannealing of Decaying Exploration Based On Heuristic Measure in Deep Q-Network [82.20059754270302]
  We propose an algorithm based on the idea of reannealing, that aims at encouraging exploration only when it is needed. We perform an illustrative case study showing that it has potential to both accelerate training and obtain a better policy.
  arXiv  Detail & Related papers  (2020-09-29T20:40:00Z)
• AutoOD: Automated Outlier Detection via Curiosity-guided Search and Self-imitation Learning [72.99415402575886]
  Outlier detection is an important data mining task with numerous practical applications. We propose AutoOD, an automated outlier detection framework, which aims to search for an optimal neural network model. Experimental results on various real-world benchmark datasets demonstrate that the deep model identified by AutoOD achieves the best performance.
  arXiv  Detail & Related papers  (2020-06-19T18:57:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.