Semantic-aware Next-Best-View for Multi-DoFs Mobile System in Search-and-Acquisition based Visual Perception

• URL: http://arxiv.org/abs/2404.16507v1
• Date: Thu, 25 Apr 2024 11:01:40 GMT
• Title: Semantic-aware Next-Best-View for Multi-DoFs Mobile System in Search-and-Acquisition based Visual Perception
• Authors: Xiaotong Yu, Chang-Wen Chen,
• Abstract summary: We formulate a novel information gain that integrates both visibility gain and semantic gain in a unified form to select the semantic-aware Next-Best-View. Several semantically relevant reconstruction metrics, including perspective directivity and region of interest (ROI)-to-full reconstruction volume ratio, are introduced to evaluate the performance of the proposed approach.
• Score: 20.480581428768854
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Efficient visual perception using mobile systems is crucial, particularly in unknown environments such as search and rescue operations, where swift and comprehensive perception of objects of interest is essential. In such real-world applications, objects of interest are often situated in complex environments, making the selection of the 'Next Best' view based solely on maximizing visibility gain suboptimal. Semantics, providing a higher-level interpretation of perception, should significantly contribute to the selection of the next viewpoint for various perception tasks. In this study, we formulate a novel information gain that integrates both visibility gain and semantic gain in a unified form to select the semantic-aware Next-Best-View. Additionally, we design an adaptive strategy with termination criterion to support a two-stage search-and-acquisition manoeuvre on multiple objects of interest aided by a multi-degree-of-freedoms (Multi-DoFs) mobile system. Several semantically relevant reconstruction metrics, including perspective directivity and region of interest (ROI)-to-full reconstruction volume ratio, are introduced to evaluate the performance of the proposed approach. Simulation experiments demonstrate the advantages of the proposed approach over existing methods, achieving improvements of up to 27.13% for the ROI-to-full reconstruction volume ratio and a 0.88234 average perspective directivity. Furthermore, the planned motion trajectory exhibits better perceiving coverage toward the target.

Related papers

• Re-Align: Aligning Vision Language Models via Retrieval-Augmented Direct Preference Optimization [19.37373012848517]
  Large Vision Language Models (VLMs) are prone to significant hallucinations, particularly in the form of cross-modal inconsistencies. We introduce Re-Align, a novel alignment framework that leverages image retrieval to construct a dual-preference dataset. We also introduce rDPO, an extension of the standard direct preference optimization that incorporates an additional visual preference objective during fine-tuning.
  arXiv  Detail & Related papers  (2025-02-18T18:59:57Z)
• Interactive Visualization Recommendation with Hier-SUCB [52.11209329270573]
  We propose an interactive personalized visualization recommendation (PVisRec) system that learns on user feedback from previous interactions. For more interactive and accurate recommendations, we propose Hier-SUCB, a contextual semi-bandit in the PVisRec setting.
  arXiv  Detail & Related papers  (2025-02-05T17:14:45Z)
• Learning Recommender Systems with Soft Target: A Decoupled Perspective [49.83787742587449]
  We propose a novel decoupled soft label optimization framework to consider the objectives as two aspects by leveraging soft labels. We present a sensible soft-label generation algorithm that models a label propagation algorithm to explore users' latent interests in unobserved feedback via neighbors.
  arXiv  Detail & Related papers  (2024-10-09T04:20:15Z)
• Visual Information Extraction in the Wild: Practical Dataset and End-to-end Solution [48.693941280097974]
  We propose a large-scale dataset consisting of camera images for visual information extraction (VIE) We propose a novel framework for end-to-end VIE that combines the stages of OCR and information extraction in an end-to-end learning fashion. We evaluate the existing end-to-end methods for VIE on the proposed dataset and observe that the performance of these methods has a distinguishable drop from SROIE to our proposed dataset due to the larger variance of layout and entities.
  arXiv  Detail & Related papers  (2023-05-12T14:11:47Z)
• Attentive Multi-View Deep Subspace Clustering Net [4.3386084277869505]
  We propose a novel Attentive Multi-View Deep Subspace Nets (AMVDSN) Our proposed method seeks to find a joint latent representation that explicitly considers both consensus and view-specific information. The experimental results on seven real-world data sets have demonstrated the effectiveness of our proposed algorithm against some state-of-the-art subspace learning approaches.
  arXiv  Detail & Related papers  (2021-12-23T12:57:26Z)
• A Variational Information Bottleneck Approach to Multi-Omics Data Integration [98.6475134630792]
  We propose a deep variational information bottleneck (IB) approach for incomplete multi-view observations. Our method applies the IB framework on marginal and joint representations of the observed views to focus on intra-view and inter-view interactions that are relevant for the target. Experiments on real-world datasets show that our method consistently achieves gain from data integration and outperforms state-of-the-art benchmarks.
  arXiv  Detail & Related papers  (2021-02-05T06:05:39Z)
• Embodied Visual Active Learning for Semantic Segmentation [33.02424587900808]
  We study the task of embodied visual active learning, where an agent is set to explore a 3d environment with the goal to acquire visual scene understanding. We develop a battery of agents - both learnt and pre-specified - and with different levels of knowledge of the environment. We extensively evaluate the proposed models using the Matterport3D simulator and show that a fully learnt method outperforms comparable pre-specified counterparts.
  arXiv  Detail & Related papers  (2020-12-17T11:02:34Z)
• Visual Tracking by TridentAlign and Context Embedding [71.60159881028432]
  We propose novel TridentAlign and context embedding modules for Siamese network-based visual tracking methods. The performance of the proposed tracker is comparable to that of state-of-the-art trackers, while the proposed tracker runs at real-time speed.
  arXiv  Detail & Related papers  (2020-07-14T08:00:26Z)
• Embedded Deep Bilinear Interactive Information and Selective Fusion for Multi-view Learning [70.67092105994598]
  We propose a novel multi-view learning framework to make the multi-view classification better aimed at the above-mentioned two aspects. In particular, we train different deep neural networks to learn various intra-view representations. Experiments on six publicly available datasets demonstrate the effectiveness of the proposed method.
  arXiv  Detail & Related papers  (2020-07-13T01:13:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.