A Study on Learning Social Robot Navigation with Multimodal Perception
- URL: http://arxiv.org/abs/2309.12568v1
- Date: Fri, 22 Sep 2023 01:47:47 GMT
- Title: A Study on Learning Social Robot Navigation with Multimodal Perception
- Authors: Bhabaranjan Panigrahi, Amir Hossain Raj, Mohammad Nazeri and Xuesu
Xiao
- Abstract summary: We present a study on learning social robot navigation with multimodal perception using a large-scale real-world dataset.
We compare unimodal and multimodal learning approaches against a set of classical navigation approaches in different social scenarios.
The results show that multimodal learning has a clear advantage over unimodal learning in both dataset and human studies.
- Score: 6.052803245103173
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Autonomous mobile robots need to perceive the environments with their onboard
sensors (e.g., LiDARs and RGB cameras) and then make appropriate navigation
decisions. In order to navigate human-inhabited public spaces, such a
navigation task becomes more than only obstacle avoidance, but also requires
considering surrounding humans and their intentions to somewhat change the
navigation behavior in response to the underlying social norms, i.e., being
socially compliant. Machine learning methods are shown to be effective in
capturing those complex and subtle social interactions in a data-driven manner,
without explicitly hand-crafting simplified models or cost functions.
Considering multiple available sensor modalities and the efficiency of learning
methods, this paper presents a comprehensive study on learning social robot
navigation with multimodal perception using a large-scale real-world dataset.
The study investigates social robot navigation decision making on both the
global and local planning levels and contrasts unimodal and multimodal learning
against a set of classical navigation approaches in different social scenarios,
while also analyzing the training and generalizability performance from the
learning perspective. We also conduct a human study on how learning with
multimodal perception affects the perceived social compliance. The results show
that multimodal learning has a clear advantage over unimodal learning in both
dataset and human studies. We open-source our code for the community's future
use to study multimodal perception for learning social robot navigation.
Related papers
- Principles and Guidelines for Evaluating Social Robot Navigation
Algorithms [44.51586279645062]
Social robot navigation is difficult to evaluate because it involves dynamic human agents and their perceptions of the appropriateness of robot behavior.
Our contributions include (a) a definition of a socially navigating robot as one that respects the principles of safety, comfort, legibility, politeness, social competency, agent understanding, proactivity, and responsiveness to context, (b) guidelines for the use of metrics, development of scenarios, benchmarks, datasets, and simulators to evaluate social navigation, and (c) a social navigation metrics framework to make it easier to compare results from different simulators, robots and datasets.
arXiv Detail & Related papers (2023-06-29T07:31:43Z) - Navigating to Objects in the Real World [76.1517654037993]
We present a large-scale empirical study of semantic visual navigation methods comparing methods from classical, modular, and end-to-end learning approaches.
We find that modular learning works well in the real world, attaining a 90% success rate.
In contrast, end-to-end learning does not, dropping from 77% simulation to 23% real-world success rate due to a large image domain gap between simulation and reality.
arXiv Detail & Related papers (2022-12-02T01:10:47Z) - Learning Social Navigation from Demonstrations with Conditional Neural
Processes [2.627046865670577]
This paper presents a data-driven navigation architecture that uses Conditional Neural Processes to learn global and local controllers of the mobile robot from observations.
Our results demonstrate that the proposed framework can successfully carry out navigation tasks regarding social norms in the data.
arXiv Detail & Related papers (2022-10-07T14:37:56Z) - Gesture2Path: Imitation Learning for Gesture-aware Navigation [54.570943577423094]
We present Gesture2Path, a novel social navigation approach that combines image-based imitation learning with model-predictive control.
We deploy our method on real robots and showcase the effectiveness of our approach for the four gestures-navigation scenarios.
arXiv Detail & Related papers (2022-09-19T23:05:36Z) - Socially Compliant Navigation Dataset (SCAND): A Large-Scale Dataset of
Demonstrations for Social Navigation [92.66286342108934]
Social navigation is the capability of an autonomous agent, such as a robot, to navigate in a'socially compliant' manner in the presence of other intelligent agents such as humans.
Our dataset contains 8.7 hours, 138 trajectories, 25 miles of socially compliant, human teleoperated driving demonstrations.
arXiv Detail & Related papers (2022-03-28T19:09:11Z) - What Matters in Learning from Offline Human Demonstrations for Robot
Manipulation [64.43440450794495]
We conduct an extensive study of six offline learning algorithms for robot manipulation.
Our study analyzes the most critical challenges when learning from offline human data.
We highlight opportunities for learning from human datasets.
arXiv Detail & Related papers (2021-08-06T20:48:30Z) - ReLMM: Practical RL for Learning Mobile Manipulation Skills Using Only
Onboard Sensors [64.2809875343854]
We study how robots can autonomously learn skills that require a combination of navigation and grasping.
Our system, ReLMM, can learn continuously on a real-world platform without any environment instrumentation.
After a grasp curriculum training phase, ReLMM can learn navigation and grasping together fully automatically, in around 40 hours of real-world training.
arXiv Detail & Related papers (2021-07-28T17:59:41Z) - From Learning to Relearning: A Framework for Diminishing Bias in Social
Robot Navigation [3.3511723893430476]
We argue that social navigation models can replicate, promote, and amplify societal unfairness such as discrimination and segregation.
Our proposed framework consists of two components: textitlearning which incorporates social context into the learning process to account for safety and comfort, and textitrelearning to detect and correct potentially harmful outcomes before the onset.
arXiv Detail & Related papers (2021-01-07T17:42:35Z) - Human Trajectory Forecasting in Crowds: A Deep Learning Perspective [89.4600982169]
We present an in-depth analysis of existing deep learning-based methods for modelling social interactions.
We propose two knowledge-based data-driven methods to effectively capture these social interactions.
We develop a large scale interaction-centric benchmark TrajNet++, a significant yet missing component in the field of human trajectory forecasting.
arXiv Detail & Related papers (2020-07-07T17:19:56Z) - Robot Perception enables Complex Navigation Behavior via Self-Supervised
Learning [23.54696982881734]
We propose an approach to unify successful robot perception systems for active target-driven navigation tasks via reinforcement learning (RL)
Our method temporally incorporates compact motion and visual perception data, directly obtained using self-supervision from a single image sequence.
We demonstrate our approach on two real-world driving dataset, KITTI and Oxford RobotCar, using the new interactive CityLearn framework.
arXiv Detail & Related papers (2020-06-16T07:45:47Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.