Social NCE: Contrastive Learning of Socially-aware Motion Representations

• URL: http://arxiv.org/abs/2012.11717v2
• Date: Thu, 15 Apr 2021 17:54:33 GMT
• Title: Social NCE: Contrastive Learning of Socially-aware Motion Representations
• Authors: Yuejiang Liu, Qi Yan, Alexandre Alahi
• Abstract summary: Experimental results show that the proposed method dramatically reduces the collision rates of recent trajectory forecasting, behavioral cloning and reinforcement learning algorithms. Our method makes few assumptions about neural architecture designs, and hence can be used as a generic way to promote the robustness of neural motion models.
• Score: 87.82126838588279
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Learning socially-aware motion representations is at the core of recent advances in human trajectory forecasting and robot navigation in crowded spaces. Despite promising progress, existing neural motion models often struggle to generalize in closed-loop operations (e.g., output colliding trajectories), when the training set lacks examples collected from dangerous scenarios. In this work, we propose to address this issue via contrastive learning with negative data augmentation. Concretely, we introduce a social contrastive loss that encourages the encoded motion representation to preserve sufficient information for distinguishing a positive future event from a set of negative ones. We explicitly draw these negative samples based on our domain knowledge of unfavorable circumstances in the multi-agent context. Experimental results show that the proposed method dramatically reduces the collision rates of recent trajectory forecasting, behavioral cloning and reinforcement learning algorithms, outperforming current state-of-the-art models on several benchmarks. Our method makes few assumptions about neural architecture designs, and hence can be used as a generic way to promote the robustness of neural motion models.

Related papers

• Continual Learning of Conjugated Visual Representations through Higher-order Motion Flows [21.17248975377718]
  Learning with neural networks presents several challenges due to the non-i.i.d. nature of the data. It also offers novel opportunities to develop representations that are consistent with the information flow. In this paper we investigate the case of unsupervised continual learning of pixel-wise features subject to multiple motion-induced constraints.
  arXiv  Detail & Related papers  (2024-09-16T19:08:32Z)
• Segue: Side-information Guided Generative Unlearnable Examples for Facial Privacy Protection in Real World [64.4289385463226]
  We propose Segue: Side-information guided generative unlearnable examples. To improve transferability, we introduce side information such as true labels and pseudo labels. It can resist JPEG compression, adversarial training, and some standard data augmentations.
  arXiv  Detail & Related papers  (2023-10-24T06:22:37Z)
• Safety-compliant Generative Adversarial Networks for Human Trajectory Forecasting [95.82600221180415]
  Human forecasting in crowds presents the challenges of modelling social interactions and outputting collision-free multimodal distribution. We introduce SGANv2, an improved safety-compliant SGAN architecture equipped with motion-temporal interaction modelling and a transformer-based discriminator design.
  arXiv  Detail & Related papers  (2022-09-25T15:18:56Z)
• Action-based Contrastive Learning for Trajectory Prediction [4.675212251005813]
  Trajectory prediction is an essential task for successful human robot interaction, such as in autonomous driving. In this work, we address the problem of predicting future pedestrian trajectories in a first person view setting with a moving camera. We propose a novel action-based contrastive learning loss, that utilizes pedestrian action information to improve the learned trajectory embeddings.
  arXiv  Detail & Related papers  (2022-07-18T15:02:27Z)
• Investigating Pose Representations and Motion Contexts Modeling for 3D Motion Prediction [63.62263239934777]
  We conduct an indepth study on various pose representations with a focus on their effects on the motion prediction task. We propose a novel RNN architecture termed AHMR (Attentive Hierarchical Motion Recurrent network) for motion prediction. Our approach outperforms the state-of-the-art methods in short-term prediction and achieves much enhanced long-term prediction proficiency.
  arXiv  Detail & Related papers  (2021-12-30T10:45:22Z)
• Congestion-aware Multi-agent Trajectory Prediction for Collision Avoidance [110.63037190641414]
  We propose to learn congestion patterns explicitly and devise a novel "Sense--Learn--Reason--Predict" framework. By decomposing the learning phases into two stages, a "student" can learn contextual cues from a "teacher" while generating collision-free trajectories. In experiments, we demonstrate that the proposed model is able to generate collision-free trajectory predictions in a synthetic dataset.
  arXiv  Detail & Related papers  (2021-03-26T02:42:33Z)
• Adversarial Refinement Network for Human Motion Prediction [61.50462663314644]
  Two popular methods, recurrent neural networks and feed-forward deep networks, are able to predict rough motion trend. We propose an Adversarial Refinement Network (ARNet) following a simple yet effective coarse-to-fine mechanism with novel adversarial error augmentation.
  arXiv  Detail & Related papers  (2020-11-23T05:42:20Z)
• Pedestrian Trajectory Prediction with Convolutional Neural Networks [0.3787359747190393]
  We propose a new approach to pedestrian trajectory prediction, with the introduction of a novel 2D convolutional model. This new model outperforms recurrent models, and it achieves state-of-the-art results on the ETH and TrajNet datasets. We also present an effective system to represent pedestrian positions and powerful data augmentation techniques.
  arXiv  Detail & Related papers  (2020-10-12T15:51:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.