The Multi-Agent Behavior Dataset: Mouse Dyadic Social Interactions

• URL: http://arxiv.org/abs/2104.02710v2
• Date: Wed, 7 Apr 2021 16:16:29 GMT
• Title: The Multi-Agent Behavior Dataset: Mouse Dyadic Social Interactions
• Authors: Jennifer J. Sun, Tomomi Karigo, Dipam Chakraborty, Sharada P. Mohanty, David J. Anderson, Pietro Perona, Yisong Yue, Ann Kennedy
• Abstract summary: We present a multi-agent dataset from behavioral neuroscience, the Caltech Mouse Social Interactions (CalMS21) dataset. Our dataset consists of trajectory data of social interactions, recorded from videos of freely behaving mice in a standard resident-intruder assay. The CalMS21 dataset is part of the Multi-Agent Behavior Challenge 2021 and for our next step, our goal is to incorporate datasets from other domains studying multi-agent behavior.
• Score: 39.265388879471686
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multi-agent behavior modeling aims to understand the interactions that occur between agents. We present a multi-agent dataset from behavioral neuroscience, the Caltech Mouse Social Interactions (CalMS21) Dataset. Our dataset consists of trajectory data of social interactions, recorded from videos of freely behaving mice in a standard resident-intruder assay. The CalMS21 dataset is part of the Multi-Agent Behavior Challenge 2021 and for our next step, our goal is to incorporate datasets from other domains studying multi-agent behavior. To help accelerate behavioral studies, the CalMS21 dataset provides a benchmark to evaluate the performance of automated behavior classification methods in three settings: (1) for training on large behavioral datasets all annotated by a single annotator, (2) for style transfer to learn inter-annotator differences in behavior definitions, and (3) for learning of new behaviors of interest given limited training data. The dataset consists of 6 million frames of unlabelled tracked poses of interacting mice, as well as over 1 million frames with tracked poses and corresponding frame-level behavior annotations. The challenge of our dataset is to be able to classify behaviors accurately using both labelled and unlabelled tracking data, as well as being able to generalize to new annotators and behaviors.

Related papers

• From Forest to Zoo: Great Ape Behavior Recognition with ChimpBehave [0.0]
  We introduce ChimpBehave, a novel dataset featuring over 2 hours of video (approximately 193,000 video frames) of zoo-housed chimpanzees. ChimpBehave meticulously annotated with bounding boxes and behavior labels for action recognition. We benchmark our dataset using a state-of-the-art CNN-based action recognition model.
  arXiv  Detail & Related papers  (2024-05-30T13:11:08Z)
• Learning Mutual Excitation for Hand-to-Hand and Human-to-Human Interaction Recognition [22.538114033191313]
  We propose a mutual excitation graph convolutional network (me-GCN) by stacking mutual excitation graph convolution layers. Me-GC learns mutual information in each layer and each stage of graph convolution operations. Our proposed me-GC outperforms state-of-the-art GCN-based and Transformer-based methods.
  arXiv  Detail & Related papers  (2024-02-04T10:00:00Z)
• Towards a Unified Transformer-based Framework for Scene Graph Generation and Human-object Interaction Detection [116.21529970404653]
  We introduce SG2HOI+, a unified one-step model based on the Transformer architecture. Our approach employs two interactive hierarchical Transformers to seamlessly unify the tasks of SGG and HOI detection. Our approach achieves competitive performance when compared to state-of-the-art HOI methods.
  arXiv  Detail & Related papers  (2023-11-03T07:25:57Z)
• Coarse-to-Fine Knowledge-Enhanced Multi-Interest Learning Framework for Multi-Behavior Recommendation [52.89816309759537]
  Multi-types of behaviors (e.g., clicking, adding to cart, purchasing, etc.) widely exist in most real-world recommendation scenarios. The state-of-the-art multi-behavior models learn behavior dependencies indistinguishably with all historical interactions as input. We propose a novel Coarse-to-fine Knowledge-enhanced Multi-interest Learning framework to learn shared and behavior-specific interests for different behaviors.
  arXiv  Detail & Related papers  (2022-08-03T05:28:14Z)
• Incorporating Heterogeneous User Behaviors and Social Influences for Predictive Analysis [32.31161268928372]
  We aim to incorporate heterogeneous user behaviors and social influences for behavior predictions. This paper proposes a variant of Long-Short Term Memory (LSTM) which can consider context while a behavior sequence. A residual learning-based decoder is designed to automatically construct multiple high-order cross features based on social behavior representation.
  arXiv  Detail & Related papers  (2022-07-24T17:05:37Z)
• MABe22: A Multi-Species Multi-Task Benchmark for Learned Representations of Behavior [28.878568752724235]
  We introduce MABe22, a benchmark to assess the quality of learned behavior representations. This dataset is collected from a variety of biology experiments. We test self-supervised video and trajectory representation learning methods to demonstrate the use of our benchmark.
  arXiv  Detail & Related papers  (2022-07-21T15:51:30Z)
• Learning from Temporal Spatial Cubism for Cross-Dataset Skeleton-based Action Recognition [88.34182299496074]
  Action labels are only available on a source dataset, but unavailable on a target dataset in the training stage. We utilize a self-supervision scheme to reduce the domain shift between two skeleton-based action datasets. By segmenting and permuting temporal segments or human body parts, we design two self-supervised learning classification tasks.
  arXiv  Detail & Related papers  (2022-07-17T07:05:39Z)
• SMEMO: Social Memory for Trajectory Forecasting [34.542209630734234]
  We present a neural network based on an end-to-end trainable working memory, which acts as an external storage. We show that our method is capable of learning explainable cause-effect relationships between motions of different agents, obtaining state-of-the-art results on trajectory forecasting datasets.
  arXiv  Detail & Related papers  (2022-03-23T14:40:20Z)
• MSeg: A Composite Dataset for Multi-domain Semantic Segmentation [100.17755160696939]
  We present MSeg, a composite dataset that unifies semantic segmentation datasets from different domains. We reconcile the generalization and bring the pixel-level annotations into alignment by relabeling more than 220,000 object masks in more than 80,000 images. A model trained on MSeg ranks first on the WildDash-v1 leaderboard for robust semantic segmentation, with no exposure to WildDash data during training.
  arXiv  Detail & Related papers  (2021-12-27T16:16:35Z)
• Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Dataset [84.3946567650148]
  With over 100,000 scenes, each 20 seconds long at 10 Hz, our new dataset contains more than 570 hours of unique data over 1750 km of roadways. We use a high-accuracy 3D auto-labeling system to generate high quality 3D bounding boxes for each road agent. We introduce a new set of metrics that provides a comprehensive evaluation of both single agent and joint agent interaction motion forecasting models.
  arXiv  Detail & Related papers  (2021-04-20T17:19:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.