First and Second Order Dynamics in a Hierarchical SOM system for Action Recognition

• URL: http://arxiv.org/abs/2104.06059v1
• Date: Tue, 13 Apr 2021 09:46:40 GMT
• Title: First and Second Order Dynamics in a Hierarchical SOM system for Action Recognition
• Authors: Zahra Gharaee and Peter G\"ardenfors and Magnus Johnsson
• Abstract summary: We present a novel action recognition system that employs a hierarchy of Self-Organizing Maps together with a custom supervised neural network that learns to categorize actions. The system preprocesses the input from a Kinect like 3D camera to exploit the information not only about joint positions, but also their first and second order dynamics.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Human recognition of the actions of other humans is very efficient and is based on patterns of movements. Our theoretical starting point is that the dynamics of the joint movements is important to action categorization. On the basis of this theory, we present a novel action recognition system that employs a hierarchy of Self-Organizing Maps together with a custom supervised neural network that learns to categorize actions. The system preprocesses the input from a Kinect like 3D camera to exploit the information not only about joint positions, but also their first and second order dynamics. We evaluate our system in two experiments with publicly available data sets, and compare its performance to the performance with less sophisticated preprocessing of the input. The results show that including the dynamics of the actions improves the performance. We also apply an attention mechanism that focuses on the parts of the body that are the most involved in performing the actions.

Related papers

• Intrinsic Motivation in Dynamical Control Systems [5.635628182420597]
  We investigate an information-theoretic approach to intrinsic motivation, based on maximizing an agent's empowerment. We show that this approach generalizes previous attempts to formalize intrinsic motivation. This opens the door for designing practical artificial, intrinsically motivated controllers.
  arXiv  Detail & Related papers  (2022-12-29T05:20:08Z)
• Learning Action-Effect Dynamics for Hypothetical Vision-Language Reasoning Task [50.72283841720014]
  We propose a novel learning strategy that can improve reasoning about the effects of actions. We demonstrate the effectiveness of our proposed approach and discuss its advantages over previous baselines in terms of performance, data efficiency, and generalization capability.
  arXiv  Detail & Related papers  (2022-12-07T05:41:58Z)
• Joint-bone Fusion Graph Convolutional Network for Semi-supervised Skeleton Action Recognition [65.78703941973183]
  We propose a novel correlation-driven joint-bone fusion graph convolutional network (CD-JBF-GCN) as an encoder and use a pose prediction head as a decoder. Specifically, the CD-JBF-GC can explore the motion transmission between the joint stream and the bone stream. The pose prediction based auto-encoder in the self-supervised training stage allows the network to learn motion representation from unlabeled data.
  arXiv  Detail & Related papers  (2022-02-08T16:03:15Z)
• Skeleton-Based Mutually Assisted Interacted Object Localization and Human Action Recognition [111.87412719773889]
  We propose a joint learning framework for "interacted object localization" and "human action recognition" based on skeleton data. Our method achieves the best or competitive performance with the state-of-the-art methods for human action recognition.
  arXiv  Detail & Related papers  (2021-10-28T10:09:34Z)
• Human Activity Recognition using Attribute-Based Neural Networks and Context Information [61.67246055629366]
  We consider human activity recognition (HAR) from wearable sensor data in manual-work processes. We show how context information can be integrated systematically into a deep neural network-based HAR system. We empirically show that our proposed architecture increases HAR performance, compared to state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-10-28T06:08:25Z)
• Action in Mind: A Neural Network Approach to Action Recognition and Segmentation [0.0]
  This thesis presents a novel computational approach for human action recognition through different implementations of multi-layer architectures based on artificial neural networks. The proposed action recognition architecture is composed of several processing layers including a preprocessing layer, an ordered vector representation layer and three layers of neural networks. For each level of development the system is trained with the input data consisting of consecutive 3D body postures and tested with generalized input data that the system has never met before.
  arXiv  Detail & Related papers  (2021-04-30T09:53:28Z)
• Self-supervised Video Object Segmentation by Motion Grouping [79.13206959575228]
  We develop a computer vision system able to segment objects by exploiting motion cues. We introduce a simple variant of the Transformer to segment optical flow frames into primary objects and the background. We evaluate the proposed architecture on public benchmarks (DAVIS2016, SegTrackv2, and FBMS59)
  arXiv  Detail & Related papers  (2021-04-15T17:59:32Z)
• TRiPOD: Human Trajectory and Pose Dynamics Forecasting in the Wild [77.59069361196404]
  TRiPOD is a novel method for predicting body dynamics based on graph attentional networks. To incorporate a real-world challenge, we learn an indicator representing whether an estimated body joint is visible/invisible at each frame. Our evaluation shows that TRiPOD outperforms all prior work and state-of-the-art specifically designed for each of the trajectory and pose forecasting tasks.
  arXiv  Detail & Related papers  (2021-04-08T20:01:00Z)
• Attention-Oriented Action Recognition for Real-Time Human-Robot Interaction [11.285529781751984]
  We propose an attention-oriented multi-level network framework to meet the need for real-time interaction. Specifically, a Pre-Attention network is employed to roughly focus on the interactor in the scene at low resolution. The other compact CNN receives the extracted skeleton sequence as input for action recognition.
  arXiv  Detail & Related papers  (2020-07-02T12:41:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.