Towards Open-World Gesture Recognition

• URL: http://arxiv.org/abs/2401.11144v2
• Date: Sat, 05 Oct 2024 05:54:59 GMT
• Title: Towards Open-World Gesture Recognition
• Authors: Junxiao Shen, Matthias De Lange, Xuhai "Orson" Xu, Enmin Zhou, Ran Tan, Naveen Suda, Maciej Lazarewicz, Per Ola Kristensson, Amy Karlson, Evan Strasnick,
• Abstract summary: In real-world applications involving gesture recognition, such as gesture recognition based on wrist-worn devices, the data distribution may change over time. We propose the use of continual learning to enable machine learning models to be adaptive to new tasks. We provide design guidelines to enhance the development of an open-world wrist-worn gesture recognition process.
• Score: 19.019579924491847
• License:
• Abstract: Providing users with accurate gestural interfaces, such as gesture recognition based on wrist-worn devices, is a key challenge in mixed reality. However, static machine learning processes in gesture recognition assume that training and test data come from the same underlying distribution. Unfortunately, in real-world applications involving gesture recognition, such as gesture recognition based on wrist-worn devices, the data distribution may change over time. We formulate this problem of adapting recognition models to new tasks, where new data patterns emerge, as open-world gesture recognition (OWGR). We propose the use of continual learning to enable machine learning models to be adaptive to new tasks without degrading performance on previously learned tasks. However, the process of exploring parameters for questions around when, and how, to train and deploy recognition models requires resource-intensive user studies may be impractical. To address this challenge, we propose a design engineering approach that enables offline analysis on a collected large-scale dataset by systematically examining various parameters and comparing different continual learning methods. Finally, we provide design guidelines to enhance the development of an open-world wrist-worn gesture recognition process.

Related papers

• Zero-Shot Underwater Gesture Recognition [3.4078654008228924]
  Hand gesture recognition allows humans to interact with machines non-verbally, which has a huge application in underwater exploration using autonomous underwater vehicles. Recently, a new gesture-based language called CADDIAN has been devised for divers, and supervised learning methods have been applied to recognize the gestures with high accuracy. In this work, we advocate the need for zero-shot underwater gesture recognition (ZSUGR), where the objective is to train a model with visual samples of gestures from a few seen'' classes only and transfer the gained knowledge at test time to recognize semantically-similar unseen gesture classes as well.
  arXiv  Detail & Related papers  (2024-07-19T08:16:46Z)
• Deep self-supervised learning with visualisation for automatic gesture recognition [1.6647755388646919]
  Gesture is an important mean of non-verbal communication, with visual modality allows human to convey information during interaction, facilitating peoples and human-machine interactions. In this work, we explore three different means to recognise hand signs using deep learning: supervised learning based methods, self-supervised methods and visualisation based techniques applied to 3D moving skeleton data.
  arXiv  Detail & Related papers  (2024-06-18T09:44:55Z)
• Continuous Learning Based Novelty Aware Emotion Recognition System [0.0]
  Current works in human emotion recognition follow the traditional closed learning approach governed by rigid rules without any consideration of novelty. In this work, we propose a continuous learning based approach to deal with novelty in the automatic emotion recognition task.
  arXiv  Detail & Related papers  (2023-06-14T20:34:07Z)
• Continual Learning from Demonstration of Robotics Skills [5.573543601558405]
  Methods for teaching motion skills to robots focus on training for a single skill at a time. We propose an approach for continual learning from demonstration using hypernetworks and neural ordinary differential equation solvers.
  arXiv  Detail & Related papers  (2022-02-14T16:26:52Z)
• Dynamic Modeling of Hand-Object Interactions via Tactile Sensing [133.52375730875696]
  In this work, we employ a high-resolution tactile glove to perform four different interactive activities on a diversified set of objects. We build our model on a cross-modal learning framework and generate the labels using a visual processing pipeline to supervise the tactile model. This work takes a step on dynamics modeling in hand-object interactions from dense tactile sensing.
  arXiv  Detail & Related papers  (2021-09-09T16:04:14Z)
• What Matters in Learning from Offline Human Demonstrations for Robot Manipulation [64.43440450794495]
  We conduct an extensive study of six offline learning algorithms for robot manipulation. Our study analyzes the most critical challenges when learning from offline human data. We highlight opportunities for learning from human datasets.
  arXiv  Detail & Related papers  (2021-08-06T20:48:30Z)
• SHREC 2021: Track on Skeleton-based Hand Gesture Recognition in the Wild [62.450907796261646]
  Recognition of hand gestures can be performed directly from the stream of hand skeletons estimated by software. Despite the recent advancements in gesture and action recognition from skeletons, it is unclear how well the current state-of-the-art techniques can perform in a real-world scenario. This paper presents the results of the SHREC 2021: Track on Skeleton-based Hand Gesture Recognition in the Wild contest.
  arXiv  Detail & Related papers  (2021-06-21T10:57:49Z)
• Gesture Similarity Analysis on Event Data Using a Hybrid Guided Variational Auto Encoder [3.1148846501645084]
  We propose a neuromorphic gesture analysis system which naturally declutters the background and analyzes gestures at high temporal resolution. Our results show that the features learned by the VAE provides a similarity measure capable of clustering and pseudo labeling of new gestures.
  arXiv  Detail & Related papers  (2021-03-31T23:58:34Z)
• Relational Graph Learning on Visual and Kinematics Embeddings for Accurate Gesture Recognition in Robotic Surgery [84.73764603474413]
  We propose a novel online approach of multi-modal graph network (i.e., MRG-Net) to dynamically integrate visual and kinematics information. The effectiveness of our method is demonstrated with state-of-the-art results on the public JIGSAWS dataset.
  arXiv  Detail & Related papers  (2020-11-03T11:00:10Z)
• Visual Imitation Made Easy [102.36509665008732]
  We present an alternate interface for imitation that simplifies the data collection process while allowing for easy transfer to robots. We use commercially available reacher-grabber assistive tools both as a data collection device and as the robot's end-effector. We experimentally evaluate on two challenging tasks: non-prehensile pushing and prehensile stacking, with 1000 diverse demonstrations for each task.
  arXiv  Detail & Related papers  (2020-08-11T17:58:50Z)
• A System for Real-Time Interactive Analysis of Deep Learning Training [66.06880335222529]
  Currently available systems are limited to monitoring only the logged data that must be specified before the training process starts. We present a new system that enables users to perform interactive queries on live processes generating real-time information.
  arXiv  Detail & Related papers  (2020-01-05T11:33:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.