A Locality-based Neural Solver for Optical Motion Capture

• URL: http://arxiv.org/abs/2309.00428v2
• Date: Mon, 4 Sep 2023 09:21:14 GMT
• Title: A Locality-based Neural Solver for Optical Motion Capture
• Authors: Xiaoyu Pan, Bowen Zheng, Xinwei Jiang, Guanglong Xu, Xianli Gu, Jingxiang Li, Qilong Kou, He Wang, Tianjia Shao, Kun Zhou and Xiaogang Jin
• Abstract summary: Given noisy marker data, we propose a new heterogeneous graph neural network which treats markers and joints as different types of nodes. We show that our method outperforms state-of-the-art methods in terms of prediction accuracy of occluded marker position error.
• Score: 37.28597049192196
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present a novel locality-based learning method for cleaning and solving optical motion capture data. Given noisy marker data, we propose a new heterogeneous graph neural network which treats markers and joints as different types of nodes, and uses graph convolution operations to extract the local features of markers and joints and transform them to clean motions. To deal with anomaly markers (e.g. occluded or with big tracking errors), the key insight is that a marker's motion shows strong correlations with the motions of its immediate neighboring markers but less so with other markers, a.k.a. locality, which enables us to efficiently fill missing markers (e.g. due to occlusion). Additionally, we also identify marker outliers due to tracking errors by investigating their acceleration profiles. Finally, we propose a training regime based on representation learning and data augmentation, by training the model on data with masking. The masking schemes aim to mimic the occluded and noisy markers often observed in the real data. Finally, we show that our method achieves high accuracy on multiple metrics across various datasets. Extensive comparison shows our method outperforms state-of-the-art methods in terms of prediction accuracy of occluded marker position error by approximately 20%, which leads to a further error reduction on the reconstructed joint rotations and positions by 30%. The code and data for this paper are available at https://github.com/non-void/LocalMoCap.

Related papers

• OsmLocator: locating overlapping scatter marks with a non-training generative perspective [48.50108853199417]
  Locating overlapping marks faces many difficulties such as no texture, less contextual information, hallow shape and tiny size. Here, we formulate it as a optimization problem on clustering-based re-visualization from a non-training generative perspective. We especially built a dataset named 2023 containing hundreds of scatter images with different markers and various levels of overlapping severity, and tested the proposed method and compared it to existing methods.
  arXiv  Detail & Related papers  (2023-12-18T12:39:48Z)
• ERASE: Error-Resilient Representation Learning on Graphs for Label Noise Tolerance [53.73316938815873]
  We propose a method called ERASE (Error-Resilient representation learning on graphs for lAbel noiSe tolerancE) to learn representations with error tolerance. ERASE combines prototype pseudo-labels with propagated denoised labels and updates representations with error resilience. Our method can outperform multiple baselines with clear margins in broad noise levels and enjoy great scalability.
  arXiv  Detail & Related papers  (2023-12-13T17:59:07Z)
• Domain Adaptive Synapse Detection with Weak Point Annotations [63.97144211520869]
  We present AdaSyn, a framework for domain adaptive synapse detection with weak point annotations. In the WASPSYN challenge at I SBI 2023, our method ranks the 1st place.
  arXiv  Detail & Related papers  (2023-08-31T05:05:53Z)
• Adaptive Local-Component-aware Graph Convolutional Network for One-shot Skeleton-based Action Recognition [54.23513799338309]
  We present an Adaptive Local-Component-aware Graph Convolutional Network for skeleton-based action recognition. Our method provides a stronger representation than the global embedding and helps our model reach state-of-the-art.
  arXiv  Detail & Related papers  (2022-09-21T02:33:07Z)
• NeuralMarker: A Framework for Learning General Marker Correspondence [25.822657926255573]
  We tackle the problem of estimating correspondences from a general marker, such as a movie poster, to an image that captures such a marker. We propose a novel framework NeuralMarker, training a neural network estimating dense marker correspondences under various challenging conditions. We show that NeuralMarker significantly outperforms previous methods and enables new interesting applications, including Augmented Reality (AR) and video editing.
  arXiv  Detail & Related papers  (2022-09-19T10:04:38Z)
• SparseAlign: A Super-Resolution Algorithm for Automatic Marker Localization and Deformation Estimation in Cryo-Electron Tomography [0.17398560678845074]
  We extend a grid-free super-resolution algorithm first proposed in the context of single-molecule localization microscopy. Our approach does not require labelled marker locations; instead, we use an image-based loss where we compare the forward projection of markers with the observed data. We show that our approach automatically finds markers and reliably estimates sample deformation without labelled marker data.
  arXiv  Detail & Related papers  (2022-01-21T14:03:32Z)
• SparseDet: Improving Sparsely Annotated Object Detection with Pseudo-positive Mining [76.95808270536318]
  We propose an end-to-end system that learns to separate proposals into labeled and unlabeled regions using Pseudo-positive mining. While the labeled regions are processed as usual, self-supervised learning is used to process the unlabeled regions. We conduct exhaustive experiments on five splits on the PASCAL-VOC and COCO datasets achieving state-of-the-art performance.
  arXiv  Detail & Related papers  (2022-01-12T18:57:04Z)
• Fiducial marker recovery and detection from severely truncated data in navigation assisted spine surgery [5.731349529280083]
  Fiducial markers are commonly used in navigation assisted minimally invasive spine surgery (MISS) In practice, these markers might be located outside the field-of-view due to the limited detector sizes of C-arm cone-beam computed tomography (CBCT) systems. We propose two fiducial marker detection methods: direct detection from distorted markers (direct method) and detection after marker recovery (recovery method)
  arXiv  Detail & Related papers  (2021-08-25T13:52:58Z)
• Attentional Graph Neural Network for Parking-slot Detection [10.095984750382478]
  We propose an attentional graph neural network based parking-slot detection method, which refers the marking-points in an around-view image as graph-structured data. Without any manually designed post-processing, the proposed method is end-to-end trainable. Experiments have been conducted on public benchmark dataset, where the proposed method achieves state-of-the-art accuracy.
  arXiv  Detail & Related papers  (2021-04-06T15:14:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.