GMM-IKRS: Gaussian Mixture Models for Interpretable Keypoint Refinement and Scoring

• URL: http://arxiv.org/abs/2408.17149v1
• Date: Fri, 30 Aug 2024 09:39:59 GMT
• Title: GMM-IKRS: Gaussian Mixture Models for Interpretable Keypoint Refinement and Scoring
• Authors: Emanuele Santellani, Martin Zach, Christian Sormann, Mattia Rossi, Andreas Kuhn, Friedrich Fraundorfer,
• Abstract summary: Keypoints come with a score permitting to rank them according to their quality. While learned keypoints often exhibit better properties than handcrafted ones, their scores are not easily interpretable. We propose a framework that can refine, and at the same time characterize with an interpretable score, the keypoints extracted by any method.
• Score: 9.322937309882022
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The extraction of keypoints in images is at the basis of many computer vision applications, from localization to 3D reconstruction. Keypoints come with a score permitting to rank them according to their quality. While learned keypoints often exhibit better properties than handcrafted ones, their scores are not easily interpretable, making it virtually impossible to compare the quality of individual keypoints across methods. We propose a framework that can refine, and at the same time characterize with an interpretable score, the keypoints extracted by any method. Our approach leverages a modified robust Gaussian Mixture Model fit designed to both reject non-robust keypoints and refine the remaining ones. Our score comprises two components: one relates to the probability of extracting the same keypoint in an image captured from another viewpoint, the other relates to the localization accuracy of the keypoint. These two interpretable components permit a comparison of individual keypoints extracted across different methods. Through extensive experiments we demonstrate that, when applied to popular keypoint detectors, our framework consistently improves the repeatability of keypoints as well as their performance in homography and two/multiple-view pose recovery tasks.

Related papers

• Exploring Key Point Analysis with Pairwise Generation and Graph Partitioning [61.73411954056032]
  Key Point Analysis (KPA) continues to be a significant and unresolved issue within the field of argument mining. We propose a novel approach for KPA with pairwise generation and graph partitioning.
  arXiv  Detail & Related papers  (2024-04-17T13:44:29Z)
• Visibility-Aware Keypoint Localization for 6DoF Object Pose Estimation [56.07676459156789]
  Localizing 3D keypoints in a 2D image is an effective way to establish 3D-2D correspondences for 6DoF object pose estimation. In this paper, we address this issue by localizing the important keypoints in terms of visibility. We construct VAPO (Visibility-Aware POse estimator) by integrating the visibility-aware importance with a state-of-the-art pose estimation algorithm.
  arXiv  Detail & Related papers  (2024-03-21T16:59:45Z)
• Learning Feature Matching via Matchable Keypoint-Assisted Graph Neural Network [52.29330138835208]
  Accurately matching local features between a pair of images is a challenging computer vision task. Previous studies typically use attention based graph neural networks (GNNs) with fully-connected graphs over keypoints within/across images. We propose MaKeGNN, a sparse attention-based GNN architecture which bypasses non-repeatable keypoints and leverages matchable ones to guide message passing.
  arXiv  Detail & Related papers  (2023-07-04T02:50:44Z)
• Hybrid model for Single-Stage Multi-Person Pose Estimation [3.592448408054345]
  We propose a hybrid model for single-stage multi-person pose estimation, named HybridPose. It is capable of not only detecting densely placed keypoints, but also filtering the non-existent keypoints in an image.
  arXiv  Detail & Related papers  (2023-05-02T02:55:29Z)
• Self-Supervised Equivariant Learning for Oriented Keypoint Detection [35.94215211409985]
  We introduce a self-supervised learning framework using rotation-equivariant CNNs to learn to detect robust oriented keypoints. We propose a dense orientation alignment loss by an image pair generated by synthetic transformations for training a histogram-based orientation map. Our method outperforms the previous methods on an image matching benchmark and a camera pose estimation benchmark.
  arXiv  Detail & Related papers  (2022-04-19T02:26:07Z)
• Recognition of Freely Selected Keypoints on Human Limbs [18.176606453818557]
  We use the Vision Transformer architecture to extend the capability of the model to detect arbitrary keypoints on the limbs of persons. Our approaches achieve similar results to TokenPose on the fixed keypoints and are capable of detecting arbitrary keypoints on the limbs.
  arXiv  Detail & Related papers  (2022-04-13T11:58:28Z)
• Accurate Grid Keypoint Learning for Efficient Video Prediction [87.71109421608232]
  Keypoint-based video prediction methods can consume substantial computing resources in training and deployment. In this paper, we design a new grid keypoint learning framework, aiming at a robust and explainable intermediate keypoint representation for long-term efficient video prediction. Our method outperforms the state-ofthe-art video prediction methods while saves 98% more than computing resources.
  arXiv  Detail & Related papers  (2021-07-28T05:04:30Z)
• Bottom-Up Human Pose Estimation Via Disentangled Keypoint Regression [81.05772887221333]
  We study the dense keypoint regression framework that is previously inferior to the keypoint detection and grouping framework. We present a simple yet effective approach, named disentangled keypoint regression (DEKR) We empirically show that the proposed direct regression method outperforms keypoint detection and grouping methods.
  arXiv  Detail & Related papers  (2021-04-06T05:54:46Z)
• UKPGAN: A General Self-Supervised Keypoint Detector [43.35270822722044]
  UKPGAN is a general self-supervised 3D keypoint detector. Our keypoints align well with human annotated keypoint labels. Our model is stable under both rigid and non-rigid transformations.
  arXiv  Detail & Related papers  (2020-11-24T09:08:21Z)
• Keypoint Autoencoders: Learning Interest Points of Semantics [4.551313396927381]
  We propose Keypoint Autoencoder, an unsupervised learning method for detecting keypoints. We encourage selecting sparse semantic keypoints by enforcing the reconstruction from keypoints to the original point cloud. A downstream task of classifying shape with sparse keypoints is conducted to demonstrate the distinctiveness of our selected keypoints.
  arXiv  Detail & Related papers  (2020-08-11T03:43:18Z)
• Bottom-Up Human Pose Estimation by Ranking Heatmap-Guided Adaptive Keypoint Estimates [76.51095823248104]
  We present several schemes that are rarely or unthoroughly studied before for improving keypoint detection and grouping (keypoint regression) performance. First, we exploit the keypoint heatmaps for pixel-wise keypoint regression instead of separating them for improving keypoint regression. Second, we adopt a pixel-wise spatial transformer network to learn adaptive representations for handling the scale and orientation variance. Third, we present a joint shape and heatvalue scoring scheme to promote the estimated poses that are more likely to be true poses.
  arXiv  Detail & Related papers  (2020-06-28T01:14:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.