Meta-Point Learning and Refining for Category-Agnostic Pose Estimation

• URL: http://arxiv.org/abs/2403.13647v1
• Date: Wed, 20 Mar 2024 14:54:33 GMT
• Title: Meta-Point Learning and Refining for Category-Agnostic Pose Estimation
• Authors: Junjie Chen, Jiebin Yan, Yuming Fang, Li Niu,
• Abstract summary: Category-agnostic pose estimation (CAPE) aims to predict keypoints for arbitrary classes given a few support images annotated with keypoints. We propose a novel framework for CAPE based on such potential keypoints (named as meta-points)
• Score: 46.98479393474727
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Category-agnostic pose estimation (CAPE) aims to predict keypoints for arbitrary classes given a few support images annotated with keypoints. Existing methods only rely on the features extracted at support keypoints to predict or refine the keypoints on query image, but a few support feature vectors are local and inadequate for CAPE. Considering that human can quickly perceive potential keypoints of arbitrary objects, we propose a novel framework for CAPE based on such potential keypoints (named as meta-points). Specifically, we maintain learnable embeddings to capture inherent information of various keypoints, which interact with image feature maps to produce meta-points without any support. The produced meta-points could serve as meaningful potential keypoints for CAPE. Due to the inevitable gap between inherency and annotation, we finally utilize the identities and details offered by support keypoints to assign and refine meta-points to desired keypoints in query image. In addition, we propose a progressive deformable point decoder and a slacked regression loss for better prediction and supervision. Our novel framework not only reveals the inherency of keypoints but also outperforms existing methods of CAPE. Comprehensive experiments and in-depth studies on large-scale MP-100 dataset demonstrate the effectiveness of our framework.

Related papers

• GMM-IKRS: Gaussian Mixture Models for Interpretable Keypoint Refinement and Scoring [9.322937309882022]
  Keypoints come with a score permitting to rank them according to their quality. While learned keypoints often exhibit better properties than handcrafted ones, their scores are not easily interpretable. We propose a framework that can refine, and at the same time characterize with an interpretable score, the keypoints extracted by any method.
  arXiv  Detail & Related papers  (2024-08-30T09:39:59Z)
• SCAPE: A Simple and Strong Category-Agnostic Pose Estimator [6.705257644513057]
  Category-Agnostic Pose Estimation (CAPE) aims to localize keypoints on an object of any category given few exemplars in an in-context manner. We introduce two key modules: a global keypoint feature perceptor to inject global semantic information into support keypoints, and a keypoint attention refiner to enhance inter-node correlation between keypoints. SCAPE outperforms prior arts by 2.2 and 1.3 PCK under 1-shot and 5-shot settings with faster inference speed and lighter model capacity.
  arXiv  Detail & Related papers  (2024-07-18T13:02:57Z)
• Open-Vocabulary Animal Keypoint Detection with Semantic-feature Matching [74.75284453828017]
  Open-Vocabulary Keypoint Detection (OVKD) task is innovatively designed to use text prompts for identifying arbitrary keypoints across any species. We have developed a novel framework named Open-Vocabulary Keypoint Detection with Semantic-feature Matching (KDSM) This framework combines vision and language models, creating an interplay between language features and local keypoint visual features.
  arXiv  Detail & Related papers  (2023-10-08T07:42:41Z)
• SC3K: Self-supervised and Coherent 3D Keypoints Estimation from Rotated, Noisy, and Decimated Point Cloud Data [17.471342278936365]
  We propose a new method to infer keypoints from arbitrary object categories in practical scenarios where point cloud data (PCD) are noisy, down-sampled and arbitrarily rotated. We achieve these desiderata by proposing a new self-supervised training strategy for keypoints estimation. We compare the keypoints estimated by the proposed approach with those of the state-of-the-art unsupervised approaches.
  arXiv  Detail & Related papers  (2023-08-10T08:10:01Z)
• Few-shot Keypoint Detection with Uncertainty Learning for Unseen Species [28.307200505494126]
  We propose a versatile Few-shot Keypoint Detection (FSKD) pipeline, which can detect a varying number of keypoints of different kinds. Our FSKD involves main and auxiliary keypoint representation learning, similarity learning, and keypoint localization. We show the effectiveness of our FSKD on (i) novel keypoint detection for unseen species, and (ii) few-shot Fine-Grained Visual Recognition (FGVR) and (iii) Semantic Alignment (SA) downstream tasks.
  arXiv  Detail & Related papers  (2021-12-12T08:39:47Z)
• Accurate Grid Keypoint Learning for Efficient Video Prediction [87.71109421608232]
  Keypoint-based video prediction methods can consume substantial computing resources in training and deployment. In this paper, we design a new grid keypoint learning framework, aiming at a robust and explainable intermediate keypoint representation for long-term efficient video prediction. Our method outperforms the state-ofthe-art video prediction methods while saves 98% more than computing resources.
  arXiv  Detail & Related papers  (2021-07-28T05:04:30Z)
• End-to-End Learning of Keypoint Representations for Continuous Control from Images [84.8536730437934]
  We show that it is possible to learn efficient keypoint representations end-to-end, without the need for unsupervised pre-training, decoders, or additional losses. Our proposed architecture consists of a differentiable keypoint extractor that feeds the coordinates directly to a soft actor-critic agent.
  arXiv  Detail & Related papers  (2021-06-15T09:17:06Z)
• Bottom-Up Human Pose Estimation Via Disentangled Keypoint Regression [81.05772887221333]
  We study the dense keypoint regression framework that is previously inferior to the keypoint detection and grouping framework. We present a simple yet effective approach, named disentangled keypoint regression (DEKR) We empirically show that the proposed direct regression method outperforms keypoint detection and grouping methods.
  arXiv  Detail & Related papers  (2021-04-06T05:54:46Z)
• Keypoint Autoencoders: Learning Interest Points of Semantics [4.551313396927381]
  We propose Keypoint Autoencoder, an unsupervised learning method for detecting keypoints. We encourage selecting sparse semantic keypoints by enforcing the reconstruction from keypoints to the original point cloud. A downstream task of classifying shape with sparse keypoints is conducted to demonstrate the distinctiveness of our selected keypoints.
  arXiv  Detail & Related papers  (2020-08-11T03:43:18Z)
• Point-Set Anchors for Object Detection, Instance Segmentation and Pose Estimation [85.96410825961966]
  We argue that the image features extracted at a central point contain limited information for predicting distant keypoints or bounding box boundaries. To facilitate inference, we propose to instead perform regression from a set of points placed at more advantageous positions. We apply this proposed framework, called Point-Set Anchors, to object detection, instance segmentation, and human pose estimation.
  arXiv  Detail & Related papers  (2020-07-06T15:59:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.