ShapeY: Measuring Shape Recognition Capacity Using Nearest Neighbor Matching

• URL: http://arxiv.org/abs/2111.08174v1
• Date: Tue, 16 Nov 2021 01:21:54 GMT
• Title: ShapeY: Measuring Shape Recognition Capacity Using Nearest Neighbor Matching
• Abstract summary: We have developed a new approach to measuring the shape recognition performance of a vision system based on nearest neighbor view matching within the system's embedding space. Our performance benchmark, ShapeY, allows for precise control of task difficulty, by enforcing that view matching span a specified degree of 3D viewpoint change and/or appearance change.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Object recognition in humans depends primarily on shape cues. We have developed a new approach to measuring the shape recognition performance of a vision system based on nearest neighbor view matching within the system's embedding space. Our performance benchmark, ShapeY, allows for precise control of task difficulty, by enforcing that view matching span a specified degree of 3D viewpoint change and/or appearance change. As a first test case we measured the performance of ResNet50 pre-trained on ImageNet. Matching error rates were high. For example, a 27 degree change in object pitch led ResNet50 to match the incorrect object 45% of the time. Appearance changes were also highly disruptive. Examination of false matches indicates that ResNet50's embedding space is severely "tangled". These findings suggest ShapeY can be a useful tool for charting the progress of artificial vision systems towards human-level shape recognition capabilities.

Related papers

• ShapeY: A Principled Framework for Measuring Shape Recognition Capacity via Nearest-Neighbor Matching [0.0]
  Object recognition in humans relies heavily on shape cues and the ability to recognize objects across varying 3D viewpoints.<n>Unlike humans, deep networks often rely on non-shape cues such as texture and background, leading to vulnerabilities in generalization and robustness.<n>We introduce ShapeY, a novel and principled benchmarking framework designed to evaluate shape-based recognition capability in OR systems.
  arXiv  Detail & Related papers  (2026-04-27T23:42:52Z)
• Leveraging Geometric Priors for Unaligned Scene Change Detection [53.523333385654546]
  Unaligned Scene Change Detection aims to detect scene changes between image pairs captured at different times without assuming viewpoint alignment.<n>We introduce geometric priors for the first time to address the core challenges of unaligned SCD.<n>We propose a training-free framework that integrates them with the powerful representations of a visual foundation model.
  arXiv  Detail & Related papers  (2025-09-14T14:31:08Z)
• Geometric-aware Pretraining for Vision-centric 3D Object Detection [77.7979088689944]
  We propose a novel geometric-aware pretraining framework called GAPretrain. GAPretrain serves as a plug-and-play solution that can be flexibly applied to multiple state-of-the-art detectors. We achieve 46.2 mAP and 55.5 NDS on the nuScenes val set using the BEVFormer method, with a gain of 2.7 and 2.1 points, respectively.
  arXiv  Detail & Related papers  (2023-04-06T14:33:05Z)
• Enhancing Deformable Local Features by Jointly Learning to Detect and Describe Keypoints [8.390939268280235]
  Local feature extraction is a standard approach in computer vision for tackling important tasks such as image matching and retrieval. We propose DALF, a novel deformation-aware network for jointly detecting and describing keypoints. Our approach also enhances the performance of two real-world applications: deformable object retrieval and non-rigid 3D surface registration.
  arXiv  Detail & Related papers  (2023-04-02T18:01:51Z)
• PCV: A Point Cloud-Based Network Verifier [8.239631885389382]
  We describe a point cloud-based network verifier that successfully deals state of the art 3D PointNet. We calculate the impact on model accuracy versus property factor and can test PointNet network's robustness against a small collection of perturbing input states.
  arXiv  Detail & Related papers  (2023-01-27T15:58:54Z)
• AGO-Net: Association-Guided 3D Point Cloud Object Detection Network [86.10213302724085]
  We propose a novel 3D detection framework that associates intact features for objects via domain adaptation. We achieve new state-of-the-art performance on the KITTI 3D detection benchmark in both accuracy and speed.
  arXiv  Detail & Related papers  (2022-08-24T16:54:38Z)
• Progressive Coordinate Transforms for Monocular 3D Object Detection [52.00071336733109]
  We propose a novel and lightweight approach, dubbed em Progressive Coordinate Transforms (PCT) to facilitate learning coordinate representations. In this paper, we propose a novel and lightweight approach, dubbed em Progressive Coordinate Transforms (PCT) to facilitate learning coordinate representations.
  arXiv  Detail & Related papers  (2021-08-12T15:22:33Z)
• Location-Sensitive Visual Recognition with Cross-IOU Loss [177.86369890708457]
  This paper proposes a unified solution named location-sensitive network (LSNet) for object detection, instance segmentation, and pose estimation. Based on a deep neural network as the backbone, LSNet predicts an anchor point and a set of landmarks which together define the shape of the target object.
  arXiv  Detail & Related papers  (2021-04-11T02:17:14Z)
• Diverse Plausible Shape Completions from Ambiguous Depth Images [7.652701739127332]
  PSSNet is a network architecture for generating plausible 3D reconstructions from a single 2.5D depth image. We perform experiments using Shapenet mugs and partially-occluded YCB objects and find that our method performs comparably in datasets with little ambiguity.
  arXiv  Detail & Related papers  (2020-11-18T16:42:51Z)
• GSNet: Joint Vehicle Pose and Shape Reconstruction with Geometrical and Scene-aware Supervision [65.13980934546957]
  We present a novel end-to-end framework named as GSNet (Geometric and Scene-aware Network) It jointly estimates 6DoF poses and reconstructs detailed 3D car shapes from single urban street view. We evaluate GSNet on the largest multi-task ApolloCar3D benchmark and achieve state-of-the-art performance both quantitatively and qualitatively.
  arXiv  Detail & Related papers  (2020-07-26T13:05:55Z)
• Disp R-CNN: Stereo 3D Object Detection via Shape Prior Guided Instance Disparity Estimation [51.17232267143098]
  We propose a novel system named Disp R-CNN for 3D object detection from stereo images. We use a statistical shape model to generate dense disparity pseudo-ground-truth without the need of LiDAR point clouds. Experiments on the KITTI dataset show that, even when LiDAR ground-truth is not available at training time, Disp R-CNN achieves competitive performance and outperforms previous state-of-the-art methods by 20% in terms of average precision.
  arXiv  Detail & Related papers  (2020-04-07T17:48:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.