Benchmarking PhD-Level Coding in 3D Geometric Computer Vision

• URL: http://arxiv.org/abs/2603.30038v1
• Date: Tue, 31 Mar 2026 17:50:55 GMT
• Title: Benchmarking PhD-Level Coding in 3D Geometric Computer Vision
• Abstract summary: We introduce GeoCodeBench, a benchmark that evaluates coding for 3D vision.<n>We evaluate eight representative open- and closed-source models to reflect the current ecosystem.<n>The best model, GPT-5, attains only 36.6% pass rate.
• Score: 43.42170296855238
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: AI-assisted coding has rapidly reshaped software practice and research workflows, yet today's models still struggle to produce correct code for complex 3D geometric vision. If models could reliably write such code, the research of our community would change substantially. To measure progress toward that goal, we introduce GeoCodeBench, a PhD-level benchmark that evaluates coding for 3D vision. Each problem is a fill-in-the-function implementation task curated from representative papers at recent venues: we first let a tool propose candidate functions from official repositories, then perform careful human screening to select core 3D geometric components. For every target, we generate diverse, edge-case unit tests, enabling fully automatic, reproducible scoring. We evaluate eight representative open- and closed-source models to reflect the current ecosystem. The best model, GPT-5, attains only 36.6% pass rate, revealing a large gap between current capabilities and dependable 3D scientific coding. GeoCodeBench organizes tasks into a two-level hierarchy: General 3D capability (geometric transformations and mechanics/optics formulation) and Research capability (novel algorithm implementation and geometric logic routing). Scores are positively correlated across these axes, but research-oriented tasks are markedly harder. Context ablations further show that "more paper text" is not always better: cutting off at the Method section statistically outperforms full-paper inputs, highlighting unresolved challenges in long-context scientific comprehension. Together, these findings position GeoCodeBench as a rigorous testbed for advancing from generic coding to trustworthy 3D geometric vision coding.

Related papers

• GeoPurify: A Data-Efficient Geometric Distillation Framework for Open-Vocabulary 3D Segmentation [57.8059956428009]
  Recent attempts to transfer features from 2D Vision-Language Models to 3D semantic segmentation expose a persistent trade-off.<n>We propose GeoPurify that applies a small Student Affinity Network to 2D VLM-generated 3D point features using geometric priors distilled from a 3D self-supervised teacher model.<n>Benefiting from latent geometric information and the learned affinity network, GeoPurify effectively mitigates the trade-off and achieves superior data efficiency.
  arXiv  Detail & Related papers  (2025-10-02T16:37:56Z)
• UniUGG: Unified 3D Understanding and Generation via Geometric-Semantic Encoding [65.60549881706959]
  We introduce UniUGG, the first unified understanding and generation framework for 3D modalities.<n>Our framework employs an LLM to comprehend and decode sentences and 3D representations.<n>We propose a spatial decoder leveraging a latent diffusion model to generate high-quality 3D representations.
  arXiv  Detail & Related papers  (2025-08-16T07:27:31Z)
• SeqAffordSplat: Scene-level Sequential Affordance Reasoning on 3D Gaussian Splatting [85.87902260102652]
  We introduce the novel task of Sequential 3D Gaussian Affordance Reasoning.<n>We then propose SeqSplatNet, an end-to-end framework that directly maps an instruction to a sequence of 3D affordance masks.<n>Our method sets a new state-of-the-art on our challenging benchmark, effectively advancing affordance reasoning from single-step interactions to complex, sequential tasks at the scene level.
  arXiv  Detail & Related papers  (2025-07-31T17:56:55Z)
• E3D-Bench: A Benchmark for End-to-End 3D Geometric Foundation Models [78.1674905950243]
  We present the first comprehensive benchmark for 3D geometric foundation models (GFMs)<n>GFMs directly predict dense 3D representations in a single feed-forward pass, eliminating the need for slow or unavailable precomputed camera parameters.<n>We evaluate 16 state-of-the-art GFMs, revealing their strengths and limitations across tasks and domains.<n>All code, evaluation scripts, and processed data will be publicly released to accelerate research in 3D spatial intelligence.
  arXiv  Detail & Related papers  (2025-06-02T17:53:09Z)
• Point-PRC: A Prompt Learning Based Regulation Framework for Generalizable Point Cloud Analysis [39.85109385954641]
  Recent works demonstrate the performances of 3D point cloud recognition can be boosted remarkably by parameter-efficient prompt tuning.<n>We present a comprehensive regulation framework that allows the learnable prompts to actively interact with the well-learned general knowledge in large 3D models.<n>Surprisingly, our method not only realizes consistently increasing generalization ability but also enhances task-specific 3D recognition performances across various 3DDG benchmarks by a clear margin.
  arXiv  Detail & Related papers  (2024-10-27T10:35:47Z)
• Recovering 3D Human Mesh from Monocular Images: A Survey [49.00136388529404]
  Estimating human pose and shape from monocular images is a long-standing problem in computer vision. This survey focuses on the task of monocular 3D human mesh recovery.
  arXiv  Detail & Related papers  (2022-03-03T18:56:08Z)
• Deep Learning Based 3D Segmentation: A Survey [42.44509605101214]
  3D segmentation is a fundamental and challenging problem in computer vision with applications in autonomous driving and robotics. Deep learning techniques have recently become the tool of choice for 3D segmentation tasks. This paper comprehensively surveys the recent progress in deep learning-based 3D segmentation techniques.
  arXiv  Detail & Related papers  (2021-03-09T13:58:35Z)
• AutoML Segmentation for 3D Medical Image Data: Contribution to the MSD Challenge 2018 [2.9864637081333085]
  A 3D convolutional neural network with encoder-decoder architecture was developed and is presented in this paper. It works on anisotropic voxel-geometries and has anisotropic depth, i.e., the number of down steps is a task-specific parameter.
  arXiv  Detail & Related papers  (2020-05-20T11:47:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.