Imaging-101: Benchmarking LLM Coding Agents on Scientific Computational Imaging

• URL: http://arxiv.org/abs/2607.10789v1
• Date: Sun, 12 Jul 2026 14:42:57 GMT
• Title: Imaging-101: Benchmarking LLM Coding Agents on Scientific Computational Imaging
• Abstract summary: We introduce Imaging-101, a benchmark of 57 expert-verified computational imaging tasks spanning six scientific domains.<n>Three evaluation tracks (planning, function-level unit tests, and end-to-end reconstruction) probe distinct agent capabilities across the full pipeline.
• Score: 9.890461939894694
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Computational imaging, which recovers hidden signals from indirect, noisy measurements, underpins quantitative discovery across scientific disciplines, yet building a correct reconstruction pipeline demands deep domain expertise and remains laborious even for domain scientists. We introduce Imaging-101, a benchmark of 57 expert-verified computational imaging tasks spanning six scientific domains, each grounded in a peer-reviewed paper and canonicalized into a standardized four-stage pipeline (preprocessing, forward physics modeling, inverse solver, and visualization) Three evaluation tracks (planning, function-level unit tests, and end-to-end reconstruction) probe distinct agent capabilities across the full pipeline. Evaluating seven frontier LLMs uncovers systematic challenges in applying coding agents to computational imaging that go beyond those exposed by general coding benchmarks, spanning algorithm selection, physical convention handling, and pipeline integration. These findings highlight concrete capability gaps and point toward skill-augmented, domain-specialized agents as a practical path to reliable computational imaging assistance.

Related papers

• Does AI Understand Imaging? A Systematic Benchmark of Agentic AI for Computational Imaging Tasks [3.146480929844211]
  Vision-language models (VLMs) and agentic AI have shown strong performance on semantic visual tasks.<n>It remains unclear whether they can handle the physics and inverse problems that underlie computational imaging.<n>We present ImagingBench, a benchmark of 20 computational imaging tasks spanning five categories.
  arXiv  Detail & Related papers  (2026-07-08T09:23:56Z)
• SpatialWorld: Benchmarking Interactive Spatial Reasoning of Multimodal Agents in Real-World Tasks [73.92333717662558]
  SpatialWorld is a benchmark for evaluating the interactive spatial understanding of multimodal agents in complex real-world tasks.<n>It features 760 human-annotated tasks across diverse domains (e.g., household routines, travel, social collaboration)<n>For reliable evaluation, each task includes a human-validated initial state, a reference trajectory, and a terminal-state verifier.
  arXiv  Detail & Related papers  (2026-06-08T15:51:51Z)
• Can Coding Agents Reproduce Findings in Computational Materials Science? [49.254975563645786]
  We present AutoMat, a benchmark for evaluating large language models' ability to reproduce scientific claims.<n>By working closely with subject matter experts, we curate a set of claims from real materials science papers to test whether coding agents can recover and execute the end-to-end workflow.<n>Results show that current LLM-based agents obtain low overall success rates on AutoMat, with the best-performing setting achieving a success rate of only 54.1%.
  arXiv  Detail & Related papers  (2026-05-01T17:42:12Z)
• Accelerating 3D Photoacoustic Computed Tomography with End-to-End Physics-Aware Neural Operators [74.65171736966131]
  Photoacoustic computed tomography (PACT) combines optical contrast with ultrasonic resolution, achieving deep-tissue imaging beyond the optical diffusion limit.<n>Current implementations require dense transducer arrays and prolonged acquisition times, limiting clinical translation.<n>We introduce Pano, an end-to-end physics-aware model that directly learns the inverse acoustic mapping from sensor measurements to volumetric reconstructions.
  arXiv  Detail & Related papers  (2025-09-11T23:12:55Z)
• Bridging the Gap Between Ideal and Real-world Evaluation: Benchmarking AI-Generated Image Detection in Challenging Scenarios [54.07895223545793]
  This paper introduces the Real-World Robustness dataset (RRDataset) for comprehensive evaluation of detection models across three dimensions.<n>RRDataset includes high-quality images from seven major scenarios.<n>We benchmarked 17 detectors and 10 vision-language models (VLMs) on RRDataset and conducted a large-scale human study.
  arXiv  Detail & Related papers  (2025-09-11T06:15:52Z)
• Computational Imaging for Enhanced Computer Vision [0.0]
  This paper presents a comprehensive survey of computational imaging (CI) techniques and their transformative impact on computer vision (CV) applications.<n> Conventional imaging methods fail to deliver high-fidelity visual data in challenging conditions, such as low light, motion blur, or high dynamic range.<n> Computational imaging techniques, including light field imaging, high dynamic range (blurring) imaging, deblurring, high-speed imaging, and glare mitigation, address these limitations.
  arXiv  Detail & Related papers  (2025-09-10T16:02:42Z)
• Rewards-based image analysis in microscopy [2.906546126874626]
  Analyzing imaging and hyperspectral data is crucial across scientific fields, including biology, medicine, chemistry, and physics.<n>Currently, this task relies on complex, human-designed iterative steps such as denoising, spatial sampling, keypoint detection, feature generation, clustering, dimensionality reduction, and physics-based deconvolutions.<n>The introduction of machine learning over the past decade has accelerated tasks like image segmentation and object detection via supervised learning, and dimensionality reduction via unsupervised methods.<n>Here, we discuss advances in reward-based, which adopt expert decision-making principles and demonstrate strong transfer learning across
  arXiv  Detail & Related papers  (2025-02-23T19:19:38Z)
• Estimating Task-based Performance Bounds for Accelerated MRI Image Reconstruction Methods by Use of Learned-Ideal Observers [7.765750378590293]
  The performance of the ideal observer (IO) acting on imaging measurements has long been advocated as a figure-of-merit to guide the optimization of imaging systems.<n> estimation of IO performance can provide valuable guidance when designing under-sampled data-acquisition techniques.
  arXiv  Detail & Related papers  (2025-01-16T01:09:30Z)
• High-Resolution Vision Transformers for Pixel-Level Identification of Structural Components and Damage [1.8923948104852863]
  We develop a semantic segmentation network based on vision transformers and Laplacian pyramids scaling networks. The proposed framework has been evaluated through comprehensive experiments on a dataset of bridge inspection report images.
  arXiv  Detail & Related papers  (2023-08-06T03:34:25Z)
• Metrics reloaded: Recommendations for image analysis validation [59.60445111432934]
  Metrics Reloaded is a comprehensive framework guiding researchers in the problem-aware selection of metrics. The framework was developed in a multi-stage Delphi process and is based on the novel concept of a problem fingerprint. Based on the problem fingerprint, users are guided through the process of choosing and applying appropriate validation metrics.
  arXiv  Detail & Related papers  (2022-06-03T15:56:51Z)
• Gradient-Induced Co-Saliency Detection [81.54194063218216]
  Co-saliency detection (Co-SOD) aims to segment the common salient foreground in a group of relevant images. In this paper, inspired by human behavior, we propose a gradient-induced co-saliency detection method.
  arXiv  Detail & Related papers  (2020-04-28T08:40:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.