Designing Any Imaging System from Natural Language: Agent-Constrained Composition over a Finite Primitive Basis

• URL: http://arxiv.org/abs/2603.25636v1
• Date: Thu, 26 Mar 2026 16:47:27 GMT
• Title: Designing Any Imaging System from Natural Language: Agent-Constrained Composition over a Finite Primitive Basis
• Abstract summary: spec.md is a structured specification format that translates a one-sentence natural-language description into a validated forward model with bounded reconstruction error.<n>A design-to-real error theorem decomposes total reconstruction error into five independently bounded terms, each linked to a corrective action.<n>Ten novel designs -- composing primitives into chains from 3D to 5D -- demonstrate compositional reach beyond any single-modality tool.
• Score: 1.7259824817932294
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Designing a computational imaging system -- selecting operators, setting parameters, validating consistency -- requires weeks of specialist effort per modality, creating an expertise bottleneck that excludes the broader scientific community from prototyping imaging instruments. We introduce spec.md, a structured specification format, and three autonomous agents -- Plan, Judge, and Execute -- that translate a one-sentence natural-language description into a validated forward model with bounded reconstruction error. A design-to-real error theorem decomposes total reconstruction error into five independently bounded terms, each linked to a corrective action. On 6 real-data modalities spanning all 5 carrier families, the automated pipeline matches expert-library quality (98.1 +/- 4.2%). Ten novel designs -- composing primitives into chains from 3D to 5D -- demonstrate compositional reach beyond any single-modality tool.

Related papers

• TRACE-Bench: Decomposing and Diagnosing Multi-Reference Image Generation [74.69068352546073]
  TRACE-Bench is an evaluation tool for multi-reference images spanning diverse artistic styles and real-world subjects.<n>It is built from 631 formula templates and around 4,000 reference images spanning diverse artistic styles and real-world subjects.<n>Its formula structure drives an operator-aligned evaluation protocol for per-capability scoring and a diagnostic tree analysis.
  arXiv  Detail & Related papers  (2026-08-17T16:15:50Z)
• Agentic Designer: Progressive Multi-Agent Collaboration for Structure-Aware Interior Layout Generation [84.5255199507027]
  Agentic Designer is a progressive framework that formulates structure-aware interior layout generation as an iterative and constraint-verified decision process.<n>By layout decomposing into modular stages of proposal, verification, synthesis and adjustment, the framework coordinates three specialized agents, a Generator, an Evaluator, and a Refiner.<n>This mechanism enforces stepwise geometric validation and correction before each placement is committed, thereby preventing error accumulation.
  arXiv  Detail & Related papers  (2026-07-23T02:46:11Z)
• OptiAgent: End-to-End Optimization Modeling via Multi-Agent Iterative Refinement [0.0]
  OptiAgent is a multi-agent framework that is able to output a solver-ready mathematical formulation as well as executable code.<n>Our architecture prioritizes the mathematical modeling step, where dedicated agents extract structures.<n>We introduce a novel multi-loop validation architecture with four specialized feedback mechanisms, each targeting a distinct failure mode.
  arXiv  Detail & Related papers  (2026-07-06T17:27:16Z)
• TO-Agents: A Multi-Agent AI Pipeline for Preference-Guided Topology Optimization [6.670146926866281]
  We present TO-Agents, a multi-agent AI framework that connects natural-language design intent with iterative topology optimization.<n>The framework converts a human-provided problem description into validated solver inputs, runs a topology optimization solver, and renders the resulting 3D topology.<n>We evaluate the framework on two long-horizon design tasks: a cantilever beam benchmark and a phone-stand product design.
  arXiv  Detail & Related papers  (2026-05-20T18:32:56Z)
• CREval: An Automated Interpretable Evaluation for Creative Image Manipulation under Complex Instructions [59.85102794322254]
  We propose CREval, a fully automated question-answer (QA)-based evaluation pipeline that overcomes the incompleteness and poor interpretability of opaque Multimodal Large Language Models (MLLMs) scoring.<n>We also introduce CREval-Bench, a comprehensive benchmark specifically designed for creative image manipulation under complex instructions.<n>The results show that while closed-source models generally outperform open-source ones on complex and creative tasks, all models still struggle to complete such edits effectively.
  arXiv  Detail & Related papers  (2026-03-27T08:42:09Z)
• Eleven Primitives and Three Gates: The Universal Structure of Computational Imaging [8.513179174301722]
  We prove that every imaging forward model decomposes into a directed acyclic graph over exactly 11 physically typed primitives.<n>We prove that every reconstruction failure has exactly three independent root causes: information deficiency, carrier noise, and operator mismatch.
  arXiv  Detail & Related papers  (2026-03-13T18:54:35Z)
• Agentic Planning with Reasoning for Image Styling via Offline RL [66.10749901925941]
  Direct prompt-based editing fails on complex transformations because vague and subjective prompts often require nuanced understanding of what should be changed in the image.<n>We present a tool-based agentic RL post-training framework that addresses this through structured planning with chain-of-thought reasoning.
  arXiv  Detail & Related papers  (2026-03-07T11:14:37Z)
• SCHEMA for Gemini 3 Pro Image: A Structured Methodology for Controlled AI Image Generation on Google's Native Multimodal Model [0.0]
  SCHEMA is an engineered framework built on systematic professional practice encompassing 850 verified API predictions.<n>SchEMA scales practitioner control from exploratory (approximately 5%) to directive (approximately 95%)<n>Key findings include an observed 91% Mandatory compliance rate and 94% Prohibitions compliance rate across 621 structured prompts.
  arXiv  Detail & Related papers  (2026-02-21T16:51:40Z)
• Multi-Agent Procedural Graph Extraction with Structural and Logical Refinement [66.51979814832332]
  model formulates procedural graph extraction as a multi-round reasoning process with dedicated structural and logical refinement.<n>Experiments demonstrate that model achieves substantial improvements in both structural correctness and logical consistency over strong baselines.
  arXiv  Detail & Related papers  (2026-01-27T04:00:48Z)
• Factuality Matters: When Image Generation and Editing Meet Structured Visuals [46.627460447235855]
  We construct a large-scale dataset of 1.3 million high-quality structured image pairs.<n>We train a unified model that integrates a VLM with FLUX.1 Kontext.<n>A three-stage training curriculum enables progressive feature alignment, knowledge infusion, and reasoning-augmented generation.
  arXiv  Detail & Related papers  (2025-10-06T17:56:55Z)
• DeOcc-1-to-3: 3D De-Occlusion from a Single Image via Self-Supervised Multi-View Diffusion [50.90541069907167]
  We propose DeOcc-1-to-3, an end-to-end framework for occlusion-aware multi-view generation.<n>Our self-supervised training pipeline leverages occluded-unoccluded image pairs and pseudo-ground-truth views to teach the model structure-aware completion and view consistency.
  arXiv  Detail & Related papers  (2025-06-26T17:58:26Z)
• Large Spatial Model: End-to-end Unposed Images to Semantic 3D [79.94479633598102]
  Large Spatial Model (LSM) processes unposed RGB images directly into semantic radiance fields. LSM simultaneously estimates geometry, appearance, and semantics in a single feed-forward operation. It can generate versatile label maps by interacting with language at novel viewpoints.
  arXiv  Detail & Related papers  (2024-10-24T17:54:42Z)
• Composing Ensembles of Pre-trained Models via Iterative Consensus [95.10641301155232]
  We propose a unified framework for composing ensembles of different pre-trained models. We use pre-trained models as "generators" or "scorers" and compose them via closed-loop iterative consensus optimization. We demonstrate that consensus achieved by an ensemble of scorers outperforms the feedback of a single scorer.
  arXiv  Detail & Related papers  (2022-10-20T18:46:31Z)
• Structure by Architecture: Structured Representations without Regularization [31.75200752252397]
  We study the problem of self-supervised structured representation learning using autoencoders for downstream tasks such as generative modeling. We design a novel autoencoder architecture capable of learning a structured representation without the need for aggressive regularization. We demonstrate how these models learn a representation that improves results in a variety of downstream tasks including generation, disentanglement, and extrapolation.
  arXiv  Detail & Related papers  (2020-06-14T04:37:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.