QuPAINT: Physics-Aware Multimodal Reasoning for Quantum Material Characterization

• URL: http://arxiv.org/abs/2609.12202v1
• Date: Thu, 10 Sep 2026 20:56:07 GMT
• Title: QuPAINT: Physics-Aware Multimodal Reasoning for Quantum Material Characterization
• Abstract summary: QuPAINT is a physics-aware framework for transferable quantum flake characterization.<n>QMat-Instruct is a multimodal instruction dataset with image-specific reasoning traces generated from verified annotations.<n>QuPAINT-8B substantially outperforms prior methods and establishes state-of-the-art performance for both general and monolayer flake detection.
• Score: 12.888415301529891
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Characterizing two-dimensional (2D) quantum materials by optical microscopy requires localizing exfoliated flakes and determining their layer thickness from subtle optical contrast and interference color to select suitable flakes for device fabrication. However, models face synthetic-to-real domain shifts and variation across materials, substrates, laboratories, and imaging conditions. We present QuPAINT, a physics-aware multimodal framework for transferable quantum flake characterization. The Synthetic Materials Framework (Synthia) generates diverse synthetic microscopy images while preserving layer-dependent optical behavior. Using these images, we construct QMat-Instruct, a multimodal instruction dataset with image-specific reasoning traces generated from verified annotations and constrained to observable optical cues. QuPAINT integrates these signals through Physics-Informed Attention (PIA), which injects substrate-relative optical priors into the visual representation to support grounded multimodal reasoning. For evaluation, we introduce QF-Bench, to our knowledge, the largest real-world benchmark for this problem, spanning diverse microscopy and substrate conditions. Using its verified annotations, we study counting, visual grounding, reasoning quality, confidence calibration, and transfer to an unseen material. QuPAINT-8B substantially outperforms prior methods and establishes state-of-the-art performance for both general and monolayer flake detection. Additional experiments show that image-grounded supervision improves strict spatial grounding and confidence calibration while preserving robust general flake detection on the unseen material.

Related papers

• Physics-Informed Deep Learning Model for Cross-Modality Super-Resolution in Fluorescence Microscopy [0.0]
  Cross-modality image translation offers a route to super-resolution fluorescence microscopy from low-resolution images.<n>We propose a physics-informed generative adversarial network for confocal-to-STED image translation.
  arXiv  Detail & Related papers  (2026-07-23T11:16:26Z)
• Quantum Optical Neuron for Image Classification via Multiphoton Interference [0.0]
  We experimentally demonstrate a camera-free quantum-optical images that performs inference directly at the measurement layer.<n>We realize both a single-perceptron quantum optical neuron and a two-neuron shallow network, achieving high accuracy on benchmark datasets.<n>This approach paves the way toward neuromorphic quantum photonic processors capable of extracting task-relevant information.
  arXiv  Detail & Related papers  (2026-03-30T18:01:45Z)
• QuPAINT: Physics-Aware Instruction Tuning Approach to Quantum Material Discovery [12.888415301529891]
  Characterizing two-dimensional quantum materials from optical microscopy images is challenging due to subtle layer-dependent contrast, limited labeled data, and significant variation across laboratories and imaging setups.<n>This work presents a new physics-aware multimodal framework that addresses these limitations from both the data and model perspectives.<n>We first present Synthia, a physics-based synthetic data generator that simulates realistic optical responses of quantum material flakes under thin-film interference.<n>We introduce QMat-Instruct, the first large-scale instruction dataset for quantum materials, comprising multimodal, physics-informed question-answer pairs designed to teach Multimodal Large Language Models (ML
  arXiv  Detail & Related papers  (2026-02-19T15:44:41Z)
• Exploring Physical Intelligence Emergence via Omni-Modal Architecture and Physical Data Engine [50.62040226184694]
  We present OmniFysics, a compact omni-modal model that unifies understanding across images, audio, video, and text.<n>To inject explicit physical knowledge, we build a physical data engine with two components.<n>Experiments show competitive performance on standard multimodal benchmarks and improved results on physics-oriented evaluations.
  arXiv  Detail & Related papers  (2026-02-05T14:04:51Z)
• Physics-Informed Graph Neural Network with Frequency-Aware Learning for Optical Aberration Correction [36.496034762040054]
  Optical aberrations significantly degrade image quality in microscopy, particularly when imaging deeper into samples.<n>We propose ZRNet, a physics-informed framework that jointly performs Zernike prediction and optical image Restoration.<n>Our approach achieves state-of-the-art performance in both image restoration and Zernike coefficient prediction across diverse microscopy modalities and biological samples with complex, large-amplitudes.
  arXiv  Detail & Related papers  (2025-12-05T12:58:55Z)
• MaterialRefGS: Reflective Gaussian Splatting with Multi-view Consistent Material Inference [83.38607296779423]
  We show that multi-view consistent material inference with more physically-based environment modeling is key to learning accurate reflections with Gaussian Splatting.<n>Our method faithfully recovers both illumination and geometry, achieving state-of-the-art rendering quality in novel views synthesis.
  arXiv  Detail & Related papers  (2025-10-13T13:29:20Z)
• GOBench: Benchmarking Geometric Optics Generation and Understanding of MLLMs [66.55945133516776]
  We introduce GOBench, the first benchmark to evaluate MLLMs' ability across two tasks: Generating Optically Authentic Imagery and Understanding Underlying Optical Phenomena.<n>We use MLLMs to construct GOBench-Gen-1k dataset. We then organize subjective experiments to assess the generated imagery based on Optical Authenticity, Aesthetic Quality, and Instruction Fidelity.<n>For the understanding task, we apply crafted evaluation instructions to test optical understanding ability of eleven prominent MLLMs. The experimental results demonstrate that current models face significant challenges in both optical generation and understanding.
  arXiv  Detail & Related papers  (2025-06-01T12:46:14Z)
• Examining the Impact of Optical Aberrations to Image Classification and Object Detection Models [58.98742597810023]
  Vision models have to behave in a robust way to disturbances such as noise or blur.<n>This paper studies two datasets of blur corruptions, which we denote OpticsBench and LensCorruptions.<n> Evaluations for image classification and object detection on ImageNet and MSCOCO show that for a variety of different pre-trained models, the performance on OpticsBench and LensCorruptions varies significantly.
  arXiv  Detail & Related papers  (2025-04-25T17:23:47Z)
• PIGUIQA: A Physical Imaging Guided Perceptual Framework for Underwater Image Quality Assessment [59.9103803198087]
  We propose a Physical Imaging Guided perceptual framework for Underwater Image Quality Assessment (UIQA)<n>By leveraging underwater radiative transfer theory, we integrate physics-based imaging estimations to establish quantitative metrics for these distortions.<n>The proposed model accurately predicts image quality scores and achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2024-12-20T03:31:45Z)
• Quantum enhanced probing of multilayered-samples [0.0]
  Quantum Optical Coherence Tomography relies on non-classical light sources to reconstruct the internal structure of multilayered materials. Here, by utilizing a full theoretical model, in combination with a fast genetic algorithm to post-process the data, we successfully extract the morphology of complex multilayered samples. Our results could potentially lead to the development of practical high-resolution probing of complex structures.
  arXiv  Detail & Related papers  (2022-12-25T21:22:25Z)
• Interaction-free imaging of multi-pixel objects [58.720142291102135]
  Quantum imaging is well-suited to study sensitive samples which require low-light conditions, like biological tissues. In this context, interaction-free measurements (IFM) allow us infer the presence of an opaque object without the photon interacting with the sample. Here we extend the IFM imaging schemes to multi-pixel, semi-transparent objects, by encoding the information about the pixels into an internal degree of freedom.
  arXiv  Detail & Related papers  (2021-06-08T06:49:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.