PathAR: Structure-First Autoregressive Synthesis of Multimodal Pathology Images

• URL: http://arxiv.org/abs/2606.01543v1
• Date: Mon, 01 Jun 2026 01:43:48 GMT
• Title: PathAR: Structure-First Autoregressive Synthesis of Multimodal Pathology Images
• Abstract summary: We propose pathology Autorgressive modeling (PathAR) to factorize structure and appearance for modality-label-conditioned pathology generation.<n>PathAR stabilizes morphology under heterogeneous modality-specific appearances and enables spatially aligned image-mask pair generation.
• Score: 51.428093790826814
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Data scarcity in multimodal pathology motivates unified generative models that synthesize modality-specific appearance while preserving anatomically coherent structure. Although modalities differ in appearance statistics, morphological structures such as cellular topology and tissue boundaries are largely preserved across acquisition protocols. However, existing methods often model these factors within a homogeneous token stream, implicitly coupling structure with appearance and weakening structural controllability under modality shifts. To address this, we propose pathology Autorgressive modeling (PathAR), a structure-first autoregressive synthesis framework that explicitly factorizes structure and appearance for modality-label-conditioned pathology generation.PathAR employs a dual vector quantization (Dual-VQ) tokenizer to decompose samples into mask-grounded structure and appearance tokens, and an interleaved autoregressive (IAR) transformer with asymmetric attention visibility to enforce structure-to-appearance dependence. PathAR stabilizes morphology under heterogeneous modality-specific appearances and enables spatially aligned image--mask pair generation. Extensive experiments show that PathAR improves structural consistency and modality fidelity over baselines, maintains sample diversity, supports downstream segmentation in data-scarce regimes, and demonstrates extensibility to finer-grained intra-modality organ-label variation.

Related papers

• Do Not Break the Vessels: Structure-Preserving Mean Flow for Vascular Image Translation [10.186762856964831]
  Cross-modal translation methods mainly emphasize pixel-level fidelity or visual realism.<n>We propose a Structure-Preserving Mean Flow (SPMF) framework that formulates vascular image translation as a topology-invariant transport process.<n> Experiments on paired NIRII-to-2PF and fundus datasets demonstrate consistent improvements over state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-06-30T03:31:42Z)
• Hierarchical Consistency Learning for Test-time Adaptation in Camouflage Perception [50.278200968044665]
  Camouflaged object detection (COD) aims to localize targets that exhibit minimal perceptual differences from backgrounds through physical attributes.<n>Existing methods, constrained by the static train-then-freeze paradigm, suffer from domain rigidity and dependency annotation.<n>We propose the hierarchical consistency learning framework, which integrates test-time adaptation for dynamic representation recalibration.
  arXiv  Detail & Related papers  (2026-05-25T09:57:46Z)
• TopoOR: A Unified Topological Scene Representation for the Operating Room [54.50897207203292]
  Surgical Scene Graphs abstract the complexity of surgical operating rooms (OR) into a structure of entities and their relations.<n>We introduce TopoOR, a new paradigm that models multimodal operating rooms as a higher-order structure.<n>We also propose a higher-order attention mechanism that explicitly preserves manifold structure and modality-specific features.
  arXiv  Detail & Related papers  (2026-03-10T10:19:42Z)
• Protein Autoregressive Modeling via Multiscale Structure Generation [51.92004892768298]
  We present protein autoregressive modeling (PAR), the first multi-scale autoregressive framework for protein backbone generation.<n>We adopt noisy context learning and scheduled sampling, enabling robust backbone generation.<n>On the unconditional generation benchmark, PAR effectively learns protein distributions and produces backbones of high design quality.
  arXiv  Detail & Related papers  (2026-02-04T18:59:49Z)
• PAINT: Pathology-Aware Integrated Next-Scale Transformation for Virtual Immunohistochemistry [17.230315436967356]
  Virtualchemistry aims to computationally synthesize molecular staining patterns from routine Hematoxylin and Eosin (H&E) images.<n>We propose Pathology-Aware Integrated Next-Scale Transformation (PAINT), a visual autoregressive framework that reformulates the synthesis process as a structure-first conditional generation task.
  arXiv  Detail & Related papers  (2026-01-22T14:49:30Z)
• PathoSyn: Imaging-Pathology MRI Synthesis via Disentangled Deviation Diffusion [11.223559964746705]
  We present PathoSyn, a unified generative framework for Magnetic Resonance Imaging (MRI) image synthesis.<n>PathoSyn reformulates imaging-pathology as a disentangled additive deviation on a stable anatomical manifold.<n>We show that PathoSyn significantly outperforms holistic diffusion and mask-conditioned baselines in both perceptual realism and anatomical fidelity.
  arXiv  Detail & Related papers  (2025-12-29T01:13:50Z)
• MIRROR: Multi-Modal Pathological Self-Supervised Representation Learning via Modality Alignment and Retention [57.044719143401664]
  Histopathology and transcriptomics are fundamental modalities in oncology, encapsulating the morphological and molecular aspects of the disease.<n>We present MIRROR, a novel multi-modal representation learning method designed to foster both modality alignment and retention.<n>Extensive evaluations on TCGA cohorts for cancer subtyping and survival analysis highlight MIRROR's superior performance.
  arXiv  Detail & Related papers  (2025-03-01T07:02:30Z)
• RankByGene: Gene-Guided Histopathology Representation Learning Through Cross-Modal Ranking Consistency [11.813883157319381]
  We propose a novel framework that aligns gene and image features using a ranking-based alignment loss.<n>To further enhance the alignment's stability, we employ self-supervised knowledge distillation with a teacher-student network architecture.
  arXiv  Detail & Related papers  (2024-11-22T17:08:28Z)
• Bayesian Unsupervised Disentanglement of Anatomy and Geometry for Deep Groupwise Image Registration [59.062085785106234]
  This article presents a general Bayesian learning framework for multi-modal groupwise image registration.<n>We propose a novel hierarchical variational auto-encoding architecture to realise the inference procedure of the latent variables.<n>Experiments were conducted to validate the proposed framework, including four different datasets from cardiac, brain, and abdominal medical images.
  arXiv  Detail & Related papers  (2024-01-04T08:46:39Z)
• Amortised Inference in Structured Generative Models with Explaining Away [16.92791301062903]
  We extend the output of amortised variational inference to incorporate structured factors over multiple variables. We show that appropriately parameterised factors can be combined efficiently with variational message passing in elaborate graphical structures. We then fit the structured model to high-dimensional neural spiking time-series from the hippocampus of freely moving rodents.
  arXiv  Detail & Related papers  (2022-09-12T12:52:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.