Structured-Prior-Guided Diffusion Inpainting with Physical Consistency for Traffic Sign Augmentation

• URL: http://arxiv.org/abs/2609.02348v1
• Date: Wed, 02 Sep 2026 09:22:23 GMT
• Title: Structured-Prior-Guided Diffusion Inpainting with Physical Consistency for Traffic Sign Augmentation
• Abstract summary: General-purpose inpainting models distort digits, deform geometry and perspective, and shift colours when applied directly to sign regions.<n>We propose a structured-prior-guided diffusion inpainting framework with physical consistency.<n>We train by self-supervised reconstruction on a large set of images collected in-house at AMAP, then evaluate zero-shot on the public TT100K-2021 dataset.
• Score: 12.366721313845483
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Traffic sign detection faces a long-tailed data distribution. Many rare signs matter as much as common ones from a regulatory standpoint, yet they have very few samples. Generative data augmentation is one way out. General-purpose inpainting models, however, distort digits, deform geometry and perspective, and shift colours when applied directly to sign regions. We trace this to a single gap: the conditioning signal is too abstract for the physical composition of a sign. We propose a structured-prior-guided diffusion inpainting framework with physical consistency. It injects the semantic, appearance and geometric priors of a sign through three orthogonal pathways: a JSON-formatted text prompt, a front-view vector template rendered with measured dominant colours (via IP-Adapter), and an affine-aligned vector template (via ControlNet). Two physical consistency losses constrain colour with a CIELAB chromaticity $L_1$ term and edge structure with a Sobel gradient term. We train by self-supervised reconstruction on a large set of images collected in-house at AMAP, then evaluate zero-shot on the public TT100K-2021 dataset, a different source. Our method uses a Stable Diffusion 1.5 backbone of about 1.4B parameters. It beats seven representative competitors on every metric of reconstruction fidelity, physical consistency and semantic controllability. Its OCR exact-match rate reaches 91.1\%, against 44.2\% for the 12B industrial model FLUX.1 Fill [dev], and it needs only $1/14$ of that model's inference time. Leave-one-out ablations confirm that each of the three prior pathways and both loss terms contribute on their own. In downstream detection, the synthetic data raises the group-pooled AP50 of rare classes by $1.23\times$ to $7.40\times$ over a real-data-only baseline. Code and pre-trained models are available at https://github.com/52hz-whale/TrafficSignInpaint.

Related papers

• Training-Free Inpainting Across Domains with a Frozen Text-to-Image Diffusion Model [0.8716913598251385]
  We show that a frozen generic text-to-image diffusion model can perform conditional inpainting across three evaluated natural-image domains.<n>Step-PI augments known-region projection with boundary-interior latent feedback, persistent PI state, and a four-field release schedule.
  arXiv  Detail & Related papers  (2026-09-01T07:57:53Z)
• When Does Consensus Mean Correctness? Measuring the Agreement-Accuracy Coupling with Semantics-Preserving Re-Rendering [0.8594140167290097]
  A model's agreement across perturbed inputs is used as a label-free reliability signal.<n>Rerendering beats resampling on both accuracy and reliability.<n>Fine-tuning on the model's own cross-render consensus inverts.
  arXiv  Detail & Related papers  (2026-08-06T07:08:05Z)
• Inferring Missing Trajectory Data with Temporal Convolutional Networks [0.0]
  We propose a Temporal Convolutional Network (TCN) with symmetric dilation that relaxes the standard causality constraint.<n>The model is trained with a composite loss that combines weighted mean squared error, boundary--continuity penalties, and a smoothness regularizer.<n>Trained on a synthetic dataset of $1,000$ (train), $200$ (validation), and $300$ (test) two-dimensional trajectories with randomly placed 20% masked segments, the model achieves good R$2$, MSE and MAE metrics.
  arXiv  Detail & Related papers  (2026-07-27T23:46:15Z)
• Multi-Modal Spatio-Temporal Graph Neural Network with Mixture of Experts for Soil Organic Carbon Prediction [0.0]
  Existing approaches face two limitations: they pair hand-crafted co-modals with classical ML or single-modal deep models that miss rich spectral and temporal information.<n>We introduce SpTG-NN, a multi-temporal graph neural network addressing both SpTGNN and grid-based architectures.<n>A fine-tuned TerraMind encoder extracts node features from Sentinel-2, Sentinel-1 and DEM signals.
  arXiv  Detail & Related papers  (2026-06-15T11:25:38Z)
• PROSE: Training-Free Egocentric Scene Registration with Vision-Language Models [48.66414784633783]
  PROSE (Prompted Scene rEgistration) lifts each RGB sequence into an object-level 3D scene graph.<n> PROSE adds no learned parameters and requires no depth sensor, training, or annotated graph.<n>On the egocentric Aria Digital Twin and Aria Everyday Activities benchmarks, it outperforms both geometric and learned scene-graph baselines in registration accuracy.
  arXiv  Detail & Related papers  (2026-06-15T11:11:14Z)
• Surflo: Consistent 3D Surface Flow Model with Global State [57.57234680235885]
  Surflo decodes oriented 3D surface points by independently transporting them from noise onto the surface via flow matching.<n>To suppress the local inconsistencies inherent to independent per-point decoding, an inference-time guidance term correlates nearby points by injecting a photometric gradient.<n>Surflo matches or surpasses feed-forward baselines on surface metrics, runs an order of magnitude faster than optimization-based methods that require hundreds of views, and is the only feed-forward approach to combine a global latent with arbitrary-resolution decoding.
  arXiv  Detail & Related papers  (2026-06-11T17:48:38Z)
• DEMIST: \underline{DE}coupled \underline{M}ulti-stream latent d\underline{I}ffusion for Quantitative Myelin Map \underline{S}yn\underline{T}hesis [5.105133772716832]
  Quantitative magnetization transfer (qMT) imaging provides myelin-sensitive biomarkers, such as the pool size ratio (PSR)<n>We propose DEMIST to synthesize PSR maps from standard T1w and FLAIR images using a 3D latent diffusion model with three complementary conditioning mechanisms.
  arXiv  Detail & Related papers  (2025-11-16T00:19:48Z)
• FreeSeg-Diff: Training-Free Open-Vocabulary Segmentation with Diffusion Models [49.80911683739506]
  We focus on the task of image segmentation, which is traditionally solved by training models on closed-vocabulary datasets.<n>We leverage different and relatively small-sized, open-source foundation models for zero-shot open-vocabulary segmentation.<n>Our approach (dubbed FreeSeg-Diff), which does not rely on any training, outperforms many training-based approaches on both Pascal VOC and COCO datasets.
  arXiv  Detail & Related papers  (2024-03-29T10:38:25Z)
• Consistent Diffusion Models: Mitigating Sampling Drift by Learning to be Consistent [97.64313409741614]
  We propose to enforce a emphconsistency property which states that predictions of the model on its own generated data are consistent across time. We show that our novel training objective yields state-of-the-art results for conditional and unconditional generation in CIFAR-10 and baseline improvements in AFHQ and FFHQ.
  arXiv  Detail & Related papers  (2023-02-17T18:45:04Z)
• Cross-Model Pseudo-Labeling for Semi-Supervised Action Recognition [98.25592165484737]
  We propose a more effective pseudo-labeling scheme, called Cross-Model Pseudo-Labeling (CMPL) CMPL achieves $17.6%$ and $25.1%$ Top-1 accuracy on Kinetics-400 and UCF-101 using only the RGB modality and $1%$ labeled data, respectively.
  arXiv  Detail & Related papers  (2021-12-17T18:59:41Z)
• Towards a method to anticipate dark matter signals with deep learning at the LHC [58.720142291102135]
  We study several simplified dark matter (DM) models and their signatures at the LHC using neural networks. We focus on the usual monojet plus missing transverse energy channel, but to train the algorithms we organize the data in 2D histograms instead of event-by-event arrays. This results in a large performance boost to distinguish between standard model (SM) only and SM plus new physics signals.
  arXiv  Detail & Related papers  (2021-05-25T15:38:13Z)
• UnrealPerson: An Adaptive Pipeline towards Costless Person Re-identification [102.58619642363959]
  This paper presents UnrealPerson, a novel pipeline that makes full use of unreal image data to decrease the costs in both the training and deployment stages. With 3,000 IDs and 120,000 instances, our method achieves a 38.5% rank-1 accuracy when being directly transferred to MSMT17.
  arXiv  Detail & Related papers  (2020-12-08T08:15:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.