Fine-Tuning Vision-Language Models for Understanding Current Damage and Scoring Priority with Quality Guard Agent

• URL: http://arxiv.org/abs/2605.27452v1
• Date: Sun, 24 May 2026 21:11:50 GMT
• Title: Fine-Tuning Vision-Language Models for Understanding Current Damage and Scoring Priority with Quality Guard Agent
• Abstract summary: Bridge inspection in Japan requires mandatory visual assessments every five years.<n> qualitative damage ratings (levels a-e) assigned by different engineers exhibit significant inter-rater variability.<n>This paper presents a methodology for automating bridge damage understanding and repair priority scoring.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Bridge inspection in Japan requires mandatory visual assessments every five years, yet qualitative damage ratings (levels a-e) assigned by different engineers exhibit significant inter-rater variability -- a critical barrier to consistent infrastructure management. The aging of skilled engineers further threatens inspection capacity. This paper presents a methodology for automating bridge damage understanding and repair priority scoring using fine-tuned Vision-Language Models (VLMs). We fine-tune LLaVA-1.5-7B with QLoRA on up to 4,000 paired bridge damage images and inspection text records, then evaluate on a fixed test set of 800 images. The model outputs natural language descriptions identifying structural members and damage patterns, from which a rule-based scoring engine calculates a five-level repair priority index. A progressive training study (1k/2k/3k/4k samples) reveals that 2k training samples achieve near-optimal validation loss in only 2.9 hours of training; beyond 2k, validation loss improves by no more than 0.2% per doubling of training samples, exhibiting clear diminishing returns. Furthermore, semantic similarity on the held-out test set peaks at 3k (0.6909) and degrades at 4k (0.6739), indicating that quality-curated mid-scale data outperforms larger but noisier corpora. Inference optimization combining torch.compile() and batch processing (batch_size=8) achieves 10.06 seconds per image -- a 70.2% reduction over the unoptimized baseline. Our approach contributes to data governance in bridge inspection, reduces inter-rater variability, and provides AI-assisted triage to augment expert engineers in inspection workflows. Furthermore, we introduce a two-stage Quality Guard using a fine-tuned Swallow-8B SLM to reject low-quality VLM outputs before priority scoring, preventing spurious scores from damaged or unrecognised images.

Related papers

• Max-Window Scale Estimation for Near-Lossless HiF8 W8A8 Quantization-Aware Training [5.976336341654394]
  Quantization-aware training (QAT) with low-bit floating-point formats enables efficient LLM deployment, yet introduces subtle failure modes invisible to standard training metrics.<n>We present a systematic study of HiF8 W8A8 QAT for OpenPangu-Embedded-1B through the lens of Delayed Scaling (DTS)<n>We identify and disentangle two failure modes: (i)amax saturation, where delayed scale estimates corrupt knowledge-sensitive representations via forward-pass clipping, and (ii)catastrophic forgetting, where an aggressive learning rate overwrites pretrained commonsense knowledge independently of quantization
  arXiv  Detail & Related papers  (2026-05-25T09:19:57Z)
• Visual Aesthetic Benchmark: Can Frontier Models Judge Beauty? [59.923111838399144]
  We introduce the Visual Aesthetic Benchmark (VAB), which casts aesthetic evaluation as comparative selection over candidate sets with matched subject matter.<n>VAB contains 400 tasks and 1,195 images across fine art, photography, and illustration, with labels derived from the consensus of 10 independent expert judges per task.<n>We find that the strongest system identifies both the best and the worst image correctly across three random permutations of the candidate order in only 26.5% of tasks, far below the 68.9% achieved by human experts.
  arXiv  Detail & Related papers  (2026-05-12T19:33:28Z)
• OccuBench: Evaluating AI Agents on Real-World Professional Tasks via Language Environment Simulation [57.505743202759646]
  OccuBench is a benchmark covering 100 real-world professional task scenarios across 10 industry categories and 65 specialized domains.<n>Our multi-agent synthesis pipeline automatically produces evaluation instances with guaranteed solvability, calibrated difficulty, and document-grounded diversity.
  arXiv  Detail & Related papers  (2026-04-13T00:27:32Z)
• MinerU2.5-Pro: Pushing the Limits of Data-Centric Document Parsing at Scale [92.09717763663873]
  We present MinerU2.5-Pro, which advances the state of the art purely through data engineering and training strategy design.<n>At its core is a Data Engine co-designed around coverage, informativeness, and annotation accuracy.<n>We show that MinerU2.5-Pro achieves 95.69 on OmniDocBench v1.6, improving over the same-architecture baseline by 2.71 points and surpassing all existing methods.
  arXiv  Detail & Related papers  (2026-04-06T15:44:18Z)
• Quantized Vision-Language Models for Damage Assessment: A Comparative Study of LLaVA-1.5-7B Quantization Levels [0.0]
  This paper presents a comprehensive study of quantized Vision-Language Models (VLMs) for automated bridge damage assessment.<n>We focus on the trade-offs between description quality, inference speed, and resource requirements.<n>We develop an end-to-end pipeline combining LLaVA-1.5-7B for visual damage analysis, structured extraction, and rule-based priority scoring.
  arXiv  Detail & Related papers  (2026-03-24T12:33:18Z)
• Synthetic Defect Image Generation for Power Line Insulator Inspection Using Multimodal Large Language Models [3.72557681438567]
  Utility companies increasingly rely on drone imagery for routine inspection, but training accurate defect-type classifiers remains difficult.<n>We address this data-scarcity setting by using an off-the-shelf multimodal large language model (MLLM) as a training-free image generator.<n>We evaluate on ceramic insulator defect-type classification using a public dataset with a realistic low training-data regime.
  arXiv  Detail & Related papers  (2026-03-09T08:06:27Z)
• Beyond Human Performance: A Vision-Language Multi-Agent Approach for Quality Control in Pharmaceutical Manufacturing [1.5362004217750165]
  Colony-forming unit (CFU) detection is critical in pharmaceutical manufacturing.<n>To address this, we developed a multi-agent framework combining deep learning (DL) with vision-language models (VLMs)<n>Initial DL-based automation reduced human verification by 50 percent across vaccine manufacturing sites.
  arXiv  Detail & Related papers  (2026-02-24T04:48:05Z)
• LLaVA-Critic-R1: Your Critic Model is Secretly a Strong Policy Model [99.71684530652942]
  We show that LLaVA-Critic-R1 emerges as a top-performing critic but also as a competitive policy model.<n>Applying self-critique at test time yields an average +13.8% improvement on five representative reasoning tasks.<n>Our results reveal that RL training on critic data can produce a unified model excelling at both evaluation and generation.
  arXiv  Detail & Related papers  (2025-08-31T03:08:02Z)
• Reinforcement Learning for Reasoning in Large Language Models with One Training Example [117.86853102104256]
  We show that reinforcement learning with verifiable reward using one training example (1-shot RLVR) is effective in incentivizing the math reasoning capabilities of large language models (LLMs)<n>We identify some interesting phenomena during 1-shot RLVR, including cross-category generalization, increased frequency of self-reflection, and sustained test performance improvement.
  arXiv  Detail & Related papers  (2025-04-29T09:24:30Z)
• Benchmarking Reasoning Robustness in Large Language Models [76.79744000300363]
  We find significant performance degradation on novel or incomplete data.<n>These findings highlight the reliance on recall over rigorous logical inference.<n>This paper introduces a novel benchmark, termed as Math-RoB, that exploits hallucinations triggered by missing information to expose reasoning gaps.
  arXiv  Detail & Related papers  (2025-03-06T15:36:06Z)
• SPot-the-Difference Self-Supervised Pre-training for Anomaly Detection and Segmentation [17.954335118363964]
  We release the Visual Anomaly (VisA) dataset consisting of 10,821 high-resolution color images (9,621 normal and 1,200 anomalous samples) covering 12 objects in 3 domains. We propose a new self-supervised framework - SPot-the-difference (SPD) - which can regularize contrastive self-supervised pre-training. Experiments on VisA and MVTec-AD dataset show that SPD consistently improves contrastive pre-training baselines and even the supervised pre-training.
  arXiv  Detail & Related papers  (2022-07-28T18:00:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.