Vision-Language Foundation Models for Comprehensive Automated Pavement Condition Assessment

• URL: http://arxiv.org/abs/2604.08212v1
• Date: Thu, 09 Apr 2026 13:11:30 GMT
• Title: Vision-Language Foundation Models for Comprehensive Automated Pavement Condition Assessment
• Abstract summary: General-purpose vision-language models struggle with specialized technical fields requiring precise terminology, structured reasoning, and adherence to engineering standards.<n>This work addresses whether domain-specific instruction tuning can enable comprehensive pavement condition assessment through vision-language models.
• Score: 6.830580384536778
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: General-purpose vision-language models demonstrate strong performance in everyday domains but struggle with specialized technical fields requiring precise terminology, structured reasoning, and adherence to engineering standards. This work addresses whether domain-specific instruction tuning can enable comprehensive pavement condition assessment through vision-language models. PaveInstruct, a dataset containing 278,889 image-instruction-response pairs spanning 32 task types, was created by unifying annotations from nine heterogeneous pavement datasets. PaveGPT, a pavement foundation model trained on this dataset, was evaluated against state-of-the-art vision-language models across perception, understanding, and reasoning tasks. Instruction tuning transformed model capabilities, achieving improvements exceeding 20% in spatial grounding, reasoning, and generation tasks while producing ASTM D6433-compliant outputs. These results enable transportation agencies to deploy unified conversational assessment tools that replace multiple specialized systems, simplifying workflows and reducing technical expertise requirements. The approach establishes a pathway for developing instruction-driven AI systems across infrastructure domains including bridge inspection, railway maintenance, and building condition assessment.

Related papers

• DrawingVQA: A Real-World Benchmark for Multi-Depth Visual-Textual Reasoning on Construction Drawings [0.4078247440919472]
  DrawingVQA is the first benchmark designed to evaluate multimodal large language models (MLLMs) on real-world construction drawings.<n>DrawingVQA bridges this gap with 33 "Issued for Construction" drawings and 92 expertly curated question-answer pairs.
  arXiv  Detail & Related papers  (2026-07-16T19:44:03Z)
• AffordanceVLA: A Vision-Language-Action Model Empowering Action Generation through Affordance-Aware Understanding [55.535374902204296]
  AffordanceVLA is a unified framework that introduces structured affordance forecasting as a task-oriented intermediate representation.<n>We show that AffordanceVLA achieves strong performance across diverse manipulation scenarios.
  arXiv  Detail & Related papers  (2026-06-04T13:28:51Z)
• The Semantic Training Gap: Ontology-Grounded Tool Architectures for Industrial AI Agent Systems [0.0]
  Large language model (LLM)-based AI agents are increasingly deployed in manufacturing environments for analytics, quality management, and decision support.<n>These agents demonstrate statistical fluency with domain terminology but lack grounded understanding of operational semantics.<n>This paper identifies and formalizes the semantic training gap: a structural disconnect between how AI systems acquire domain vocabulary through training and how manufacturing operations define meaning through ontological relationships.
  arXiv  Detail & Related papers  (2026-05-11T20:48:18Z)
• Research on Vision-Language Question Answering Models for Industrial Robots [6.470944338393257]
  A hierarchical cross-modal fusion model is proposed for vision-language question answering (VLQA) in industrial robotics.<n>The framework integrates advanced object detection, multi-scale visual encoding, syntactic parsing, and task-aware semantic attention to unite vision and language signals into a joint reasoning space.
  arXiv  Detail & Related papers  (2026-05-02T15:11:48Z)
• Can Large Multimodal Models Inspect Buildings? A Hierarchical Benchmark for Structural Pathology Reasoning [7.005450139695288]
  Large Multimodal Models (LMMs) promise a paradigm shift toward active reasoning.<n>We present textitDefectBench, the first benchmark designed to interrogate LMMs beyond basic semantic recognition.<n>This work provides both a rigorous benchmarking standard and a high-quality open-source database.
  arXiv  Detail & Related papers  (2026-03-20T17:24:46Z)
• Unleashing the Capabilities of Large Vision-Language Models for Intelligent Perception of Roadside Infrastructure [12.667510244197047]
  General-purpose models often struggle to capture the necessary fine-grained attributes and domain rules.<n>We propose a domain-adapted framework that transforms Large Vision Language Models into specialized agents for intelligent infrastructure analysis.<n>Our framework achieves a detection performance of 58.9 mAP and an attribute recognition accuracy of 95.5%.
  arXiv  Detail & Related papers  (2026-01-15T16:16:34Z)
• Foundation Models for Trajectory Planning in Autonomous Driving: A Review of Progress and Open Challenges [53.47232506143113]
  Multi-modal foundation models have transformed the technology for autonomous driving.<n>We provide a comprehensive examination of such methods through a unifying taxonomy.<n>We assess these approaches with respect to the openness of their source code and datasets.
  arXiv  Detail & Related papers  (2025-10-31T18:05:02Z)
• A Survey on Efficient Vision-Language-Action Models [153.11669266922993]
  Vision-Language-Action models (VLAs) represent a significant frontier in embodied intelligence, aiming to bridge digital knowledge with physical-world interaction.<n>Motivated by the urgent need to address these challenges, this survey presents the first comprehensive review of Efficient Vision-Language-Action models.
  arXiv  Detail & Related papers  (2025-10-27T17:57:33Z)
• Ground-Compose-Reinforce: Grounding Language in Agentic Behaviours using Limited Data [17.614395125762304]
  Grounding language in perception and action is a key challenge when building situated agents that can interact with humans, or other agents, via language.<n>We propose Ground-Compose-Reinforce, an end-to-end, neurosymbolic framework for training RL agents directly from high-level task specifications.
  arXiv  Detail & Related papers  (2025-07-14T19:05:15Z)
• OWL: Optimized Workforce Learning for General Multi-Agent Assistance in Real-World Task Automation [65.15955645757705]
  We introduce Workforce, a hierarchical multi-agent framework that decouples strategic planning from specialized execution.<n>During inference, Workforce seamlessly adapts to new domains by adding or modifying worker agents.<n>For training, we introduce optimized Workforce Learning (OWL), which improves generalization across domains.
  arXiv  Detail & Related papers  (2025-05-29T17:51:58Z)
• Flex: End-to-End Text-Instructed Visual Navigation from Foundation Model Features [59.892436892964376]
  We investigate the minimal data requirements and architectural adaptations necessary to achieve robust closed-loop performance with vision-based control policies.<n>Our findings are synthesized in Flex (Fly lexically), a framework that uses pre-trained Vision Language Models (VLMs) as frozen patch-wise feature extractors.<n>We demonstrate the effectiveness of this approach on a quadrotor fly-to-target task, where agents trained via behavior cloning successfully generalize to real-world scenes.
  arXiv  Detail & Related papers  (2024-10-16T19:59:31Z)
• Terrain characterisation for online adaptability of automated sonar processing: Lessons learnt from operationally applying ATR to sidescan sonar in MCM applications [0.0]
  This paper presents two online seafloor characterisation techniques to improve explainability during Autonomous Underwater Vehicles (AUVs) missions. Both techniques rely on an unsupervised machine learning approach to extract terrain features which relate to the human understanding of terrain complexity. The first technnique provides a quantitative, application-driven terrain characterisation metric based on the performance of an ATR algorithm. The second method provides a way to incorporate subject matter expertise and enables contextualisation and explainability in support for scenario-dependent subjective terrain characterisation.
  arXiv  Detail & Related papers  (2024-04-29T12:48:42Z)
• Towards an Approach to Pattern-based Domain-Specific Requirements Engineering [0.0]
  We propose the Pattern-based Domain-specific Requirements Engineering Approach for the specification of functional and performance requirements. This approach emerges from an academia-industry collaboration and is our first attempt to frame an approach which allows for analyzing domain knowledge. Our contribution is two-fold: First, we present a solution to pattern-based domain-specific requirements engineering and its exemplary integration into quality assurance techniques.
  arXiv  Detail & Related papers  (2024-04-26T11:38:55Z)
• Foundational Models Defining a New Era in Vision: A Survey and Outlook [151.49434496615427]
  Vision systems to see and reason about the compositional nature of visual scenes are fundamental to understanding our world. The models learned to bridge the gap between such modalities coupled with large-scale training data facilitate contextual reasoning, generalization, and prompt capabilities at test time. The output of such models can be modified through human-provided prompts without retraining, e.g., segmenting a particular object by providing a bounding box, having interactive dialogues by asking questions about an image or video scene or manipulating the robot's behavior through language instructions.
  arXiv  Detail & Related papers  (2023-07-25T17:59:18Z)
• DeepStruct: Pretraining of Language Models for Structure Prediction [64.84144849119554]
  We pretrain language models on a collection of task-agnostic corpora to generate structures from text. Our structure pretraining enables zero-shot transfer of the learned knowledge that models have about the structure tasks. We show that a 10B parameter language model transfers non-trivially to most tasks and obtains state-of-the-art performance on 21 of 28 datasets.
  arXiv  Detail & Related papers  (2022-05-21T00:58:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.