Do VLMs See What Sensors Feel? A Scalable Expert-Guided Design for Wheelchair Accessibility Assessment from Street View

• URL: http://arxiv.org/abs/2606.07642v1
• Date: Mon, 01 Jun 2026 18:46:43 GMT
• Title: Do VLMs See What Sensors Feel? A Scalable Expert-Guided Design for Wheelchair Accessibility Assessment from Street View
• Abstract summary: This paper examines whether vision-language models (VLMs) can identify accessibility barriers from Google Street View imagery.<n>We propose an expert-guided retrieval-augmented framework that combines GSV images, ADA-informed guidance, and expert-derived rubrics to evaluate accessibility dimensions.
• Score: 8.943780916614612
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Assessing built-environment interaction, such as wheelchair accessibility, is difficult because real-world mobility is shaped by distributed, context-dependent, and temporary barriers that are hard to capture at scale. To support scalable assessment, this paper examines whether vision-language models (VLMs) can identify accessibility barriers from Google Street View (GSV) imagery. We propose an expert-guided retrieval-augmented framework that combines GSV images, ADA-informed guidance, and expert-derived rubrics to evaluate accessibility dimensions. We collect a campus-scale dataset at the University of Florida, linking 407 unique GSV locations with GPS-derived wheelchair dwell behavior as a mobility-friction signal. Results show that VLM ratings are both negatively correlated and distributionally similar with dwell time, indicating partial but consistent alignment with a behavioral proxy for mobility friction. Visual cue analysis shows that certain environmental objects, such as curb ramps and crosswalks, are associated with higher VLM accessibility scores, while alignment remains limited for subtle surface conditions, transient obstructions, and viewpoint-dependent barriers. Overall, our findings show the potential of expert-guided VLMs for scalable accessibility assessment aligning with sensor-derived indicators of real-world wheelchair navigation.

Related papers

• Towards Generalizable Visually Grounded Exploration of Household Devices [74.66117923766632]
  Key bottleneck of realizing general autonomous embodied agents lies in Generalizable Visually Grounded Exploration.<n>We introduce VGEBench, a benchmark designed to evaluate the generalizable visually grounded exploration capabilities of Vision-Language Models.<n>Unlike static datasets, we construct a Logic-Driven State Machine framework. This framework simulates multi-turn interaction loops, compelling agents to achieve goals by active visual perception and feedback-driven correction.
  arXiv  Detail & Related papers  (2026-09-01T07:42:45Z)
• Zero-Shot Semantic Re-Identification for Autonomous Driving: A VLM Baseline Study [0.5599792629509229]
  Re-Identification in autonomous driving is typically formulated as a visual matching problem, where observations of vehicles, pedestrians, and cyclists are associated across time, frames, or camera views.<n>We propose a baseline study of a zero-shot pipeline using Vision-Language Models (VLMs) to generate textual descriptions of detected traffic participants.<n>We show that zero-shot semantic descriptions can support effective object re-identification, achieving retrieval performance comparable to a supervised CNN baseline.
  arXiv  Detail & Related papers  (2026-06-08T11:35:34Z)
• Cross-View Urban Traffic Dataset: Drone-Supervised Ground Truth for Monocular Bird's-Eye View Localization [0.3001251717100776]
  We introduce a dataset and benchmark for cross-view urban traffic perception built from ego-centric bicycle videos and aerial drone videos recorded at real urban intersections.<n>The benchmark targets two linked tasks: cross-view identity matching between street-view and drone-view object tracks, and ego-to-bird's-eye-view prediction using aerial supervision.
  arXiv  Detail & Related papers  (2026-06-05T11:01:42Z)
• VISTA: Vision-Grounded and Physics-Validated Adaptation of UMI data for VLA Training [52.05483137072975]
  Universal Manipulation Interface (UMI) enables scalable real-world robot data collection without hardware-specific teleoperation.<n>We present VISTA, a framework that bridges this dual gap through three synergistic components.<n>We release the physical-validation pipeline, UMI-VQA, validated trajectory data, and the pre-trained model for the community.
  arXiv  Detail & Related papers  (2026-06-03T10:38:45Z)
• Benchmarking Egocentric Visual-Inertial SLAM at City Scale [50.1245744173948]
  This paper introduces a new dataset and benchmark for visual-inertial SLAM with egocentric, multi-modal data.<n>We record hours and kilometers of trajectories through a city center with glasses-like devices equipped with various sensors.<n>We show that state-of-the-art systems developed by academia are not robust to these challenges and we identify components that are responsible for this.
  arXiv  Detail & Related papers  (2025-09-30T17:59:31Z)
• SEPose: A Synthetic Event-based Human Pose Estimation Dataset for Pedestrian Monitoring [20.537672896807063]
  SEPose is a synthetic multi-person pose estimation dataset for fixed pedestrian perception generated using dynamic vision sensors in the CARLA simulator.<n>We train existing state-of-the-art models such as RVT and YOLOv8 on our dataset and evaluate them on real event-based data to demonstrate the sim-to-real generalization capabilities of the proposed dataset.
  arXiv  Detail & Related papers  (2025-07-16T04:54:11Z)
• LLM-Guided Indoor Navigation with Multimodal Map Understanding [1.5325823985727567]
  We explore the potential of a Large Language Model (LLM), i.e., ChatGPT, to generate context-aware navigation instructions from indoor map images.<n>Our findings demonstrate the potential of LLMs for supporting personalized indoor navigation, with an average of 86.59% correct indications and a maximum of 97.14%.<n>These results have key implications for AI-driven navigation and assistive technologies.
  arXiv  Detail & Related papers  (2025-03-12T09:32:43Z)
• Are VLMs Ready for Autonomous Driving? An Empirical Study from the Reliability, Data, and Metric Perspectives [56.528835143531694]
  We introduce DriveBench, a benchmark dataset designed to evaluate Vision-Language Models (VLMs)<n>Our findings reveal that VLMs often generate plausible responses derived from general knowledge or textual cues rather than true visual grounding.<n>We propose refined evaluation metrics that prioritize robust visual grounding and multi-modal understanding.
  arXiv  Detail & Related papers  (2025-01-07T18:59:55Z)
• OpenLKA: an open dataset of lane keeping assist from market autonomous vehicles [23.083443555590065]
  Lane Keeping Assist (LKA) has become a standard feature in recent car models.<n>LKA system's operational characteristics and safety performance remain underexplored.<n>We extensively tested mainstream LKA systems from leading U.S. automakers in Tampa, Florida.
  arXiv  Detail & Related papers  (2025-01-06T04:46:10Z)
• Learning Perceptual Locomotion on Uneven Terrains using Sparse Visual Observations [75.60524561611008]
  This work aims to exploit the use of sparse visual observations to achieve perceptual locomotion over a range of commonly seen bumps, ramps, and stairs in human-centred environments. We first formulate the selection of minimal visual input that can represent the uneven surfaces of interest, and propose a learning framework that integrates such exteroceptive and proprioceptive data. We validate the learned policy in tasks that require omnidirectional walking over flat ground and forward locomotion over terrains with obstacles, showing a high success rate.
  arXiv  Detail & Related papers  (2021-09-28T20:25:10Z)
• Pluggable Weakly-Supervised Cross-View Learning for Accurate Vehicle Re-Identification [53.6218051770131]
  Cross-view consistent feature representation is key for accurate vehicle ReID. Existing approaches resort to supervised cross-view learning using extensive extra viewpoints annotations. We present a pluggable Weakly-supervised Cross-View Learning (WCVL) module for vehicle ReID.
  arXiv  Detail & Related papers  (2021-03-09T11:51:09Z)
• Counterfactual Vision-and-Language Navigation via Adversarial Path Sampling [65.99956848461915]
  Vision-and-Language Navigation (VLN) is a task where agents must decide how to move through a 3D environment to reach a goal.<n>One of the problems of the VLN task is data scarcity since it is difficult to collect enough navigation paths with human-annotated instructions for interactive environments.<n>We propose an adversarial-driven counterfactual reasoning model that can consider effective conditions instead of low-quality augmented data.
  arXiv  Detail & Related papers  (2019-11-17T18:02:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.