Evaluating VLMs for Autonomous Agent-Driven Geometry Clipping Detection in Video Game QA

• URL: http://arxiv.org/abs/2607.25921v1
• Date: Tue, 28 Jul 2026 16:10:47 GMT
• Title: Evaluating VLMs for Autonomous Agent-Driven Geometry Clipping Detection in Video Game QA
• Abstract summary: We study the use of Vision-Language Models (VLMs) for anomaly detection in an agent-driven game Quality Assurance pipeline.<n>We benchmark six recent VLMs under a zero-shot prompting setting and analyse their sensitivity to four prompt variants.<n>Our results show that while the VLMs can capture visual cues associated with geometry clipping, they all produce substantial false positives on visually ambiguous frames.
• Score: 6.404338288187941
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this work, we study the use of Vision-Language Models (VLMs) for anomaly detection in an agent-driven game Quality Assurance (QA) pipeline focusing on geometry clipping. In this evaluation, a custom exploration agent navigates a game level to collect visual observations, while the automatic annotation pipeline provides frame-level clipping labels. This setup allows us to evaluate recent VLMs on a controlled anomaly detection task without manual annotation. We benchmark six recent VLMs (Gemini, GPT, Qwen, Gemma, Llama, and Ministral) under a zero-shot prompting setting and analyse their sensitivity to four prompt variants. Our results show that while the VLMs can capture visual cues associated with geometry clipping, they all produce substantial false positives on visually ambiguous frames such as near-contact geometry and partial occlusions. Gemini-3.1-Flash achieves the best overall accuracy and is the most robust to prompt variation, while open-source models exhibit large precision--recall swings depending on the prompt design. These findings suggest that current VLMs are best suited as high-recall candidate filters within multi-stage QA pipelines rather than as standalone bug detectors.

Related papers

• Parser-Free VLM Verification for Federated Weakly Supervised Video Anomaly Detection [0.27998963147546135]
  We introduce a lightweight federated MIL-VLM cascade in which only a compact MIL scorer is trained across clients, while a frozen VLM verifies high-scoring suspect segments post hoc.<n>We study two VLM feedback interfaces: parsed text-generation decisions and a logit-based interface that extracts a continuous anomaly score from next-token Yes/No probabilities.<n>In contrast, the logit interface provides a fixed-free signal that improves both frame-level AUC and frame-level AP over the MIL baseline across both VLMs.
  arXiv  Detail & Related papers  (2026-09-07T13:04:00Z)
• ReflexTrack: A Feedback-Driven Agent for Training-Free Referring Video Object Segmentation [63.38679125666075]
  Referring object segmentation (RVOS) requires segmenting a target specified by natural language throughout a video.<n>Recent agent approaches combine multimodal large language models with promptable segmentation models to perform RVOS without task-specific training.<n>We introduce ReflexTrack, a training-free, feedback-driven agent that closes the loop at both spatial and temporal levels.
  arXiv  Detail & Related papers  (2026-07-27T07:39:20Z)
• Beyond Scene Priors: Fine-Grained Traffic Scene Reasoning with Benchmarking and Query-Guided Small-Object Focus [21.376452292092836]
  We present the Fine-Grained Traffic Reasoning Benchmark (FGTR-Bench) and the Text-Guided Small-Object Reasoning MLLM (TSR-MLLM)<n>FGTR-Bench comprises 40,236 single-image Multiple-Choice Questions (MCQs) created via multi-agent generation, consistency checks, and expert audits.<n>TSR-MLLM, built on Qwen3-VL-4B, uses a query-conditioned Text-Guided Small-Object Focus (TG-SOF) map to resolve evidence dilution.
  arXiv  Detail & Related papers  (2026-07-05T07:27:02Z)
• Employing Vision-Language Models for Face Image Quality Assessment [0.023141219541346198]
  Face Image Quality Assessment (FIQA) is a crucial control step in biometric pipelines.<n>State-of-the-art FIQA methods achieve high utility but typically operate as "black boxes"<n>We investigate the potential of off-the-shelf Vision-Language Models (VLMs) to bridge this gap by performing FIQA in a zero-shot setting.
  arXiv  Detail & Related papers  (2026-05-17T14:57:52Z)
• ReVSI: Rebuilding Visual Spatial Intelligence Evaluation for Accurate Assessment of VLM 3D Reasoning [59.558706734431276]
  Current evaluations of spatial intelligence can be systematically invalid under modern vision-language model (VLM) settings.<n>We introduce ReVSI, a benchmark and protocol that ensures each QA pair is answerable and correct under the model's actual inputs.
  arXiv  Detail & Related papers  (2026-04-27T10:45:51Z)
• AutoVQA-G: Self-Improving Agentic Framework for Automated Visual Question Answering and Grounding Annotation [7.909534692798243]
  Manual annotation of high-quality visual question answering with grounding datasets is crucial for advancing vision-Thought models.<n>Existing automated methods are often hindered by two key issues.<n>We introduce AutoVQA-G, a self-improving agentic framework for automated VQA-G annotation.
  arXiv  Detail & Related papers  (2026-04-19T15:22:00Z)
• RESP: Reference-guided Sequential Prompting for Visual Glitch Detection in Video Games [12.71084447285112]
  We present RESP, a practical multi-frame framework for gameplay glitch detection with vision-language models (VLMs)<n>For each test frame, we select a reference frame from earlier in the same video, establishing a visual baseline.<n> RESP aggregates noisy frame predictions into a stable video-level decision without fine-tuning the VLM.
  arXiv  Detail & Related papers  (2026-04-13T07:04:15Z)
• How Far Can VLMs Go for Visual Bug Detection? Studying 19,738 Keyframes from 41 Hours of Gameplay Videos [41.46982669196867]
  Video-based quality assurance (QA) for long-form gameplay is labor-intensive and error-prone.<n>Recent benchmarks suggest that vision language models (VLMs) can achieve promising results in detecting visual glitches on curated datasets.
  arXiv  Detail & Related papers  (2026-03-24T01:59:06Z)
• Visually Prompted Benchmarks Are Surprisingly Fragile [82.98001690512461]
  Key challenge in evaluating VLMs is testing their ability to analyze visual content independently from their textual priors.<n>We demonstrate how details in benchmark setup, including visual marker design and dataset size, have a significant influence on model performance and leaderboard rankings.<n>To mitigate this instability, we curate existing datasets to create VPBench, a larger visually prompted benchmark with 16 visual marker variants.
  arXiv  Detail & Related papers  (2025-12-19T18:26:58Z)
• MAGNET: A Multi-agent Framework for Finding Audio-Visual Needles by Reasoning over Multi-Video Haystacks [67.31276358668424]
  We introduce a novel task named AV-HaystacksQA, where the goal is to identify salient segments across different videos in response to a query and link them together to generate the most informative answer.<n> AVHaystacks is an audio-visual benchmark comprising 3100 annotated QA pairs designed to assess the capabilities of LMMs in multi-video retrieval and temporal grounding task.<n>We propose a model-agnostic, multi-agent framework to address this challenge, achieving up to 89% and 65% relative improvements over baseline methods on BLEU@4 and GPT evaluation scores in QA task on our proposed AVHaystack
  arXiv  Detail & Related papers  (2025-06-08T06:34:29Z)
• GEOBench-VLM: Benchmarking Vision-Language Models for Geospatial Tasks [84.86699025256705]
  We present GEOBench-VLM, a benchmark specifically designed to evaluate Vision-Language Models (VLMs) on geospatial tasks.<n>Our benchmark features over 10,000 manually verified instructions and spanning diverse visual conditions, object types, and scales.<n>We evaluate several state-of-the-art VLMs to assess performance on geospatial-specific challenges.
  arXiv  Detail & Related papers  (2024-11-28T18:59:56Z)
• DetToolChain: A New Prompting Paradigm to Unleash Detection Ability of MLLM [81.75988648572347]
  We present DetToolChain, a novel prompting paradigm to unleash the zero-shot object detection ability of multimodal large language models (MLLMs) Our approach consists of a detection prompting toolkit inspired by high-precision detection priors and a new Chain-of-Thought to implement these prompts. We show that GPT-4V with our DetToolChain improves state-of-the-art object detectors by +21.5% AP50 on MS Novel class set for open-vocabulary detection.
  arXiv  Detail & Related papers  (2024-03-19T06:54:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.