Semantic Robustness Probing via Inpainting: An Interactive Tool for Safety-Critical Object Detection

• URL: http://arxiv.org/abs/2605.27155v2
• Date: Wed, 27 May 2026 08:15:42 GMT
• Title: Semantic Robustness Probing via Inpainting: An Interactive Tool for Safety-Critical Object Detection
• Abstract summary: Testing object detectors in safety-critical domains requires semantically meaningful probes beyond pixel-level corruptions.<n>We present SemProbe, a tool for semantic robustness probing.
• Score: 0.47248250311484113
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Testing object detectors in safety-critical domains requires semantically meaningful probes beyond pixel-level corruptions. We present SemProbe, a tool for semantic robustness probing: users upload deployment images, create masks manually or automatically, select operational design domain-derived factors (or custom prompts), and run diffusion-based controlled inpainting. The system supports batch jobs, parallel seed/workflow variations, and configurable generation parameters. After each output, model inference runs automatically and displays annotated before/after comparisons with performance deltas. All probes are logged as structured artifacts, enabling traceable robustness evidence aligned with safety evaluation workflows. We demonstrate \textsc{SemProbe} on hand detection for dimension saws, targeting factors from insurance-oriented test criteria.

Related papers

• ScrambleToolBench: Agents Search Exhaustively Even When Their Own Map Points to the Next Step [74.83535434786145]
  We introduce ScrambleToolBench, an interactive terminal benchmark designed to isolate behavioral reasoning.<n>The benchmark requires agents to uncover hidden tool behaviors entirely through trial-and-error interaction.<n>Our evaluation of state-of-the-art language models reveals that successful initial discovery does not translate into robust adaptation.
  arXiv  Detail & Related papers  (2026-08-03T15:07:46Z)
• TraceScope: Interactive URL Triage via Decoupled Checklist Adjudication [14.375064108289115]
  We present TraceScope, a decoupled triage pipeline that operationalizes this workflow at scale.<n>To prevent the observer effect and ensure safety, a sandboxed operator agent drives a real GUI browser guided by visual motivation to elicit page behavior.<n>An adjudicator agent circumvents LLM context limitations by querying evidence on demand to verify a MITRE ATT&CK checklist, and generates an audit-ready report with extracted indicators of compromise (IOCs)<n>Our evaluation reveals that TraceScope demonstrates superior performance in a real-world scenario as well, successfully detecting sophisticated phishing attempts that current state-of-the-art defenses fail to identify
  arXiv  Detail & Related papers  (2026-04-23T16:31:42Z)
• Attesting LLM Pipelines: Enforcing Verifiable Training and Release Claims [2.0403981727850233]
  Modern Large Language Model (LLM) systems are assembled from third-party artifacts such as pre-trained weights, fine-tuning adapters, datasets, dependency packages, and container images.<n>This speed comes with supply-chain risks, including compromised dependencies, malicious hub artifacts, unsafe dedefaultization, forged provenance, and backdoored models.<n>We propose an attestation-aware promotion gate: before an artifact is admitted into trusted environments, the gate verifies claim evidence, enforces safe loading and static scanning policies, and applies secure-by-serial deployment constraints.
  arXiv  Detail & Related papers  (2026-03-30T20:37:48Z)
• ANCHOR: Branch-Point Data Generation for GUI Agents [52.22377425487]
  End-to-end GUI agents for real desktop environments require large amounts of high-quality interaction data.<n>We present a trajectory expansion framework Anchor that bootstraps scalable desktop supervision from a small set of verified seed demonstrations.<n>Experiments on standard desktop benchmarks, OSWorld and WindowsAgentArena, show that models fine-tuned on our expanded corpus achieve consistent improvements.
  arXiv  Detail & Related papers  (2026-02-06T19:55:26Z)
• ReasAlign: Reasoning Enhanced Safety Alignment against Prompt Injection Attack [52.17935054046577]
  We present ReasAlign, a model-level solution to improve safety alignment against indirect prompt injection attacks.<n>ReasAlign incorporates structured reasoning steps to analyze user queries, detect conflicting instructions, and preserve the continuity of the user's intended tasks.
  arXiv  Detail & Related papers  (2026-01-15T08:23:38Z)
• CaMeLs Can Use Computers Too: System-level Security for Computer Use Agents [60.98294016925157]
  AI agents are vulnerable to prompt injection attacks, where malicious content hijacks agent behavior to steal credentials or cause financial loss.<n>We introduce Single-Shot Planning for CUAs, where a trusted planner generates a complete execution graph with conditional branches before any observation of potentially malicious content.<n>Although this architectural isolation successfully prevents instruction injections, we show that additional measures are needed to prevent Branch Steering attacks.
  arXiv  Detail & Related papers  (2026-01-14T23:06:35Z)
• Continual Adaptation: Environment-Conditional Parameter Generation for Object Detection in Dynamic Scenarios [54.58186816693791]
  environments constantly change over time and space, posing significant challenges for object detectors trained based on a closed-set assumption.<n>We propose a new mechanism, converting the fine-tuning process to a specific- parameter generation.<n>In particular, we first design a dual-path LoRA-based domain-aware adapter that disentangles features into domain-invariant and domain-specific components.
  arXiv  Detail & Related papers  (2025-06-30T17:14:12Z)
• Towards Building Self-Aware Object Detectors via Reliable Uncertainty Quantification and Calibration [17.461451218469062]
  In this work, we introduce the Self-Aware Object Detection (SAOD) task. The SAOD task respects and adheres to the challenges that object detectors face in safety-critical environments such as autonomous driving. We extensively use our framework, which introduces novel metrics and large scale test datasets, to test numerous object detectors.
  arXiv  Detail & Related papers  (2023-07-03T11:16:39Z)
• Auditing AI models for Verified Deployment under Semantic Specifications [65.12401653917838]
  AuditAI bridges the gap between interpretable formal verification and scalability. We show how AuditAI allows us to obtain controlled variations for verification and certified training while addressing the limitations of verifying using only pixel-space perturbations.
  arXiv  Detail & Related papers  (2021-09-25T22:53:24Z)
• Robust Object Detection via Instance-Level Temporal Cycle Confusion [89.1027433760578]
  We study the effectiveness of auxiliary self-supervised tasks to improve the out-of-distribution generalization of object detectors. Inspired by the principle of maximum entropy, we introduce a novel self-supervised task, instance-level temporal cycle confusion (CycConf) For each object, the task is to find the most different object proposals in the adjacent frame in a video and then cycle back to itself for self-supervision.
  arXiv  Detail & Related papers  (2021-04-16T21:35:08Z)
• Online Monitoring of Object Detection Performance During Deployment [6.166295570030645]
  We introduce a cascaded neural network that monitors the performance of the object detector by predicting the quality of its mean average precision (mAP) on a sliding window of the input frames. We evaluate our proposed approach using different combinations of autonomous driving datasets and object detectors.
  arXiv  Detail & Related papers  (2020-11-16T07:01:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.