Auto-ART: Structured Literature Synthesis and Automated Adversarial Robustness Testing

• URL: http://arxiv.org/abs/2604.20704v1
• Date: Wed, 22 Apr 2026 15:46:11 GMT
• Title: Auto-ART: Structured Literature Synthesis and Automated Adversarial Robustness Testing
• Abstract summary: We analyze nine peer-reviewed corpus sources through seven complementary protocols.<n>We introduce Auto-ART, an open-source framework that operationalizes identified gaps.<n> Empirical validation on RobustBench demonstrates that Auto-ART's pre-screening identifies gradient masking in 92% of flagged cases.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Adversarial robustness evaluation underpins every claim of trustworthy ML deployment, yet the field suffers from fragmented protocols and undetected gradient masking. We make two contributions. (1) Structured synthesis. We analyze nine peer-reviewed corpus sources (2020--2026) through seven complementary protocols, producing the first end-to-end structured analysis of the field's consensus and unresolved challenges. (2) Auto-ART framework. We introduce Auto-ART, an open-source framework that operationalizes identified gaps: 50+ attacks, 28 defense modules, the Robustness Diagnostic Index (RDI), and gradient-masking detection. It supports multi-norm evaluation (l1/l2/linf/semantic/spatial) and compliance mapping to NIST AI RMF, OWASP LLM Top 10, and the EU AI Act. Empirical validation on RobustBench demonstrates that Auto-ART's pre-screening identifies gradient masking in 92% of flagged cases, and RDI rankings correlate highly with full AutoAttack. Multi-norm evaluation exposes a 23.5 pp gap between average and worst-case robustness on state-of-the-art models. No prior work combines such structured meta-scientific analysis with an executable evaluation framework bridging literature gaps into engineering.

Related papers

• Unlocking Biological Workflows for Robust Protein-Text Question Answering: A Dual-Dimensional RAG Framework [60.82334952881798]
  Protein-Text Question Answering (QA) is crucial for interpreting biological sequences through natural language.<n>We propose 2D-ProteinRAG, a novel framework that empowers Large Language Models (LLMs) to operate within the gold-standard biological research workflow.<n>We show that 2D-Protein consistently achieves state-of-the-art performance, outperforming fine-tuned baselines and other RAG methods.
  arXiv  Detail & Related papers  (2026-05-17T05:03:24Z)
• Strategic Heterogeneous Multi-Agent Architecture for Cost-Effective Code Vulnerability Detection [0.0]
  We propose a heterogeneous multi-agent architecture inspired by game-theoretic principles.<n>"3+1" deploys three cloud-based expert agents that analyze code from complementary perspectives.<n>Our approach achieves a 77.2% F1 score with 62.9% precision and 100% recall at $0.002 per sample.
  arXiv  Detail & Related papers  (2026-04-23T04:58:18Z)
• Reasoning-Aware AIGC Detection via Alignment and Reinforcement [55.09684020007737]
  REVEAL is a framework that generates interpretable reasoning chains before classification.<n>It achieves state-of-the-art performance across multiple benchmarks.
  arXiv  Detail & Related papers  (2026-04-21T07:29:55Z)
• Are we still able to recognize pearls? Machine-driven peer review and the risk to creativity: An explainable RAG-XAI detection framework with markers extraction [7.723181091241251]
  We introduce an explainable framework (RAG-XAI) for assessing review quality and detecting automated patterns.<n>The framework achieves near-perfect detection performance, with XGBoost, Random Forest and LightGBM reaching 99.61% accuracy, AUC-ROC above 0.999 and F1-scores of 0.9925 on the test set.
  arXiv  Detail & Related papers  (2026-04-09T08:25:49Z)
• PSR2: A Phase-based Semantic Reasoning Framework for Atomicity Violation Detection via Contract Refinement [9.36682660657815]
  PSRtextsuperscript2 is a novel collaborative static analysis framework that integrates structural path searching with deterministic semantic reasoning.<n>Experiments on 1,600 contract samples demonstrate that PSRtextsuperscript2 significantly outperforms pattern-matching baselines.<n>Our fusion logic effectively reduces the false-positive rate by nearly half compared to single module analysis.
  arXiv  Detail & Related papers  (2026-04-08T11:46:01Z)
• Dynamic analysis enhances issue resolution [53.50448142467294]
  DAIRA (Dynamic Analysis-enhanced Issue Resolution Agent) is an automated repair framework that embeds dynamic analysis into the agent's reasoning cycle.<n> Driven by a Test Tracing-Driven methodology, DAIRA utilizes lightweight monitors to extract critical runtime data.<n>When powered by Gemini 3 Flash Preview, DAIRA establishes a new state-of-the-art (SOTA) performance, achieving a 79.4% resolution rate on the SWE-bench Verified dataset.
  arXiv  Detail & Related papers  (2026-03-23T14:48:54Z)
• Synergistic Directed Execution and LLM-Driven Analysis for Zero-Day AI-Generated Malware Detection [0.12891210250935145]
  Weaponization of LLMs for automated malware generation poses an existential threat to conventional detection paradigms.<n>This paper introduces a novel hybrid analysis framework that combines emphconcolic execution with emphdeep-learning-based vulnerability classification.<n>A benchmark comprising 2,500 LLM-synthesized samples achieves 98.7% accuracy on conventional malware and 97.5% accuracy on AI-generated threats.
  arXiv  Detail & Related papers  (2026-03-10T00:25:41Z)
• Chunking, Retrieval, and Re-ranking: An Empirical Evaluation of RAG Architectures for Policy Document Question Answering [0.0]
  The integration of Large Language Models (LLMs) into the public health policy sector offers a transformative approach to navigating the vast repositories of regulatory guidance maintained by agencies such as the Centers for Disease Control and Prevention (CDC)<n>The propensity for LLMs to generate hallucinations, defined as plausible but factually incorrect assertions, presents a critical barrier to the adoption of these technologies in high-stakes environments where information integrity is non-negotiable.<n>This empirical evaluation explores the effectiveness of Retrieval-Augmented Generation (RAG) architectures in mitigating these risks by grounding generative outputs in authoritative document context.
  arXiv  Detail & Related papers  (2026-01-21T20:52:48Z)
• SSVP: Synergistic Semantic-Visual Prompting for Industrial Zero-Shot Anomaly Detection [55.54007781679915]
  We propose Synergistic Semantic-Visual Prompting (SSVP), that efficiently fuses diverse visual encodings to elevate model's fine-grained perception.<n>SSVP achieves state-of-the-art performance with 93.0% Image-AUROC and 92.2% Pixel-AUROC on MVTec-AD, significantly outperforming existing zero-shot approaches.
  arXiv  Detail & Related papers  (2026-01-14T04:42:19Z)
• The Great Nugget Recall: Automating Fact Extraction and RAG Evaluation with Large Language Models [53.12387628636912]
  We propose an automatic evaluation framework that is validated against human annotations.<n>This approach was originally developed for the TREC Question Answering (QA) Track in 2003.<n>We observe strong agreement at the run level between scores derived from fully automatic nugget evaluation and human-based variants.
  arXiv  Detail & Related papers  (2025-04-21T12:55:06Z)
• Retrieval is Not Enough: Enhancing RAG Reasoning through Test-Time Critique and Optimization [58.390885294401066]
  Retrieval-augmented generation (RAG) has become a widely adopted paradigm for enabling knowledge-grounded large language models (LLMs)<n>RAG pipelines often fail to ensure that model reasoning remains consistent with the evidence retrieved, leading to factual inconsistencies or unsupported conclusions.<n>We propose AlignRAG, a novel iterative framework grounded in Critique-Driven Alignment (CDA)<n>We introduce AlignRAG-auto, an autonomous variant that dynamically terminates refinement, removing the need to pre-specify the number of critique iterations.
  arXiv  Detail & Related papers  (2025-04-21T04:56:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.