Adversarial Question Answering Robustness: A Multi-Level Error Analysis and Mitigation Study

• URL: http://arxiv.org/abs/2601.02700v1
• Date: Tue, 06 Jan 2026 04:20:33 GMT
• Title: Adversarial Question Answering Robustness: A Multi-Level Error Analysis and Mitigation Study
• Abstract summary: Question answering (QA) systems achieve impressive performance on standard benchmarks like SQuAD, but remain vulnerable to adversarial examples.<n>This project investigates the adversarial robustness of transformer models on the AddSent adversarial dataset.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Question answering (QA) systems achieve impressive performance on standard benchmarks like SQuAD, but remain vulnerable to adversarial examples. This project investigates the adversarial robustness of transformer models on the AddSent adversarial dataset through systematic experimentation across model scales and targeted mitigation strategies. We perform comprehensive multi-level error analysis using five complementary categorization schemes, identifying negation confusion and entity substitution as the primary failure modes. Through systematic evaluation of adversarial fine-tuning ratios, we identify 80% clean + 20% adversarial data as optimal. Data augmentation experiments reveal a capacity bottleneck in small models. Scaling from ELECTRA-small (14M parameters) to ELECTRA-base (110M parameters) eliminates the robustness-accuracy trade-off, achieving substantial improvements on both clean and adversarial data. We implement three targeted mitigation strategies, with Entity-Aware contrastive learning achieving best performance: 89.89% AddSent Exact Match (EM) and 90.73% SQuAD EM, representing 94.9% closure of the adversarial gap. To our knowledge, this is the first work integrating comprehensive linguistic error analysis with Named Entity Recognition (NER)-guided contrastive learning for adversarial QA, demonstrating that targeted mitigation can achieve near-parity between clean and adversarial performance.

Related papers

• Robustness Evaluation and Detection of Transferable Adversarial Attacks in ML-Based NIDS [0.0]
  We evaluate eight adversarial evasion attacks, fifteen detection models, and three representative defense strategies using the NF-UQ-NIDS dataset.<n>We present AR-NIDS, a two-stage framework that uses an adversarially trained ensemble to distinguish normal, attack, and adversarial samples.
  arXiv  Detail & Related papers  (2026-09-05T10:34:14Z)
• Robust and Personalized Federated Learning for Aircraft-Engine Prognostics under Benign and Adversarial Client Heterogeneity [0.4776836972093627]
  Federated learning (FL) enables aircraft fleet operators to jointly train remaining-usefullife (RUL) models from engine sensor telemetry without sharing raw data.<n>This study examines two complementary challenges: benign heterogeneity, where honest operators observe different operating conditions and fault modes, and adversarial heterogeneity, where compromised operators submit poisoned updates.<n>We evaluate five attacks against four aggregation methods, including a physically motivated sensor-value backdoor designed to mask engine degradation.
  arXiv  Detail & Related papers  (2026-08-04T06:54:29Z)
• Proactive for Uncertainty: Cause-Aware Error Diagnosis and Interactive Clarification for Spoken Dialogue Systems [40.265250435329456]
  We propose a cause-aware error recovery paradigm that fundamentally rethinks robustness in SDS.<n>Unlike traditional confidence filtering, we introduce a suite of small precision-focused detectors.<n>This fine-grained diagnostic intelligence empowers the LLM to orchestrate targeted, multi-turn clarification strategies.
  arXiv  Detail & Related papers  (2026-05-25T03:57:38Z)
• Rewarding the Scientific Process: Process-Level Reward Modeling for Agentic Data Analysis [68.28714988482703]
  Process Reward Models (PRMs) have achieved remarkable success in augmenting the reasoning capabilities of Large Language Models (LLMs)<n>We present a study revealing that general-domain PRMs struggle to supervise data analysis agents.<n>We introduce DataPRM, a novel environment-aware generative process reward model.
  arXiv  Detail & Related papers  (2026-04-27T09:00:30Z)
• ECI: Effective Contrastive Information to Evaluate Hard-Negatives [0.3674670871245149]
  ECI: Effective Contrastive Information is a metric grounded in Information Theory and Information Retrieval principles.<n>We evaluate ECI across hard-negative sets mined or generated using BM25, cross-encoders, and large language models.
  arXiv  Detail & Related papers  (2026-03-22T00:21:05Z)
• Enhancing the QA Model through a Multi-domain Debiasing Framework [1.7802147489386633]
  This study evaluates the ELECTRA-small model on the Stanford Question Answering dataset (SQuAD) v1.1 and adversarial datasets AddSent and AddOneSent.<n>We develop a multi-domain debiasing framework incorporating knowledge distillation, debiasing techniques, and domain expansion.
  arXiv  Detail & Related papers  (2026-01-01T08:39:07Z)
• Elevating Intrusion Detection and Security Fortification in Intelligent Networks through Cutting-Edge Machine Learning Paradigms [5.706727902661187]
  This study proposes a robust multiclass machine learning based intrusion detection framework.<n>It integrates advanced feature selection techniques to identify critical attributes, mitigating redundancy and enhancing detection accuracy.<n>The proposed ensemble architecture achieves superior performance, with an accuracy of 98%, precision of 98%, recall of 98%, and a false positive rate of just 2%.
  arXiv  Detail & Related papers  (2025-12-22T05:14:26Z)
• Towards a Science of Scaling Agent Systems [79.64446272302287]
  We formalize a definition for agent evaluation and characterize scaling laws as the interplay between agent quantity, coordination structure, modelic, and task properties.<n>We derive a predictive model using coordination metrics, that cross-validated R2=0, enabling prediction on unseen task domains.<n>We identify three effects: (1) a tool-coordination trade-off: under fixed computational budgets, tool-heavy tasks suffer disproportionately from multi-agent overhead, and (2) a capability saturation: coordination yields diminishing or negative returns once single-agent baselines exceed 45%.
  arXiv  Detail & Related papers  (2025-12-09T06:52:21Z)
• RoHOI: Robustness Benchmark for Human-Object Interaction Detection [84.78366452133514]
  Human-Object Interaction (HOI) detection is crucial for robot-human assistance, enabling context-aware support.<n>We introduce the first benchmark for HOI detection, evaluating model resilience under diverse challenges.<n>Our benchmark, RoHOI, includes 20 corruption types based on the HICO-DET and V-COCO datasets and a new robustness-focused metric.
  arXiv  Detail & Related papers  (2025-07-12T01:58:04Z)
• Sampling-aware Adversarial Attacks Against Large Language Models [52.30089653615172]
  Existing adversarial attacks typically target harmful responses in single-point greedy generations.<n>We show that for the goal of eliciting harmful responses, repeated sampling of model outputs during the attack prompt optimization.<n>We show that integrating sampling into existing attacks boosts success rates by up to 37% and improves efficiency by up to two orders of magnitude.
  arXiv  Detail & Related papers  (2025-07-06T16:13:33Z)
• A Few Large Shifts: Layer-Inconsistency Based Minimal Overhead Adversarial Example Detection [13.109309606764754]
  We introduce a plug-in detection framework that leverages internal layer-wise inconsistencies within the target model itself.<n>Our method achieves state-of-the-art detection performance with negligible computational overhead.
  arXiv  Detail & Related papers  (2025-05-19T00:48:53Z)
• MOS: Towards Effective Smart Contract Vulnerability Detection through Mixture-of-Experts Tuning of Large Language Models [16.16186929130931]
  Smart contract vulnerabilities pose significant security risks to blockchain systems.<n>We propose a smart contract vulnerability detection framework based on mixture-of-experts tuning (MOE-Tuning) of large language models.<n> Experiments show that MOS significantly outperforms existing methods with average improvements of 6.32% in F1 score and 4.80% in accuracy.
  arXiv  Detail & Related papers  (2025-04-16T16:33:53Z)
• Provable Optimization for Adversarial Fair Self-supervised Contrastive Learning [49.417414031031264]
  This paper studies learning fair encoders in a self-supervised learning setting. All data are unlabeled and only a small portion of them are annotated with sensitive attributes.
  arXiv  Detail & Related papers  (2024-06-09T08:11:12Z)
• Advancing Adversarial Robustness Through Adversarial Logit Update [10.041289551532804]
  Adversarial training and adversarial purification are among the most widely recognized defense strategies. We propose a new principle, namely Adversarial Logit Update (ALU), to infer adversarial sample's labels. Our solution achieves superior performance compared to state-of-the-art methods against a wide range of adversarial attacks.
  arXiv  Detail & Related papers  (2023-08-29T07:13:31Z)
• Delving into Identify-Emphasize Paradigm for Combating Unknown Bias [52.76758938921129]
  We propose an effective bias-conflicting scoring method (ECS) to boost the identification accuracy. We also propose gradient alignment (GA) to balance the contributions of the mined bias-aligned and bias-conflicting samples. Experiments are conducted on multiple datasets in various settings, demonstrating that the proposed solution can mitigate the impact of unknown biases.
  arXiv  Detail & Related papers  (2023-02-22T14:50:24Z)
• Alternating Objectives Generates Stronger PGD-Based Adversarial Attacks [78.2700757742992]
  Projected Gradient Descent (PGD) is one of the most effective and conceptually simple algorithms to generate such adversaries. We experimentally verify this assertion on a synthetic-data example and by evaluating our proposed method across 25 different $ell_infty$-robust models and 3 datasets. Our strongest adversarial attack outperforms all of the white-box components of AutoAttack ensemble.
  arXiv  Detail & Related papers  (2022-12-15T17:44:31Z)
• Robust Trajectory Prediction against Adversarial Attacks [84.10405251683713]
  Trajectory prediction using deep neural networks (DNNs) is an essential component of autonomous driving systems. These methods are vulnerable to adversarial attacks, leading to serious consequences such as collisions. In this work, we identify two key ingredients to defend trajectory prediction models against adversarial attacks.
  arXiv  Detail & Related papers  (2022-07-29T22:35:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.