From text to multimodal: a survey of adversarial example generation in question answering systems

• URL: http://arxiv.org/abs/2312.16156v2
• Date: Fri, 9 Aug 2024 20:17:51 GMT
• Title: From text to multimodal: a survey of adversarial example generation in question answering systems
• Authors: Gulsum Yigit, Mehmet Fatih Amasyali,
• Abstract summary: This article aims to comprehensively review adversarial example-generation techniques in the Question Answering (QA) field. We examine the techniques employed through systematic categorization, providing a comprehensive, structured review. The paper considers the future landscape of adversarial question generation, highlighting potential research directions.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Integrating adversarial machine learning with Question Answering (QA) systems has emerged as a critical area for understanding the vulnerabilities and robustness of these systems. This article aims to comprehensively review adversarial example-generation techniques in the QA field, including textual and multimodal contexts. We examine the techniques employed through systematic categorization, providing a comprehensive, structured review. Beginning with an overview of traditional QA models, we traverse the adversarial example generation by exploring rule-based perturbations and advanced generative models. We then extend our research to include multimodal QA systems, analyze them across various methods, and examine generative models, seq2seq architectures, and hybrid methodologies. Our research grows to different defense strategies, adversarial datasets, and evaluation metrics and illustrates the comprehensive literature on adversarial QA. Finally, the paper considers the future landscape of adversarial question generation, highlighting potential research directions that can advance textual and multimodal QA systems in the context of adversarial challenges.

Related papers

• Multi-Faceted Question Complexity Estimation Targeting Topic Domain-Specificity [0.0]
  This paper presents a novel framework for domain-specific question difficulty estimation, leveraging a suite of NLP techniques and knowledge graph analysis. We introduce four key parameters: Topic Retrieval Cost, Topic Salience, Topic Coherence, and Topic Superficiality. A model trained on these features demonstrates the efficacy of our approach in predicting question difficulty.
  arXiv  Detail & Related papers  (2024-08-23T05:40:35Z)
• Towards Robust Evaluation: A Comprehensive Taxonomy of Datasets and Metrics for Open Domain Question Answering in the Era of Large Language Models [0.0]
  Open Domain Question Answering (ODQA) within natural language processing involves building systems that answer factual questions using large-scale knowledge corpora. High-quality datasets are used to train models on realistic scenarios. Standardized metrics facilitate comparisons between different ODQA systems.
  arXiv  Detail & Related papers  (2024-06-19T05:43:02Z)
• A Survey on Context-Aware Multi-Agent Systems: Techniques, Challenges and Future Directions [1.1458366773578277]
  Research interest in autonomous agents is on the rise as an emerging topic. The challenge lies in enabling these agents to learn, reason, and navigate uncertainties in dynamic environments. Context awareness emerges as a pivotal element in fortifying multi-agent systems.
  arXiv  Detail & Related papers  (2024-02-03T00:27:22Z)
• Recent Advances in Hate Speech Moderation: Multimodality and the Role of Large Models [52.24001776263608]
  This comprehensive survey delves into the recent strides in HS moderation. We highlight the burgeoning role of large language models (LLMs) and large multimodal models (LMMs) We identify existing gaps in research, particularly in the context of underrepresented languages and cultures.
  arXiv  Detail & Related papers  (2024-01-30T03:51:44Z)
• Foundational Models Defining a New Era in Vision: A Survey and Outlook [151.49434496615427]
  Vision systems to see and reason about the compositional nature of visual scenes are fundamental to understanding our world. The models learned to bridge the gap between such modalities coupled with large-scale training data facilitate contextual reasoning, generalization, and prompt capabilities at test time. The output of such models can be modified through human-provided prompts without retraining, e.g., segmenting a particular object by providing a bounding box, having interactive dialogues by asking questions about an image or video scene or manipulating the robot's behavior through language instructions.
  arXiv  Detail & Related papers  (2023-07-25T17:59:18Z)
• Enhancing Human-like Multi-Modal Reasoning: A New Challenging Dataset and Comprehensive Framework [51.44863255495668]
  Multimodal reasoning is a critical component in the pursuit of artificial intelligence systems that exhibit human-like intelligence. We present Multi-Modal Reasoning(COCO-MMR) dataset, a novel dataset that encompasses an extensive collection of open-ended questions. We propose innovative techniques, including multi-hop cross-modal attention and sentence-level contrastive learning, to enhance the image and text encoders.
  arXiv  Detail & Related papers  (2023-07-24T08:58:25Z)
• How Many Answers Should I Give? An Empirical Study of Multi-Answer Reading Comprehension [64.76737510530184]
  We design a taxonomy to categorize commonly-seen multi-answer MRC instances. We analyze how well different paradigms of current multi-answer MRC models deal with different types of multi-answer instances.
  arXiv  Detail & Related papers  (2023-06-01T08:22:21Z)
• Attacks in Adversarial Machine Learning: A Systematic Survey from the Life-cycle Perspective [69.25513235556635]
  Adversarial machine learning (AML) studies the adversarial phenomenon of machine learning, which may make inconsistent or unexpected predictions with humans. Some paradigms have been recently developed to explore this adversarial phenomenon occurring at different stages of a machine learning system. We propose a unified mathematical framework to covering existing attack paradigms.
  arXiv  Detail & Related papers  (2023-02-19T02:12:21Z)
• Foundations and Recent Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questions [68.6358773622615]
  This paper provides an overview of the computational and theoretical foundations of multimodal machine learning. We propose a taxonomy of 6 core technical challenges: representation, alignment, reasoning, generation, transference, and quantification. Recent technical achievements will be presented through the lens of this taxonomy, allowing researchers to understand the similarities and differences across new approaches.
  arXiv  Detail & Related papers  (2022-09-07T19:21:19Z)
• Modeling Transformative AI Risks (MTAIR) Project -- Summary Report [0.0]
  This report builds on an earlier diagram by Cottier and Shah which laid out some of the crucial disagreements ("cruxes") visually, with some explanation. The model starts with a discussion of reasoning via analogies and general prior beliefs about artificial intelligence. It lays out a model of different paths and enabling technologies for high-level machine intelligence, and a model of how advances in the capabilities of these systems might proceed. The model also looks specifically at the question of learned optimization, and whether machine learning systems will create mesa-optimizers.
  arXiv  Detail & Related papers  (2022-06-19T09:11:23Z)
• Conversational Question Answering: A Survey [18.447856993867788]
  This survey is an effort to present a comprehensive review of the state-of-the-art research trends of Conversational Question Answering (CQA) Our findings show that there has been a trend shift from single-turn to multi-turn QA which empowers the field of Conversational AI from different perspectives.
  arXiv  Detail & Related papers  (2021-06-02T01:06:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.