ir_explain: a Python Library of Explainable IR Methods

• URL: http://arxiv.org/abs/2404.18546v1
• Date: Mon, 29 Apr 2024 09:37:24 GMT
• Title: ir_explain: a Python Library of Explainable IR Methods
• Authors: Sourav Saha, Harsh Agarwal, Swastik Mohanty, Mandar Mitra, Debapriyo Majumdar,
• Abstract summary: irexplain is a Python library that implements a variety of techniques for Explainable IR (ExIR) within a common framework. irexplain supports the three standard categories of post-hoc explanations, namely pointwise, pairwise, and listwise explanations. The library is designed to make it easy to reproduce state-of-the-art ExIR baselines on standard test collections.
• Score: 2.6746131626710725
• License:
• Abstract: While recent advancements in Neural Ranking Models have resulted in significant improvements over traditional statistical retrieval models, it is generally acknowledged that the use of large neural architectures and the application of complex language models in Information Retrieval (IR) have reduced the transparency of retrieval methods. Consequently, Explainability and Interpretability have emerged as important research topics in IR. Several axiomatic and post-hoc explanation methods, as well as approaches that attempt to be interpretable-by-design, have been proposed. This article presents \irexplain, an open-source Python library that implements a variety of well-known techniques for Explainable IR (ExIR) within a common, extensible framework. \irexplain supports the three standard categories of post-hoc explanations, namely pointwise, pairwise, and listwise explanations. The library is designed to make it easy to reproduce state-of-the-art ExIR baselines on standard test collections, as well as to explore new approaches to explaining IR models and methods. To facilitate adoption, \irexplain is well-integrated with widely-used toolkits such as Pyserini and \irdatasets.

Related papers

• Lightning IR: Straightforward Fine-tuning and Inference of Transformer-based Language Models for Information Retrieval [43.40675637622642]
  This paper introduces Lightning IR, a PyTorch-based framework for fine-tuning and inference of transformer-based language models for information retrieval. Lightning IR supports all stages of an information retrieval pipeline: from fine-tuning and indexing to searching and re-ranking.
  arXiv  Detail & Related papers  (2024-11-07T13:03:21Z)
• Robust Neural Information Retrieval: An Adversarial and Out-of-distribution Perspective [111.58315434849047]
  robustness of neural information retrieval models (IR) models has garnered significant attention. We view the robustness of IR to be a multifaceted concept, emphasizing its necessity against adversarial attacks, out-of-distribution (OOD) scenarios and performance variance. We provide an in-depth discussion of existing methods, datasets, and evaluation metrics, shedding light on challenges and future directions in the era of large language models.
  arXiv  Detail & Related papers  (2024-07-09T16:07:01Z)
• Explaining Text Similarity in Transformer Models [52.571158418102584]
  Recent advances in explainable AI have made it possible to mitigate limitations by leveraging improved explanations for Transformers. We use BiLRP, an extension developed for computing second-order explanations in bilinear similarity models, to investigate which feature interactions drive similarity in NLP models. Our findings contribute to a deeper understanding of different semantic similarity tasks and models, highlighting how novel explainable AI methods enable in-depth analyses and corpus-level insights.
  arXiv  Detail & Related papers  (2024-05-10T17:11:31Z)
• UniArk: Improving Generalisation and Consistency for Factual Knowledge Extraction through Debiasing [19.2764682793582]
  We show the inherent misalignment between pre-training and downstream tuning objectives in language models for probing knowledge. We propose an adapter-based framework, UniArk, for generalised and consistent factual knowledge extraction.
  arXiv  Detail & Related papers  (2024-04-01T17:22:07Z)
• ExaRanker-Open: Synthetic Explanation for IR using Open-Source LLMs [60.81649785463651]
  We introduce ExaRanker-Open, where we adapt and explore the use of open-source language models to generate explanations. Our findings reveal that incorporating explanations consistently enhances neural rankers, with benefits escalating as the LLM size increases.
  arXiv  Detail & Related papers  (2024-02-09T11:23:14Z)
• Large Language Models for Information Retrieval: A Survey [58.30439850203101]
  Information retrieval has evolved from term-based methods to its integration with advanced neural models. Recent research has sought to leverage large language models (LLMs) to improve IR systems. We delve into the confluence of LLMs and IR systems, including crucial aspects such as query rewriters, retrievers, rerankers, and readers.
  arXiv  Detail & Related papers  (2023-08-14T12:47:22Z)
• Interpretability at Scale: Identifying Causal Mechanisms in Alpaca [62.65877150123775]
  We use Boundless DAS to efficiently search for interpretable causal structure in large language models while they follow instructions. Our findings mark a first step toward faithfully understanding the inner-workings of our ever-growing and most widely deployed language models.
  arXiv  Detail & Related papers  (2023-05-15T17:15:40Z)
• Understanding Post-hoc Explainers: The Case of Anchors [6.681943980068051]
  We present a theoretical analysis of a rule-based interpretability method that highlights a small set of words to explain a text's decision. After formalizing its algorithm and providing useful insights, we demonstrate mathematically that Anchors produces meaningful results.
  arXiv  Detail & Related papers  (2023-03-15T17:56:34Z)
• CARLA: A Python Library to Benchmark Algorithmic Recourse and Counterfactual Explanation Algorithms [6.133522864509327]
  CARLA (Counterfactual And Recourse LibrAry) is a python library for benchmarking counterfactual explanation methods. We provide an extensive benchmark of 11 popular counterfactual explanation methods. We also provide a benchmarking framework for research on future counterfactual explanation methods.
  arXiv  Detail & Related papers  (2021-08-02T11:00:43Z)
• Coreference Reasoning in Machine Reading Comprehension [100.75624364257429]
  We show that coreference reasoning in machine reading comprehension is a greater challenge than was earlier thought. We propose a methodology for creating reading comprehension datasets that better reflect the challenges of coreference reasoning. This allows us to show an improvement in the reasoning abilities of state-of-the-art models across various MRC datasets.
  arXiv  Detail & Related papers  (2020-12-31T12:18:41Z)
• Analyzing Knowledge Graph Embedding Methods from a Multi-Embedding Interaction Perspective [3.718476964451589]
  Real-world knowledge graphs are usually incomplete, so knowledge graph embedding methods have been proposed to address this issue. These methods represent entities and relations as embedding vectors in semantic space and predict the links between them. We propose a new multi-embedding model based on quaternion algebra and show that it achieves promising results using popular benchmarks.
  arXiv  Detail & Related papers  (2019-03-27T13:09:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.