A Large-Scale, Cross-Disciplinary Corpus of Systematic Reviews

• URL: http://arxiv.org/abs/2604.22864v1
• Date: Thu, 23 Apr 2026 07:56:50 GMT
• Title: A Large-Scale, Cross-Disciplinary Corpus of Systematic Reviews
• Abstract summary: Webis-SR4ALL-26 is a large-scale, cross-disciplinary corpus of 301,871 systematic reviews spanning all scientific fields as covered by OpenAlex.<n>Using a multi-stage pre-processing pipeline, we link reviews to resolved OpenAlex metadata and reference lists and extract, when explicitly reported, structured method artifacts relevant to retrieval and screening.
• Score: 20.88655348694586
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Existing benchmarks for systematic reviewing remain limited either in scale or in disciplinary coverage, with some collections comprising only a modest number of topics and others focusing primarily on biomedical research. We present Webis-SR4ALL-26, a large-scale, cross-disciplinary corpus of 301,871 systematic reviews spanning all scientific fields as covered by OpenAlex. Using a multi-stage pre-processing pipeline, we link reviews to resolved OpenAlex metadata and reference lists and extract, when explicitly reported, structured method artifacts relevant to retrieval and screening. These artifacts include reported search strategies (Boolean queries or keyword lists) that we normalize into executable approximations, as well as reported inclusion and exclusion criteria. Together, these layers support cross-domain benchmarking of retrieval and screening components against review reference lists, training and evaluation of extraction methods for review artifacts, and comparative meta-science analyses of systematic review practices across disciplines and time. To demonstrate one concrete use case, we report large-scale baseline retrieval signals by executing normalized search strategies in OpenAlex and comparing retrieved sets to resolved reference lists. We release the corpus and the pre-processing pipeline, along with code used for extraction validation and the retrieval demonstration.

Related papers

• When Should Queries Be Decomposed? A Stage-Aware Study of Query Decomposition for Multi-Condition Retrieval [7.14438992840294]
  Multi-condition retrieval requires systems to identify documents that satisfy multiple distinct constraints.<n> query decomposition is widely adopted as an intuitive remedy, but its effectiveness across different retrieval pipeline stages remains underexplored.<n>We propose a principled Stage-Aware Decomposition framework that retains the monolithic query during initial retrieval to preserve global semantic context, while employing sub-queries exclusively during reranking for fine-grained constraint matching.
  arXiv  Detail & Related papers  (2026-06-07T11:13:15Z)
• ReadingMachine: A Computational Methodology for Structured Corpus Reading and Large-Scale Synthesis [0.0]
  ReadingMachine is a computational methodology for structured corpus reading.<n>It decomposes analysis into inspectable stages including insight extraction and semantic clustering.<n>System is demonstrated on a heterogeneous corpus of 152 industrial policy documents.
  arXiv  Detail & Related papers  (2026-06-05T18:01:27Z)
• Beyond Semantic Similarity: Rethinking Retrieval for Agentic Search via Direct Corpus Interaction [127.64173950476702]
  We study direct corpus interaction (DCI), where an agent searches the raw corpus directly with general-purpose terminal tools.<n>This approach requires no offline indexing and adapts naturally to evolving local corpora.<n>Across IR benchmarks and end-to-end agentic search tasks, this simple setup substantially outperforms strong sparse, dense, and reranking baselines.
  arXiv  Detail & Related papers  (2026-05-03T19:13:11Z)
• Unsupervised Evaluation of Deep Audio Embeddings for Music Structure Analysis [0.0]
  Music Structure Analysis (MSA) aims to uncover the high-level organization of musical pieces.<n>We propose an unsupervised evaluation of nine open-source, generic pre-trained deep audio models, on MSA.
  arXiv  Detail & Related papers  (2026-03-28T10:18:54Z)
• An Artificial Intelligence Driven Semantic Similarity-Based Pipeline for Rapid Literature [0.0]
  We propose an automated pipeline for performing literature reviews using semantic similarity.<n>This work emphasizes minimal overhead and high relevance by using transformer based embeddings and cosine similarity.
  arXiv  Detail & Related papers  (2025-09-18T17:24:47Z)
• Reasoning-enhanced Query Understanding through Decomposition and Interpretation [87.56450566014625]
  ReDI is a Reasoning-enhanced approach for query understanding through Decomposition and Interpretation.<n>We compiled a large-scale dataset of real-world complex queries from a major search engine.<n> Experiments on BRIGHT and BEIR demonstrate that ReDI consistently surpasses strong baselines in both sparse and dense retrieval paradigms.
  arXiv  Detail & Related papers  (2025-09-08T10:58:42Z)
• Ranking Narrative Query Graphs for Biomedical Document Retrieval (Technical Report) [7.527096697768715]
  This paper extends our existing graph-based discovery system for the biomedical domain.<n>It contributes effective graph-based unsupervised ranking methods, a new query relaxation paradigm, and ontological rewriting.
  arXiv  Detail & Related papers  (2024-12-06T12:49:28Z)
• JudgeRank: Leveraging Large Language Models for Reasoning-Intensive Reranking [81.88787401178378]
  We introduce JudgeRank, a novel agentic reranker that emulates human cognitive processes when assessing document relevance. We evaluate JudgeRank on the reasoning-intensive BRIGHT benchmark, demonstrating substantial performance improvements over first-stage retrieval methods. In addition, JudgeRank performs on par with fine-tuned state-of-the-art rerankers on the popular BEIR benchmark, validating its zero-shot generalization capability.
  arXiv  Detail & Related papers  (2024-10-31T18:43:12Z)
• ExcluIR: Exclusionary Neural Information Retrieval [74.08276741093317]
  We present ExcluIR, a set of resources for exclusionary retrieval. evaluation benchmark includes 3,452 high-quality exclusionary queries. training set contains 70,293 exclusionary queries, each paired with a positive document and a negative document.
  arXiv  Detail & Related papers  (2024-04-26T09:43:40Z)
• Hierarchical Indexing for Retrieval-Augmented Opinion Summarization [60.5923941324953]
  We propose a method for unsupervised abstractive opinion summarization that combines the attributability and scalability of extractive approaches with the coherence and fluency of Large Language Models (LLMs) Our method, HIRO, learns an index structure that maps sentences to a path through a semantically organized discrete hierarchy. At inference time, we populate the index and use it to identify and retrieve clusters of sentences containing popular opinions from input reviews.
  arXiv  Detail & Related papers  (2024-03-01T10:38:07Z)
• Exposing Query Identification for Search Transparency [69.06545074617685]
  We explore the feasibility of approximate exposing query identification (EQI) as a retrieval task by reversing the role of queries and documents in two classes of search systems. We derive an evaluation metric to measure the quality of a ranking of exposing queries, as well as conducting an empirical analysis focusing on various practical aspects of approximate EQI.
  arXiv  Detail & Related papers  (2021-10-14T20:19:27Z)
• Abstract, Rationale, Stance: A Joint Model for Scientific Claim Verification [18.330265729989843]
  We propose an approach, named as ARSJoint, that jointly learns the modules for the three tasks with a machine reading comprehension framework. The experimental results on the benchmark dataset SciFact show that our approach outperforms the existing works.
  arXiv  Detail & Related papers  (2021-09-13T10:07:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.