Transforming Evidence Synthesis: A Systematic Review of the Evolution of Automated Meta-Analysis in the Age of AI

• URL: http://arxiv.org/abs/2504.20113v1
• Date: Mon, 28 Apr 2025 00:40:17 GMT
• Title: Transforming Evidence Synthesis: A Systematic Review of the Evolution of Automated Meta-Analysis in the Age of AI
• Authors: Lingbo Li, Anuradha Mathrani, Teo Susnjak,
• Abstract summary: Automated Meta-analysis is powered by natural language processing and machine learning.<n>Findings reveal a predominant focus on automating data processing (57%), while only 17% address advanced synthesis stages.<n>Despite recent breakthroughs in large language models (LLMs) and advanced AI, integration into statistical modeling and higher-order synthesis remains underdeveloped.
• Score: 0.3441021278275805
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Exponential growth in scientific literature has heightened the demand for efficient evidence-based synthesis, driving the rise of the field of Automated Meta-analysis (AMA) powered by natural language processing and machine learning. This PRISMA systematic review introduces a structured framework for assessing the current state of AMA, based on screening 978 papers from 2006 to 2024, and analyzing 54 studies across diverse domains. Findings reveal a predominant focus on automating data processing (57%), such as extraction and statistical modeling, while only 17% address advanced synthesis stages. Just one study (2%) explored preliminary full-process automation, highlighting a critical gap that limits AMA's capacity for comprehensive synthesis. Despite recent breakthroughs in large language models (LLMs) and advanced AI, their integration into statistical modeling and higher-order synthesis, such as heterogeneity assessment and bias evaluation, remains underdeveloped. This has constrained AMA's potential for fully autonomous meta-analysis. From our dataset spanning medical (67%) and non-medical (33%) applications, we found that AMA has exhibited distinct implementation patterns and varying degrees of effectiveness in actually improving efficiency, scalability, and reproducibility. While automation has enhanced specific meta-analytic tasks, achieving seamless, end-to-end automation remains an open challenge. As AI systems advance in reasoning and contextual understanding, addressing these gaps is now imperative. Future efforts must focus on bridging automation across all meta-analysis stages, refining interpretability, and ensuring methodological robustness to fully realize AMA's potential for scalable, domain-agnostic synthesis.

Related papers

• General Scales Unlock AI Evaluation with Explanatory and Predictive Power [57.7995945974989]
  benchmarking has guided progress in AI, but it has offered limited explanatory and predictive power for general-purpose AI systems.<n>We introduce general scales for AI evaluation that can explain what common AI benchmarks really measure.<n>Our fully-automated methodology builds on 18 newly-crafted rubrics that place instance demands on general scales that do not saturate.
  arXiv  Detail & Related papers  (2025-03-09T01:13:56Z)
• Empowering Meta-Analysis: Leveraging Large Language Models for Scientific Synthesis [7.059964549363294]
  This study investigates the automation of meta-analysis in scientific documents using large language models (LLMs) Our research introduces a novel approach that fine-tunes the LLM on extensive scientific datasets to address challenges in big data handling and structured data extraction.
  arXiv  Detail & Related papers  (2024-11-16T20:18:57Z)
• Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AI [129.08019405056262]
  Embodied Artificial Intelligence (Embodied AI) is crucial for achieving Artificial Intelligence (AGI) MLMs andWMs have attracted significant attention due to their remarkable perception, interaction, and reasoning capabilities. In this survey, we give a comprehensive exploration of the latest advancements in Embodied AI.
  arXiv  Detail & Related papers  (2024-07-09T14:14:47Z)
• Long-Span Question-Answering: Automatic Question Generation and QA-System Ranking via Side-by-Side Evaluation [65.16137964758612]
  We explore the use of long-context capabilities in large language models to create synthetic reading comprehension data from entire books. Our objective is to test the capabilities of LLMs to analyze, understand, and reason over problems that require a detailed comprehension of long spans of text.
  arXiv  Detail & Related papers  (2024-05-31T20:15:10Z)
• An Autonomous Large Language Model Agent for Chemical Literature Data Mining [60.85177362167166]
  We introduce an end-to-end AI agent framework capable of high-fidelity extraction from extensive chemical literature. Our framework's efficacy is evaluated using accuracy, recall, and F1 score of reaction condition data.
  arXiv  Detail & Related papers  (2024-02-20T13:21:46Z)
• Chemist-X: Large Language Model-empowered Agent for Reaction Condition Recommendation in Chemical Synthesis [55.30328162764292]
  Chemist-X is a comprehensive AI agent that automates the reaction condition optimization (RCO) task in chemical synthesis.<n>The agent uses retrieval-augmented generation (RAG) technology and AI-controlled wet-lab experiment executions.<n>Results of our automatic wet-lab experiments, achieved by fully LLM-supervised end-to-end operation with no human in the lope, prove Chemist-X's ability in self-driving laboratories.
  arXiv  Detail & Related papers  (2023-11-16T01:21:33Z)
• A curated, ontology-based, large-scale knowledge graph of artificial intelligence tasks and benchmarks [4.04540578484476]
  Intelligence Task Ontology and Knowledge Graph (ITO) is a comprehensive resource on artificial intelligence tasks, benchmark results and performance metrics. ITO is a richly structured and manually curated resource on artificial intelligence tasks, benchmark results and performance metrics. The goal of ITO is to enable precise and network-based analyses of the global landscape of AI tasks and capabilities.
  arXiv  Detail & Related papers  (2021-10-04T13:25:53Z)
• Automated Meta-Analysis: A Causal Learning Perspective [30.746257517698133]
  We work toward automating meta-analysis with a focus on controlling for risks of bias. We first extract information from scientific publications written in natural language. From a novel causal learning perspective, we propose to frame automated meta-analysis as a multiple-causal-inference problem.
  arXiv  Detail & Related papers  (2021-04-09T23:07:07Z)
• Towards an Automatic Analysis of CHO-K1 Suspension Growth in Microfluidic Single-cell Cultivation [63.94623495501023]
  We propose a novel Machine Learning architecture, which allows us to infuse a neural deep network with human-powered abstraction on the level of data. Specifically, we train a generative model simultaneously on natural and synthetic data, so that it learns a shared representation, from which a target variable, such as the cell count, can be reliably estimated.
  arXiv  Detail & Related papers  (2020-10-20T08:36:51Z)
• Evaluation Toolkit For Robustness Testing Of Automatic Essay Scoring Systems [64.4896118325552]
  We evaluate the current state-of-the-art AES models using a model adversarial evaluation scheme and associated metrics. We find that AES models are highly overstable. Even heavy modifications(as much as 25%) with content unrelated to the topic of the questions do not decrease the score produced by the models.
  arXiv  Detail & Related papers  (2020-07-14T03:49:43Z)
• Data Mining in Clinical Trial Text: Transformers for Classification and Question Answering Tasks [2.127049691404299]
  This research applies advances in natural language processing to evidence synthesis based on medical texts. The main focus is on information characterized via the Population, Intervention, Comparator, and Outcome (PICO) framework. Recent neural network architectures based on transformers show capacities for transfer learning and increased performance on downstream natural language processing tasks.
  arXiv  Detail & Related papers  (2020-01-30T11:45:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.