SyROCCo: Enhancing Systematic Reviews using Machine Learning

• URL: http://arxiv.org/abs/2406.16527v1
• Date: Mon, 24 Jun 2024 11:04:43 GMT
• Title: SyROCCo: Enhancing Systematic Reviews using Machine Learning
• Authors: Zheng Fang, Miguel Arana-Catania, Felix-Anselm van Lier, Juliana Outes Velarde, Harry Bregazzi, Mara Airoldi, Eleanor Carter, Rob Procter,
• Abstract summary: This paper explores the use of machine learning techniques to help navigate the systematic review process. The application of ML techniques to subsequent stages of a review, such as data extraction and evidence mapping, is in its infancy.
• Score: 6.805429133535976
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The sheer number of research outputs published every year makes systematic reviewing increasingly time- and resource-intensive. This paper explores the use of machine learning techniques to help navigate the systematic review process. ML has previously been used to reliably 'screen' articles for review - that is, identify relevant articles based on reviewers' inclusion criteria. The application of ML techniques to subsequent stages of a review, however, such as data extraction and evidence mapping, is in its infancy. We therefore set out to develop a series of tools that would assist in the profiling and analysis of 1,952 publications on the theme of 'outcomes-based contracting'. Tools were developed for the following tasks: assign publications into 'policy area' categories; identify and extract key information for evidence mapping, such as organisations, laws, and geographical information; connect the evidence base to an existing dataset on the same topic; and identify subgroups of articles that may share thematic content. An interactive tool using these techniques and a public dataset with their outputs have been released. Our results demonstrate the utility of ML techniques to enhance evidence accessibility and analysis within the systematic review processes. These efforts show promise in potentially yielding substantial efficiencies for future systematic reviewing and for broadening their analytical scope. Our work suggests that there may be implications for the ease with which policymakers and practitioners can access evidence. While ML techniques seem poised to play a significant role in bridging the gap between research and policy by offering innovative ways of gathering, accessing, and analysing data from systematic reviews, we also highlight their current limitations and the need to exercise caution in their application, particularly given the potential for errors and biases.

Related papers

• Machine Learning for Missing Value Imputation [0.0]
  The main objective of this article is to conduct a comprehensive and rigorous review, as well as analysis, of the state-of-the-art machine learning applications in Missing Value Imputation. More than 100 articles published between 2014 and 2023 are critically reviewed, considering the methods and findings. The latest literature is examined to scrutinize the trends in MVI methods and their evaluation.
  arXiv  Detail & Related papers  (2024-10-10T18:56:49Z)
• Ontology Embedding: A Survey of Methods, Applications and Resources [54.3453925775069]
  Ontologies are widely used for representing domain knowledge and meta data. One straightforward solution is to integrate statistical analysis and machine learning. Numerous papers have been published on embedding, but a lack of systematic reviews hinders researchers from gaining a comprehensive understanding of this field.
  arXiv  Detail & Related papers  (2024-06-16T14:49:19Z)
• A Literature Review of Literature Reviews in Pattern Analysis and Machine Intelligence [58.6354685593418]
  This paper proposes several article-level, field-normalized, and large language model-empowered bibliometric indicators to evaluate reviews. The newly emerging AI-generated literature reviews are also appraised. This work offers insights into the current challenges of literature reviews and envisions future directions for their development.
  arXiv  Detail & Related papers  (2024-02-20T11:28:50Z)
• Bias and Fairness in Large Language Models: A Survey [73.87651986156006]
  We present a comprehensive survey of bias evaluation and mitigation techniques for large language models (LLMs) We first consolidate, formalize, and expand notions of social bias and fairness in natural language processing. We then unify the literature by proposing three intuitive, two for bias evaluation, and one for mitigation.
  arXiv  Detail & Related papers  (2023-09-02T00:32:55Z)
• Instruction Tuning for Large Language Models: A Survey [52.86322823501338]
  We make a systematic review of the literature, including the general methodology of IT, the construction of IT datasets, the training of IT models, and applications to different modalities, domains and applications. We also review the potential pitfalls of IT along with criticism against it, along with efforts pointing out current deficiencies of existing strategies and suggest some avenues for fruitful research.
  arXiv  Detail & Related papers  (2023-08-21T15:35:16Z)
• Artificial intelligence technologies to support research assessment: A review [10.203602318836444]
  This literature review identifies indicators that associate with higher impact or higher quality research from article text. It includes studies that used machine learning techniques to predict citation counts or quality scores for journal articles or conference papers.
  arXiv  Detail & Related papers  (2022-12-11T06:58:39Z)
• Investigating Fairness Disparities in Peer Review: A Language Model Enhanced Approach [77.61131357420201]
  We conduct a thorough and rigorous study on fairness disparities in peer review with the help of large language models (LMs) We collect, assemble, and maintain a comprehensive relational database for the International Conference on Learning Representations (ICLR) conference from 2017 to date. We postulate and study fairness disparities on multiple protective attributes of interest, including author gender, geography, author, and institutional prestige.
  arXiv  Detail & Related papers  (2022-11-07T16:19:42Z)
• Machine Learning State-of-the-Art with Uncertainties [3.4123736336071864]
  We conduct an exemplary image classification study in order to demonstrate how confidence intervals around accuracy measurements can greatly enhance the communication of research results. We make suggestions for improving the authoring and reviewing process of machine learning articles.
  arXiv  Detail & Related papers  (2022-04-11T15:06:26Z)
• A Survey on Machine Learning Techniques for Source Code Analysis [14.129976741300029]
  We aim to summarize the current knowledge in the area of applied machine learning for source code analysis. To do so, we carried out an extensive literature search and identified 364 primary studies published between 2002 and 2021.
  arXiv  Detail & Related papers  (2021-10-18T20:13:38Z)
• Deep Learning Schema-based Event Extraction: Literature Review and Current Trends [60.29289298349322]
  Event extraction technology based on deep learning has become a research hotspot. This paper fills the gap by reviewing the state-of-the-art approaches, focusing on deep learning-based models.
  arXiv  Detail & Related papers  (2021-07-05T16:32:45Z)
• Open Source Software for Efficient and Transparent Reviews [0.11179881480027788]
  ASReview is an open source machine learning-aided pipeline applying active learning. We demonstrate by means of simulation studies that ASReview can yield far more efficient reviewing than manual reviewing.
  arXiv  Detail & Related papers  (2020-06-22T11:57:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.