Make Literature-Based Discovery Great Again through Reproducible Pipelines

• URL: http://arxiv.org/abs/2502.16450v1
• Date: Sun, 23 Feb 2025 05:38:04 GMT
• Title: Make Literature-Based Discovery Great Again through Reproducible Pipelines
• Abstract summary: The main novelty of this study is a collection of Jupyter Notebooks that illustrate the steps of the bisociative LBD process.<n>The reader can benefit from hands-on experience with LBD through open access to benchmark datasets, code reuse, and a ready-to-run Docker recipe.
• Score: 1.5602870473232306
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: By connecting disparate sources of scientific literature, literature\-/based discovery (LBD) methods help to uncover new knowledge and generate new research hypotheses that cannot be found from domain-specific documents alone. Our work focuses on bisociative LBD methods that combine bisociative reasoning with LBD techniques. The paper presents LBD through the lens of reproducible science to ensure the reproducibility of LBD experiments, overcome the inconsistent use of benchmark datasets and methods, trigger collaboration, and advance the LBD field toward more robust and impactful scientific discoveries. The main novelty of this study is a collection of Jupyter Notebooks that illustrate the steps of the bisociative LBD process, including data acquisition, text preprocessing, hypothesis formulation, and evaluation. The contributed notebooks implement a selection of traditional LBD approaches, as well as our own ensemble-based, outlier-based, and link prediction-based approaches. The reader can benefit from hands-on experience with LBD through open access to benchmark datasets, code reuse, and a ready-to-run Docker recipe that ensures reproducibility of the selected LBD methods.

Related papers

• ABCD-LINK: Annotation Bootstrapping for Cross-Document Fine-Grained Links [57.514511353084565]
  We introduce a new domain-agnostic framework for selecting a best-performing approach and annotating cross-document links.<n>We apply our framework in two distinct domains -- peer review and news.<n>The resulting novel datasets lay foundation for numerous cross-document tasks like media framing and peer review.
  arXiv  Detail & Related papers  (2025-09-01T11:32:24Z)
• A Comprehensive Review of Agricultural Parcel and Boundary Delineation from Remote Sensing Images: Recent Progress and Future Perspectives [7.516276650839454]
  This review covers Agricultural Parcel and Boundary Delineation (APBD) methods for detecting and delineating agricultural parcels.<n>We categorize the methods into three classes: traditional image processing methods (including pixel-based, edge-based and region-based); traditional machine learning methods (such as random forest, decision tree); and deep learning-based methods.<n>This review proposes some APBD-related applications and a few exciting prospects and potential hot topics in future APBD research.
  arXiv  Detail & Related papers  (2025-08-20T09:24:42Z)
• Harnessing Large Language Models for Scientific Novelty Detection [49.10608128661251]
  We propose to harness large language models (LLMs) for scientific novelty detection (ND)<n>To capture idea conception, we propose to train a lightweight retriever by distilling the idea-level knowledge from LLMs.<n> Experiments show our method consistently outperforms others on the proposed benchmark datasets for idea retrieval and ND tasks.
  arXiv  Detail & Related papers  (2025-05-30T14:08:13Z)
• Learning More Effective Representations for Dense Retrieval through Deliberate Thinking Before Search [65.53881294642451]
  Deliberate Thinking based Dense Retriever (DEBATER)<n>DEBATER enhances recent dense retrievers by enabling them to learn more effective document representations through a step-by-step thinking process.<n> Experimental results show that DEBATER significantly outperforms existing methods across several retrieval benchmarks.
  arXiv  Detail & Related papers  (2025-02-18T15:56:34Z)
• SciPIP: An LLM-based Scientific Paper Idea Proposer [30.670219064905677]
  We introduce SciPIP, an innovative framework designed to enhance the proposal of scientific ideas through improvements in both literature retrieval and idea generation.<n>Our experiments, conducted across various domains such as natural language processing and computer vision, demonstrate SciPIP's capability to generate a multitude of innovative and useful ideas.
  arXiv  Detail & Related papers  (2024-10-30T16:18:22Z)
• A Bayesian Approach to Harnessing the Power of LLMs in Authorship Attribution [57.309390098903]
  Authorship attribution aims to identify the origin or author of a document. Large Language Models (LLMs) with their deep reasoning capabilities and ability to maintain long-range textual associations offer a promising alternative. Our results on the IMDb and blog datasets show an impressive 85% accuracy in one-shot authorship classification across ten authors.
  arXiv  Detail & Related papers  (2024-10-29T04:14:23Z)
• Evaluating Deduplication Techniques for Economic Research Paper Titles with a Focus on Semantic Similarity using NLP and LLMs [0.9576327614980397]
  This study investigates efficient deduplication techniques for a large NLP dataset of economic research paper titles.<n>Our findings suggest a potentially low prevalence of duplicates based on the observed semantic similarity across different methods.
  arXiv  Detail & Related papers  (2024-10-02T00:43:10Z)
• ResearchAgent: Iterative Research Idea Generation over Scientific Literature with Large Language Models [56.08917291606421]
  ResearchAgent is an AI-based system for ideation and operationalization of novel work. ResearchAgent automatically defines novel problems, proposes methods and designs experiments, while iteratively refining them. We experimentally validate our ResearchAgent on scientific publications across multiple disciplines.
  arXiv  Detail & Related papers  (2024-04-11T13:36:29Z)
• A Survey on Data Selection for Language Models [148.300726396877]
  Data selection methods aim to determine which data points to include in a training dataset. Deep learning is mostly driven by empirical evidence and experimentation on large-scale data is expensive. Few organizations have the resources for extensive data selection research.
  arXiv  Detail & Related papers  (2024-02-26T18:54:35Z)
• A Reliable Knowledge Processing Framework for Combustion Science using Foundation Models [0.0]
  The study introduces an approach to process diverse combustion research data, spanning experimental studies, simulations, and literature. The developed approach minimizes computational and economic expenses while optimizing data privacy and accuracy. The framework consistently delivers accurate domain-specific responses with minimal human oversight.
  arXiv  Detail & Related papers  (2023-12-31T17:15:25Z)
• Literature Based Discovery (LBD): Towards Hypothesis Generation and Knowledge Discovery in Biomedical Text Mining [0.0]
  Literature Based Discovery (LBD) is the process of automatically discovering novel associations between medical terms. The process focuses on creating concept profiles for medical terms such as a disease or symptom and connecting it with a drug and treatment. This review also discusses the deep learning applications in LBD by reviewing the role of transformer models and neural networks based LBD models.
  arXiv  Detail & Related papers  (2023-10-04T02:13:11Z)
• GAIA Search: Hugging Face and Pyserini Interoperability for NLP Training Data Exploration [97.68234051078997]
  We discuss how Pyserini can be integrated with the Hugging Face ecosystem of open-source AI libraries and artifacts. We include a Jupyter Notebook-based walk through the core interoperability features, available on GitHub. We present GAIA Search - a search engine built following previously laid out principles, giving access to four popular large-scale text collections.
  arXiv  Detail & Related papers  (2023-06-02T12:09:59Z)
• SciMON: Scientific Inspiration Machines Optimized for Novelty [68.46036589035539]
  We explore and enhance the ability of neural language models to generate novel scientific directions grounded in literature. We take a dramatic departure with a novel setting in which models use as input background contexts. We present SciMON, a modeling framework that uses retrieval of "inspirations" from past scientific papers.
  arXiv  Detail & Related papers  (2023-05-23T17:12:08Z)
• Retrieval Augmentation for Commonsense Reasoning: A Unified Approach [64.63071051375289]
  We propose a unified framework of retrieval-augmented commonsense reasoning (called RACo) Our proposed RACo can significantly outperform other knowledge-enhanced method counterparts.
  arXiv  Detail & Related papers  (2022-10-23T23:49:08Z)
• pyBKT: An Accessible Python Library of Bayesian Knowledge Tracing Models [0.0]
  We introduce pyBKT, a library of model extensions for knowledge tracing. The library provides data generation, fitting, prediction, and cross-validation routines. pyBKT is open source and open license for the purpose of making knowledge tracing more accessible to communities of research and practice.
  arXiv  Detail & Related papers  (2021-05-02T03:08:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.