Automated Generation of Research Workflows from Academic Papers: A Full-text Mining Framework

• URL: http://arxiv.org/abs/2509.12955v2
• Date: Tue, 23 Sep 2025 14:57:23 GMT
• Title: Automated Generation of Research Workflows from Academic Papers: A Full-text Mining Framework
• Authors: Heng Zhang, Chengzhi Zhang,
• Abstract summary: We propose an end-to-end framework that generates comprehensive, structured research papers by mining full-text academic papers.<n>We use Flan-T5 with prompt learning to generate workflow phrases from paragraphs, yielding ROUGE-1, ROUGE-2, and ROUGEL scores of 0.4543, 0.2877, and 0.4427, respectively.<n>This approach reveals key methodological shifts over the past two decades, including the increasing emphasis on data analysis and the transition from feature engineering to ablation studies.
• Score: 7.681506465886571
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The automated generation of research workflows is essential for improving the reproducibility of research and accelerating the paradigm of "AI for Science". However, existing methods typically extract merely fragmented procedural components and thus fail to capture complete research workflows. To address this gap, we propose an end-to-end framework that generates comprehensive, structured research workflows by mining full-text academic papers. As a case study in the Natural Language Processing (NLP) domain, our paragraph-centric approach first employs Positive-Unlabeled (PU) Learning with SciBERT to identify workflow-descriptive paragraphs, achieving an F1-score of 0.9772. Subsequently, we utilize Flan-T5 with prompt learning to generate workflow phrases from these paragraphs, yielding ROUGE-1, ROUGE-2, and ROUGE-L scores of 0.4543, 0.2877, and 0.4427, respectively. These phrases are then systematically categorized into data preparation, data processing, and data analysis stages using ChatGPT with few-shot learning, achieving a classification precision of 0.958. By mapping categorized phrases to their document locations in the documents, we finally generate readable visual flowcharts of the entire research workflows. This approach facilitates the analysis of workflows derived from an NLP corpus and reveals key methodological shifts over the past two decades, including the increasing emphasis on data analysis and the transition from feature engineering to ablation studies. Our work offers a validated technical framework for automated workflow generation, along with a novel, process-oriented perspective for the empirical investigation of evolving scientific paradigms. Source code and data are available at: https://github.com/ZH-heng/research_workflow.

Related papers

• AutoMalDesc: Large-Scale Script Analysis for Cyber Threat Research [81.04845910798387]
  Generating natural language explanations for threat detections remains an open problem in cybersecurity research.<n>We present AutoMalDesc, an automated static analysis summarization framework that operates independently at scale.<n>We publish our complete dataset of more than 100K script samples, including annotated seed (0.9K) datasets, along with our methodology and evaluation framework.
  arXiv  Detail & Related papers  (2025-11-17T13:05:25Z)
• Meow: End-to-End Outline Writing for Automatic Academic Survey [24.749855249116802]
  We propose Meow, a framework that produces organized and faithful outlines efficiently.<n>We first formulate outline writing as an end-to-end task that generates hierarchical structured outlines from paper metadata.<n>We then curate a high-quality dataset of surveys from arXiv, bioRxiv, and medRxiv, and establish systematic evaluation metrics for outline quality assessment.
  arXiv  Detail & Related papers  (2025-09-19T07:20:53Z)
• ABCD-LINK: Annotation Bootstrapping for Cross-Document Fine-Grained Links [57.514511353084565]
  We introduce a new domain-agnostic framework for selecting a best-performing approach and annotating cross-document links.<n>We apply our framework in two distinct domains -- peer review and news.<n>The resulting novel datasets lay foundation for numerous cross-document tasks like media framing and peer review.
  arXiv  Detail & Related papers  (2025-09-01T11:32:24Z)
• Benchmarking Computer Science Survey Generation [18.844790013427282]
  SurGE (Survey Generation Evaluation) is a new benchmark for evaluating scientific survey generation in the computer science domain.<n>SurGE consists of (1) a collection of test instances, each including a topic description, an expert-written survey, and its full set of cited references, and (2) a large-scale academic corpus of over one million papers that serves as the retrieval pool.<n>In addition, we propose an automated evaluation framework that measures generated surveys across four dimensions: information coverage, referencing accuracy, structural organization, and content quality.
  arXiv  Detail & Related papers  (2025-08-21T15:45:10Z)
• MOLE: Metadata Extraction and Validation in Scientific Papers Using LLMs [54.5729817345543]
  MOLE is a framework that automatically extracts metadata attributes from scientific papers covering datasets of languages other than Arabic.<n>Our methodology processes entire documents across multiple input formats and incorporates robust validation mechanisms for consistent output.
  arXiv  Detail & Related papers  (2025-05-26T10:31:26Z)
• Chatting with Papers: A Hybrid Approach Using LLMs and Knowledge Graphs [3.68389405018277]
  This demo paper reports on a new workflow textitGhostWriter that combines the use of Large Language Models and Knowledge Graphs to support navigation through collections.<n>Based on the tool-suite textitEverythingData at the backend, textitGhostWriter provides an interface that enables querying and chatting'' with a collection.
  arXiv  Detail & Related papers  (2025-05-16T18:51:51Z)
• Turning Conversations into Workflows: A Framework to Extract and Evaluate Dialog Workflows for Service AI Agents [65.36060818857109]
  We present a novel framework for extracting and evaluating dialog from historical interactions.<n>Our extraction process consists of two key stages: (1) a retrieval step to select relevant conversations based on key procedural elements, and (2) a structured workflow generation process using a question-answer-based chain-of-thought (QA-CoT) prompting.
  arXiv  Detail & Related papers  (2025-02-24T16:55:15Z)
• Learning Task Representations from In-Context Learning [73.72066284711462]
  Large language models (LLMs) have demonstrated remarkable proficiency in in-context learning.<n>We introduce an automated formulation for encoding task information in ICL prompts as a function of attention heads.<n>We show that our method's effectiveness stems from aligning the distribution of the last hidden state with that of an optimally performing in-context-learned model.
  arXiv  Detail & Related papers  (2025-02-08T00:16:44Z)
• Iterative NLP Query Refinement for Enhancing Domain-Specific Information Retrieval: A Case Study in Career Services [0.13980986259786224]
  Retrieving semantically relevant documents in niche domains poses significant challenges for TF-IDF-based systems.<n>This paper introduces an iterative and semi-automated query refinement methodology tailored to Humber College's career services webpages.
  arXiv  Detail & Related papers  (2024-12-22T15:57:35Z)
• MASSW: A New Dataset and Benchmark Tasks for AI-Assisted Scientific Workflows [58.56005277371235]
  We introduce MASSW, a comprehensive text dataset on Multi-Aspect Summarization of ScientificAspects. MASSW includes more than 152,000 peer-reviewed publications from 17 leading computer science conferences spanning the past 50 years. We demonstrate the utility of MASSW through multiple novel machine-learning tasks that can be benchmarked using this new dataset.
  arXiv  Detail & Related papers  (2024-06-10T15:19:09Z)
• Learning Context-Aware Service Representation for Service Recommendation in Workflow Composition [6.17189383632496]
  This paper proposes a novel NLP-inspired approach to recommending services throughout a workflow development process. A workflow composition process is formalized as a step-wise, context-aware service generation procedure. Service embeddings are then learned by applying deep learning model from the NLP field.
  arXiv  Detail & Related papers  (2022-05-24T04:18:01Z)
• Combining Feature and Instance Attribution to Detect Artifacts [62.63504976810927]
  We propose methods to facilitate identification of training data artifacts. We show that this proposed training-feature attribution approach can be used to uncover artifacts in training data. We execute a small user study to evaluate whether these methods are useful to NLP researchers in practice.
  arXiv  Detail & Related papers  (2021-07-01T09:26:13Z)
• Learning from similarity and information extraction from structured documents [0.0]
  The aim is to improve micro F1 of per-word classification on a huge real-world document dataset. Results confirm that all proposed architecture parts are all required to beat the previous results. The best model improves the previous state-of-the-art results by an 8.25 gain in F1 score.
  arXiv  Detail & Related papers  (2020-10-17T21:34:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.