Extracting Research Instruments from Educational Literature Using LLMs

• URL: http://arxiv.org/abs/2505.21855v1
• Date: Wed, 28 May 2025 01:00:32 GMT
• Title: Extracting Research Instruments from Educational Literature Using LLMs
• Authors: Jiseung Yoo, Curran Mahowald, Meiyu Li, Wei Ai,
• Abstract summary: Large Language Models (LLMs) are transforming information extraction from academic literature.<n>This study presents an LLM-based system designed to extract detailed information about research instruments used in the education field.
• Score: 0.5351587535280344
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large Language Models (LLMs) are transforming information extraction from academic literature, offering new possibilities for knowledge management. This study presents an LLM-based system designed to extract detailed information about research instruments used in the education field, including their names, types, target respondents, measured constructs, and outcomes. Using multi-step prompting and a domain-specific data schema, it generates structured outputs optimized for educational research. Our evaluation shows that this system significantly outperforms other approaches, particularly in identifying instrument names and detailed information. This demonstrates the potential of LLM-powered information extraction in educational contexts, offering a systematic way to organize research instrument information. The ability to aggregate such information at scale enhances accessibility for researchers and education leaders, facilitating informed decision-making in educational research and policy.

Related papers

• How do Large Language Models Understand Relevance? A Mechanistic Interpretability Perspective [64.00022624183781]
  Large language models (LLMs) can assess relevance and support information retrieval (IR) tasks.<n>We investigate how different LLM modules contribute to relevance judgment through the lens of mechanistic interpretability.
  arXiv  Detail & Related papers  (2025-04-10T16:14:55Z)
• Human-artificial intelligence teaming for scientific information extraction from data-driven additive manufacturing research using large language models [3.0061386772253784]
  Data-driven research in Additive Manufacturing (AM) has gained significant success in recent years. This has led to a plethora of scientific literature to emerge. It requires substantial effort and time to extract scientific information from these works. We propose a framework that enables collaboration between AM and AI experts to continuously extract scientific information from data-driven AM literature.
  arXiv  Detail & Related papers  (2024-07-26T15:43:52Z)
• From Text to Insight: Large Language Models for Materials Science Data Extraction [4.08853418443192]
  The vast majority of materials science knowledge exists in unstructured natural language.<n>Structured data is crucial for innovative and systematic materials design.<n>The advent of large language models (LLMs) represents a significant shift.
  arXiv  Detail & Related papers  (2024-07-23T22:23:47Z)
• Retrieval-Enhanced Machine Learning: Synthesis and Opportunities [60.34182805429511]
  Retrieval-enhancement can be extended to a broader spectrum of machine learning (ML) This work introduces a formal framework of this paradigm, Retrieval-Enhanced Machine Learning (REML), by synthesizing the literature in various domains in ML with consistent notations which is missing from the current literature. The goal of this work is to equip researchers across various disciplines with a comprehensive, formally structured framework of retrieval-enhanced models, thereby fostering interdisciplinary future research.
  arXiv  Detail & Related papers  (2024-07-17T20:01:21Z)
• Tool Learning with Large Language Models: A Survey [60.733557487886635]
  Tool learning with large language models (LLMs) has emerged as a promising paradigm for augmenting the capabilities of LLMs to tackle highly complex problems. Despite growing attention and rapid advancements in this field, the existing literature remains fragmented and lacks systematic organization.
  arXiv  Detail & Related papers  (2024-05-28T08:01:26Z)
• Automating Research Synthesis with Domain-Specific Large Language Model Fine-Tuning [0.9110413356918055]
  This research pioneers the use of fine-tuned Large Language Models (LLMs) to automate Systematic Literature Reviews ( SLRs) Our study employed the latest fine-tuning methodologies together with open-sourced LLMs, and demonstrated a practical and efficient approach to automating the final execution stages of an SLR process. The results maintained high fidelity in factual accuracy in LLM responses, and were validated through the replication of an existing PRISMA-conforming SLR.
  arXiv  Detail & Related papers  (2024-04-08T00:08:29Z)
• Large Language Models for Education: A Survey and Outlook [69.02214694865229]
  We systematically review the technological advancements in each perspective, organize related datasets and benchmarks, and identify the risks and challenges associated with deploying LLMs in education. Our survey aims to provide a comprehensive technological picture for educators, researchers, and policymakers to harness the power of LLMs to revolutionize educational practices and foster a more effective personalized learning environment.
  arXiv  Detail & Related papers  (2024-03-26T21:04:29Z)
• Large Language Models for Conducting Advanced Text Analytics Information Systems Research [4.913568041651961]
  Large Language Models (LLMs) have emerged as tools that are capable of processing and extracting insights from massive unstructured textual datasets. We propose a Text Analytics for Information Systems Research (TAISR) framework to assist the Information Systems community in understanding how to operationalize LLMs.
  arXiv  Detail & Related papers  (2023-12-27T19:49:00Z)
• Exploring the Cognitive Knowledge Structure of Large Language Models: An Educational Diagnostic Assessment Approach [50.125704610228254]
  Large Language Models (LLMs) have not only exhibited exceptional performance across various tasks, but also demonstrated sparks of intelligence. Recent studies have focused on assessing their capabilities on human exams and revealed their impressive competence in different domains. We conduct an evaluation using MoocRadar, a meticulously annotated human test dataset based on Bloom taxonomy.
  arXiv  Detail & Related papers  (2023-10-12T09:55:45Z)
• A Comprehensive Survey on Deep Learning Techniques in Educational Data Mining [26.349367339930083]
  Educational Data Mining (EDM) has emerged as a vital field of research, which harnesses the power of computational techniques to analyze educational data. Deep Learning techniques have shown significant advantages in addressing the challenges associated with analyzing and modeling this data. This survey aims to systematically review the state-of-the-art in EDM with Deep Learning.
  arXiv  Detail & Related papers  (2023-09-09T11:20:40Z)
• Instruction Tuning for Large Language Models: A Survey [52.86322823501338]
  We make a systematic review of the literature, including the general methodology of supervised fine-tuning (SFT)<n>We also review the potential pitfalls of SFT along with criticism against it, along with efforts pointing out current deficiencies of existing strategies.
  arXiv  Detail & Related papers  (2023-08-21T15:35:16Z)
• Retrieval-Enhanced Machine Learning [110.5237983180089]
  We describe a generic retrieval-enhanced machine learning framework, which includes a number of existing models as special cases. REML challenges information retrieval conventions, presenting opportunities for novel advances in core areas, including optimization. REML research agenda lays a foundation for a new style of information access research and paves a path towards advancing machine learning and artificial intelligence.
  arXiv  Detail & Related papers  (2022-05-02T21:42:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.