Efficient Learning Content Retrieval with Knowledge Injection

• URL: http://arxiv.org/abs/2412.00125v1
• Date: Thu, 28 Nov 2024 12:06:14 GMT
• Title: Efficient Learning Content Retrieval with Knowledge Injection
• Authors: Batuhan Sariturk, Rabia Bayraktar, Merve Elmas Erdem,
• Abstract summary: We propose a domain-specific application that requires limited resources, utilizing versions of the Phi language model to help learners with educational content. Data required for fine-tuning was obtained from the Huawei Talent Platform. RAG system was used to support the model, which was fine-tuned by 500 Q&A pairs.
• Score: 0.0
• License:
• Abstract: With the rise of online education platforms, there is a growing abundance of educational content across various domain. It can be difficult to navigate the numerous available resources to find the most suitable training, especially in domains that include many interconnected areas, such as ICT. In this study, we propose a domain-specific chatbot application that requires limited resources, utilizing versions of the Phi language model to help learners with educational content. In the proposed method, Phi-2 and Phi-3 models were fine-tuned using QLoRA. The data required for fine-tuning was obtained from the Huawei Talent Platform, where courses are available at different levels of expertise in the field of computer science. RAG system was used to support the model, which was fine-tuned by 500 Q&A pairs. Additionally, a total of 420 Q&A pairs of content were extracted from different formats such as JSON, PPT, and DOC to create a vector database to be used in the RAG system. By using the fine-tuned model and RAG approach together, chatbots with different competencies were obtained. The questions and answers asked to the generated chatbots were saved separately and evaluated using ROUGE, BERTScore, METEOR, and BLEU metrics. The precision value of the Phi-2 model supported by RAG was 0.84 and the F1 score was 0.82. In addition to a total of 13 different evaluation metrics in 4 different categories, the answers of each model were compared with the created content and the most appropriate method was selected for real-life applications.

Related papers

• Teaching Smaller Language Models To Generalise To Unseen Compositional Questions (Full Thesis) [0.0]
  We train our models to answer diverse questions by instilling an ability to reason over a retrieved context. We acquire context from two knowledge sources; a Wikipedia corpus queried using a multi-hop dense retrieval system with novel extensions, and from rationales generated from a larger Language Model optimised to run in a lower resource environment.
  arXiv  Detail & Related papers  (2024-11-25T23:25:34Z)
• EduNLP: Towards a Unified and Modularized Library for Educational Resources [78.8523961816045]
  We present a unified, modularized, and extensive library, EduNLP, focusing on educational resource understanding. In the library, we decouple the whole workflow to four key modules with consistent interfaces including data configuration, processing, model implementation, and model evaluation. For the current version, we primarily provide 10 typical models from four categories, and 5 common downstream-evaluation tasks in the education domain on 8 subjects for users' usage.
  arXiv  Detail & Related papers  (2024-06-03T12:45:40Z)
• LLM2KB: Constructing Knowledge Bases using instruction tuned context aware Large Language Models [0.8702432681310401]
  Our paper proposes LLM2KB, a system for constructing knowledge bases using large language models. Our best performing model achieved an average F1 score of 0.6185 across 21 relations in the LM-KBC challenge held at the ISWC 2023 conference.
  arXiv  Detail & Related papers  (2023-08-25T07:04:16Z)
• QUADRo: Dataset and Models for QUestion-Answer Database Retrieval [97.84448420852854]
  Given a database (DB) of question/answer (q/a) pairs, it is possible to answer a target question by scanning the DB for similar questions. We build a large scale DB of 6.3M q/a pairs, using public questions, and design a new system based on neural IR and a q/a pair reranker. We show that our DB-based approach is competitive with Web-based methods, i.e., a QA system built on top the BING search engine.
  arXiv  Detail & Related papers  (2023-03-30T00:42:07Z)
• Utilizing Background Knowledge for Robust Reasoning over Traffic Situations [63.45021731775964]
  We focus on a complementary research aspect of Intelligent Transportation: traffic understanding. We scope our study to text-based methods and datasets given the abundant commonsense knowledge. We adopt three knowledge-driven approaches for zero-shot QA over traffic situations.
  arXiv  Detail & Related papers  (2022-12-04T09:17:24Z)
• Question-type Identification for Academic Questions in Online Learning Platform [1.3764085113103222]
  This paper explores question-type identification as a step in content understanding for an online learning platform. We have defined twelve question-type classes, including Multiple-Choice Question (MCQ), essay, and others. We trained a BERT-based ensemble model on this dataset and evaluated this model on a separate human-labeled test set.
  arXiv  Detail & Related papers  (2022-11-24T17:28:29Z)
• Lila: A Unified Benchmark for Mathematical Reasoning [59.97570380432861]
  LILA is a unified mathematical reasoning benchmark consisting of 23 diverse tasks along four dimensions. We construct our benchmark by extending 20 datasets benchmark by collecting task instructions and solutions in the form of Python programs. We introduce BHASKARA, a general-purpose mathematical reasoning model trained on LILA.
  arXiv  Detail & Related papers  (2022-10-31T17:41:26Z)
• A Coarse to Fine Question Answering System based on Reinforcement Learning [48.80863342506432]
  The system is designed using an actor-critic based deep reinforcement learning model to achieve multi-step question answering. We test our model on four QA datasets, WIKEREADING, WIKIREADING LONG, CNN and SQuAD, and demonstrate 1.3$%$-1.7$%$ accuracy improvements with 1.5x-3.4x training speed-ups.
  arXiv  Detail & Related papers  (2021-06-01T06:41:48Z)
• Educational Question Mining At Scale: Prediction, Analysis and Personalization [35.42197158180065]
  We propose a framework for mining insights from educational questions at scale. We utilize the state-of-the-art Bayesian deep learning method, in particular partial variational auto-encoders (p-VAE) We apply our proposed framework to a real-world dataset with tens of thousands of questions and tens of millions of answers from an online education platform.
  arXiv  Detail & Related papers  (2020-03-12T19:07:49Z)
• Hybrid Generative-Retrieval Transformers for Dialogue Domain Adaptation [77.62366712130196]
  We present the winning entry at the fast domain adaptation task of DSTC8, a hybrid generative-retrieval model based on GPT-2 fine-tuned to the multi-domain MetaLWOz dataset. Our model uses retrieval logic as a fallback, being SoTA on MetaLWOz in human evaluation (>4% improvement over the 2nd place system) and attaining competitive generalization performance in adaptation to the unseen MultiWOZ dataset.
  arXiv  Detail & Related papers  (2020-03-03T18:07:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.