Classifying Math KCs via Task-Adaptive Pre-Trained BERT

• URL: http://arxiv.org/abs/2105.11343v1
• Date: Mon, 24 May 2021 15:27:33 GMT
• Title: Classifying Math KCs via Task-Adaptive Pre-Trained BERT
• Authors: Jia Tracy Shen, Michiharu Yamashita, Ethan Prihar, Neil Heffernan, Xintao Wu, Sean McGrew, Dongwon Lee
• Abstract summary: This work significantly improves prior research by expanding the input types to include KC descriptions, instructional video titles, and problem descriptions. We also propose a simple evaluation measure by which we can recover 56-73% of mispredicted KC labels.
• Score: 14.53486865876146
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Educational content labeled with proper knowledge components (KCs) are particularly useful to teachers or content organizers. However, manually labeling educational content is labor intensive and error-prone. To address this challenge, prior research proposed machine learning based solutions to auto-label educational content with limited success. In this work, we significantly improve prior research by (1) expanding the input types to include KC descriptions, instructional video titles, and problem descriptions (i.e., three types of prediction task), (2) doubling the granularity of the prediction from 198 to 385 KC labels (i.e., more practical setting but much harder multinomial classification problem), (3) improving the prediction accuracies by 0.5-2.3% using Task-adaptive Pre-trained BERT, outperforming six baselines, and (4) proposing a simple evaluation measure by which we can recover 56-73% of mispredicted KC labels. All codes and data sets in the experiments are available at:https://github.com/tbs17/TAPT-BERT

Related papers

• Advancing Scientific Text Classification: Fine-Tuned Models with Dataset Expansion and Hard-Voting [0.0]
  BERT, SciBERT, BioBERT, and BlueBERT are fine-tuned on the Web of Science (WoS-46985) dataset for scientific text classification. We augment the dataset by executing seven targeted queries in the WoS database, retrieving 1,000 articles per category aligned with WoS-46985's main classes. Fine-tuning on the expanded dataset with dynamic learning rates and early stopping significantly boosts classification accuracy.
  arXiv  Detail & Related papers  (2025-04-26T21:06:49Z)
• Predicting Large Language Model Capabilities on Closed-Book QA Tasks Using Only Information Available Prior to Training [51.60874286674908]
  We focus on predicting performance on Closed-book Question Answering (CBQA) tasks, which are closely tied to pre-training data and knowledge retention. We address three major challenges: 1) mastering the entire pre-training process, especially data construction; 2) evaluating a model's knowledge retention; and 3) predicting task-specific knowledge retention using only information available prior to training. We introduce the SMI metric, an information-theoretic measure that quantifies the relationship between pre-training data, model size, and task-specific knowledge retention.
  arXiv  Detail & Related papers  (2025-02-06T13:23:53Z)
• Enriched BERT Embeddings for Scholarly Publication Classification [0.13654846342364302]
  The NSLP 2024 FoRC Task I addresses this challenge organized as a competition. The goal is to develop a classifier capable of predicting one of 123 predefined classes from the Open Research Knowledge Graph (ORKG) taxonomy of research fields for a given article.
  arXiv  Detail & Related papers  (2024-05-07T09:05:20Z)
• XAL: EXplainable Active Learning Makes Classifiers Better Low-resource Learners [71.8257151788923]
  We propose a novel Explainable Active Learning framework (XAL) for low-resource text classification. XAL encourages classifiers to justify their inferences and delve into unlabeled data for which they cannot provide reasonable explanations. Experiments on six datasets show that XAL achieves consistent improvement over 9 strong baselines.
  arXiv  Detail & Related papers  (2023-10-09T08:07:04Z)
• Rank-Aware Negative Training for Semi-Supervised Text Classification [3.105629960108712]
  Semi-supervised text classification-based paradigms (SSTC) typically employ the spirit of self-training. This paper presents a Rank-aware Negative Training (RNT) framework to address SSTC in learning with noisy label manner.
  arXiv  Detail & Related papers  (2023-06-13T08:41:36Z)
• Task-Specific Embeddings for Ante-Hoc Explainable Text Classification [6.671252951387647]
  We propose an alternative training objective in which we learn task-specific embeddings of text. Our proposed objective learns embeddings such that all texts that share the same target class label should be close together. We present extensive experiments which show that the benefits of ante-hoc explainability and incremental learning come at no cost in overall classification accuracy.
  arXiv  Detail & Related papers  (2022-11-30T19:56:25Z)
• Learning Instructions with Unlabeled Data for Zero-Shot Cross-Task Generalization [68.91386402390403]
  We propose Unlabeled Data Augmented Instruction Tuning (UDIT) to take better advantage of the instructions during instruction learning. We conduct extensive experiments to show UDIT's effectiveness in various scenarios of tasks and datasets.
  arXiv  Detail & Related papers  (2022-10-17T15:25:24Z)
• An Embarrassingly Simple Approach to Semi-Supervised Few-Shot Learning [58.59343434538218]
  We propose a simple but quite effective approach to predict accurate negative pseudo-labels of unlabeled data from an indirect learning perspective. Our approach can be implemented in just few lines of code by only using off-the-shelf operations.
  arXiv  Detail & Related papers  (2022-09-28T02:11:34Z)
• Prompt-Learning for Short Text Classification [30.53216712864025]
  In short text, the extreme short length, feature sparsity and high ambiguity pose huge challenge to classification tasks. In this paper, we propose a simple short text classification approach that makes use of prompt-learning based on knowledgeable expansion.
  arXiv  Detail & Related papers  (2022-02-23T08:07:06Z)
• Using Self-Supervised Pretext Tasks for Active Learning [7.214674613451605]
  We propose a novel active learning approach that utilizes self-supervised pretext tasks and a unique data sampler to select data that are both difficult and representative. The pretext task learner is trained on the unlabeled set, and the unlabeled data are sorted and grouped into batches by their pretext task losses. In each iteration, the main task model is used to sample the most uncertain data in a batch to be annotated.
  arXiv  Detail & Related papers  (2022-01-19T07:58:06Z)
• Knowledge-Aware Meta-learning for Low-Resource Text Classification [87.89624590579903]
  This paper studies a low-resource text classification problem and bridges the gap between meta-training and meta-testing tasks. We propose KGML to introduce additional representation for each sentence learned from the extracted sentence-specific knowledge graph.
  arXiv  Detail & Related papers  (2021-09-10T07:20:43Z)
• A Theoretical Analysis of Learning with Noisily Labeled Data [62.946840431501855]
  We first show that in the first epoch training, the examples with clean labels will be learned first. We then show that after the learning from clean data stage, continuously training model can achieve further improvement in testing error.
  arXiv  Detail & Related papers  (2021-04-08T23:40:02Z)
• Self-training Improves Pre-training for Natural Language Understanding [63.78927366363178]
  We study self-training as another way to leverage unlabeled data through semi-supervised learning. We introduce SentAugment, a data augmentation method which computes task-specific query embeddings from labeled data. Our approach leads to scalable and effective self-training with improvements of up to 2.6% on standard text classification benchmarks.
  arXiv  Detail & Related papers  (2020-10-05T17:52:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.