Encoding Explanatory Knowledge for Zero-shot Science Question Answering

• URL: http://arxiv.org/abs/2105.05737v1
• Date: Wed, 12 May 2021 15:42:50 GMT
• Title: Encoding Explanatory Knowledge for Zero-shot Science Question Answering
• Authors: Zili Zhou, Marco Valentino, Donal Landers, Andre Freitas
• Abstract summary: N-XKT is able to improve accuracy and generalization on science Question Answering (QA) N-XKT model shows a clear improvement on zero-shot QA. A systematic analysis is conducted to quantitatively analyze the performance of the N-XKT model.
• Score: 0.755972004983746
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper describes N-XKT (Neural encoding based on eXplanatory Knowledge Transfer), a novel method for the automatic transfer of explanatory knowledge through neural encoding mechanisms. We demonstrate that N-XKT is able to improve accuracy and generalization on science Question Answering (QA). Specifically, by leveraging facts from background explanatory knowledge corpora, the N-XKT model shows a clear improvement on zero-shot QA. Furthermore, we show that N-XKT can be fine-tuned on a target QA dataset, enabling faster convergence and more accurate results. A systematic analysis is conducted to quantitatively analyze the performance of the N-XKT model and the impact of different categories of knowledge on the zero-shot generalization task.

Related papers

• Mechanistic Interpretability of LoRA-Adapted Language Models for Nuclear Reactor Safety Applications [0.0]
  This paper presents a novel methodology for interpreting how Large Language Models encode and utilize domain-specific knowledge.<n>We adapted a general-purpose LLM to the nuclear domain using a parameter-efficient fine-tuning technique known as Low-Rank Adaptation.<n>By comparing the neuron activation patterns of the base model to those of the fine-tuned model, we identified a sparse set of neurons whose behavior was significantly altered.
  arXiv  Detail & Related papers  (2025-07-14T05:17:41Z)
• Towards Practical Quantum Neural Network Diagnostics with Neural Tangent Kernels [0.8437187555622164]
  We propose a framework allowing to employ the Quantum Neural Tangent Kernel (QNTK) for Quantum Neural Network (QNN) performance diagnostics. We show how a critical learning rate and a characteristic decay time for the average training error can be estimated from the spectrum of the QNTK evaluated. We then show how a QNTK-based kernel formula can be used to analyze, up to a first-order approximation, the expected inference capabilities of the quantum model under study.
  arXiv  Detail & Related papers  (2025-03-03T19:00:02Z)
• Modeling Quantum Machine Learning for Genomic Data Analysis [12.248184406275405]
  Quantum Machine Learning (QML) continues to evolve, unlocking new opportunities for diverse applications. We investigate and evaluate the applicability of QML models for binary classification of genome sequence data by employing various feature mapping techniques. We present an open-source, independent Qiskit-based implementation to conduct experiments on a benchmark genomic dataset.
  arXiv  Detail & Related papers  (2025-01-14T15:14:26Z)
• Advanced Knowledge Transfer: Refined Feature Distillation for Zero-Shot Quantization in Edge Computing [1.8067835669244101]
  AKT (Advanced Knowledge Transfer) is a novel method to enhance the training ability of low-bit quantized (Q) models. Our method addresses the fundamental gradient exploding problem in low-bit Q models.
  arXiv  Detail & Related papers  (2024-12-26T08:52:27Z)
• KBAlign: Efficient Self Adaptation on Specific Knowledge Bases [73.34893326181046]
  We present KBAlign, a self-supervised framework that enhances RAG systems through efficient model adaptation.<n>Our key insight is to leverage the model's intrinsic capabilities for knowledge alignment through two innovative mechanisms.<n> Experiments demonstrate that KBAlign can achieve 90% of the performance gain obtained through GPT-4-supervised adaptation.
  arXiv  Detail & Related papers  (2024-11-22T08:21:03Z)
• Automated Knowledge Concept Annotation and Question Representation Learning for Knowledge Tracing [59.480951050911436]
  We present KCQRL, a framework for automated knowledge concept annotation and question representation learning. We demonstrate the effectiveness of KCQRL across 15 KT algorithms on two large real-world Math learning datasets.
  arXiv  Detail & Related papers  (2024-10-02T16:37:19Z)
• Analytic Convolutional Layer: A Step to Analytic Neural Network [15.596391258983463]
  Analytic Convolutional Layer (ACL) is a mosaic of analytical convolution kernels (ACKs) and traditional convolution kernels. ACLs offer a means for neural network interpretation, thereby paving the way for the intrinsic interpretability of neural network.
  arXiv  Detail & Related papers  (2024-07-03T07:10:54Z)
• Characterizing out-of-distribution generalization of neural networks: application to the disordered Su-Schrieffer-Heeger model [38.79241114146971]
  We show how interpretability methods can increase trust in predictions of a neural network trained to classify quantum phases. In particular, we show that we can ensure better out-of-distribution generalization in the complex classification problem. This work is an example of how the systematic use of interpretability methods can improve the performance of NNs in scientific problems.
  arXiv  Detail & Related papers  (2024-06-14T13:24:32Z)
• QADYNAMICS: Training Dynamics-Driven Synthetic QA Diagnostic for Zero-Shot Commonsense Question Answering [48.25449258017601]
  State-of-the-art approaches fine-tune language models on QA pairs constructed from CommonSense Knowledge Bases. We propose QADYNAMICS, a training dynamics-driven framework for QA diagnostics and refinement.
  arXiv  Detail & Related papers  (2023-10-17T14:27:34Z)
• Pre-training Tensor-Train Networks Facilitates Machine Learning with Variational Quantum Circuits [70.97518416003358]
  Variational quantum circuits (VQCs) hold promise for quantum machine learning on noisy intermediate-scale quantum (NISQ) devices. While tensor-train networks (TTNs) can enhance VQC representation and generalization, the resulting hybrid model, TTN-VQC, faces optimization challenges due to the Polyak-Lojasiewicz (PL) condition. To mitigate this challenge, we introduce Pre+TTN-VQC, a pre-trained TTN model combined with a VQC.
  arXiv  Detail & Related papers  (2023-05-18T03:08:18Z)
• Normalizing Flow-based Neural Process for Few-Shot Knowledge Graph Completion [69.55700751102376]
  Few-shot knowledge graph completion (FKGC) aims to predict missing facts for unseen relations with few-shot associated facts. Existing FKGC methods are based on metric learning or meta-learning, which often suffer from the out-of-distribution and overfitting problems. In this paper, we propose a normalizing flow-based neural process for few-shot knowledge graph completion (NP-FKGC)
  arXiv  Detail & Related papers  (2023-04-17T11:42:28Z)
• Look beyond labels: Incorporating functional summary information in Bayesian neural networks [11.874130244353253]
  We present a simple approach to incorporate summary information about the predicted probability. The available summary information is incorporated as augmented data and modeled with a Dirichlet process. We show how the method can inform the model about task difficulty or class imbalance.
  arXiv  Detail & Related papers  (2022-07-04T07:06:45Z)
• Great Truths are Always Simple: A Rather Simple Knowledge Encoder for Enhancing the Commonsense Reasoning Capacity of Pre-Trained Models [89.98762327725112]
  Commonsense reasoning in natural language is a desired ability of artificial intelligent systems. For solving complex commonsense reasoning tasks, a typical solution is to enhance pre-trained language models(PTMs) with a knowledge-aware graph neural network(GNN) encoder. Despite the effectiveness, these approaches are built on heavy architectures, and can't clearly explain how external knowledge resources improve the reasoning capacity of PTMs.
  arXiv  Detail & Related papers  (2022-05-04T01:27:36Z)
• Injecting Numerical Reasoning Skills into Knowledge Base Question Answering Models [19.964729281684363]
  This paper proposes a new embedding-based KBQA framework which takes numerical reasoning into account. We present NumericalTransformer on top of NSM, a state-of-the-art embedding-based KBQA model, to create NT-NSM. Experiments on KBQA benchmarks demonstrate that NT-NSM is empowered with numerical reasoning skills and substantially outperforms the baselines in answering ordinal constrained questions.
  arXiv  Detail & Related papers  (2021-12-12T01:30:29Z)
• Adaptive Neuro Fuzzy Networks based on Quantum Subtractive Clustering [5.957580737396458]
  In this paper, an adaptive Neuro fuzzy network with TSK fuzzy type and an improved quantum subtractive clustering has been developed. The experimental results revealed that proposed Anfis based on quantum subtractive clustering yielded good approximation and generalization capabilities.
  arXiv  Detail & Related papers  (2021-01-26T20:59:48Z)
• Neural Networks Enhancement with Logical Knowledge [83.9217787335878]
  We propose an extension of KENN for relational data. The results show that KENN is capable of increasing the performances of the underlying neural network even in the presence relational data.
  arXiv  Detail & Related papers  (2020-09-13T21:12:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.