End-to-End Ontology Learning with Large Language Models

• URL: http://arxiv.org/abs/2410.23584v1
• Date: Thu, 31 Oct 2024 02:52:39 GMT
• Title: End-to-End Ontology Learning with Large Language Models
• Authors: Andy Lo, Albert Q. Jiang, Wenda Li, Mateja Jamnik,
• Abstract summary: Large language models (LLMs) have been applied to solve various subtasks of ontology learning. We address this gap by OLLM, a general and scalable method for building the taxonomic backbone of an ontology from scratch. In contrast to standard metrics, our metrics use deep learning techniques to define more robust structural distance measures between graphs. Our model can be effectively adapted to new domains, like arXiv, needing only a small number of training examples.
• Score: 11.755755139228219
• License:
• Abstract: Ontologies are useful for automatic machine processing of domain knowledge as they represent it in a structured format. Yet, constructing ontologies requires substantial manual effort. To automate part of this process, large language models (LLMs) have been applied to solve various subtasks of ontology learning. However, this partial ontology learning does not capture the interactions between subtasks. We address this gap by introducing OLLM, a general and scalable method for building the taxonomic backbone of an ontology from scratch. Rather than focusing on subtasks, like individual relations between entities, we model entire subcomponents of the target ontology by finetuning an LLM with a custom regulariser that reduces overfitting on high-frequency concepts. We introduce a novel suite of metrics for evaluating the quality of the generated ontology by measuring its semantic and structural similarity to the ground truth. In contrast to standard metrics, our metrics use deep learning techniques to define more robust distance measures between graphs. Both our quantitative and qualitative results on Wikipedia show that OLLM outperforms subtask composition methods, producing more semantically accurate ontologies while maintaining structural integrity. We further demonstrate that our model can be effectively adapted to new domains, like arXiv, needing only a small number of training examples. Our source code and datasets are available at https://github.com/andylolu2/ollm.

Related papers

• A Plug-and-Play Method for Rare Human-Object Interactions Detection by Bridging Domain Gap [50.079224604394]
  We present a novel model-agnostic framework called textbfContext-textbfEnhanced textbfFeature textbfAment (CEFA) CEFA consists of a feature alignment module and a context enhancement module. Our method can serve as a plug-and-play module to improve the detection performance of HOI models on rare categories.
  arXiv  Detail & Related papers  (2024-07-31T08:42:48Z)
• Large language models as oracles for instantiating ontologies with domain-specific knowledge [0.0]
  Endowing intelligent systems with semantic data commonly requires designing and instantiating with domain-specific knowledge. The resulting experience process is therefore time-consuming, error-prone, and often biased by the personal background of ontology designer. We propose a novel domain-independent approach to automatically instantiate with domain-specific knowledge.
  arXiv  Detail & Related papers  (2024-04-05T14:04:07Z)
• Adapting Large Language Models for Content Moderation: Pitfalls in Data Engineering and Supervised Fine-tuning [79.53130089003986]
  Large Language Models (LLMs) have become a feasible solution for handling tasks in various domains. In this paper, we introduce how to fine-tune a LLM model that can be privately deployed for content moderation.
  arXiv  Detail & Related papers  (2023-10-05T09:09:44Z)
• A Meta-Learning Approach to Population-Based Modelling of Structures [0.0]
  A major problem of machine-learning approaches in structural dynamics is the frequent lack of structural data. Inspired by the recently-emerging field of population-based structural health monitoring, this work attempts to create models that are able to transfer knowledge within populations of structures. The models trained using meta-learning approaches, are able to outperform conventional machine learning methods regarding inference about structures of the population.
  arXiv  Detail & Related papers  (2023-02-15T23:01:59Z)
• Vision Based Machine Learning Algorithms for Out-of-Distribution Generalisation [3.236217153362305]
  We show that simple convolutional neural network (CNN) based deep learning methods perform poorly when they have to tackle domain shifting. Experiments are conducted on two popular vision-based benchmarks, PACS and Office-Home.
  arXiv  Detail & Related papers  (2023-01-17T15:58:29Z)
• Automatic Semantic Modeling for Structural Data Source with the Prior Knowledge from Knowledge Base [15.075047172918547]
  We propose a novel method for semantically annotating structured data sources using machine learning, graph matching and modified frequent subgraph mining. Our approach outperforms two state-of-theart solutions in tricky cases where only a few models are known.
  arXiv  Detail & Related papers  (2022-12-21T10:54:59Z)
• Generalization Properties of Retrieval-based Models [50.35325326050263]
  Retrieval-based machine learning methods have enjoyed success on a wide range of problems. Despite growing literature showcasing the promise of these models, the theoretical underpinning for such models remains underexplored. We present a formal treatment of retrieval-based models to characterize their generalization ability.
  arXiv  Detail & Related papers  (2022-10-06T00:33:01Z)
• HRKD: Hierarchical Relational Knowledge Distillation for Cross-domain Language Model Compression [53.90578309960526]
  Large pre-trained language models (PLMs) have shown overwhelming performances compared with traditional neural network methods. We propose a hierarchical relational knowledge distillation (HRKD) method to capture both hierarchical and domain relational information.
  arXiv  Detail & Related papers  (2021-10-16T11:23:02Z)
• Low-Resource Task-Oriented Semantic Parsing via Intrinsic Modeling [65.51280121472146]
  We exploit what we intrinsically know about ontology labels to build efficient semantic parsing models. Our model is highly efficient using a low-resource benchmark derived from TOPv2.
  arXiv  Detail & Related papers  (2021-04-15T04:01:02Z)
• Multifaceted Context Representation using Dual Attention for Ontology Alignment [6.445605125467574]
  Ontology alignment is an important research problem that finds application in various fields such as data integration, data transfer, data preparation etc. We propose VeeAlign, a Deep Learning based model that uses a dual-attention mechanism to compute the contextualized representation of a concept in order to learn alignments. We validate our approach on various datasets from different domains and in multilingual settings, and show its superior performance over SOTA methods.
  arXiv  Detail & Related papers  (2020-10-16T18:28:38Z)
• A Trainable Optimal Transport Embedding for Feature Aggregation and its Relationship to Attention [96.77554122595578]
  We introduce a parametrized representation of fixed size, which embeds and then aggregates elements from a given input set according to the optimal transport plan between the set and a trainable reference. Our approach scales to large datasets and allows end-to-end training of the reference, while also providing a simple unsupervised learning mechanism with small computational cost.
  arXiv  Detail & Related papers  (2020-06-22T08:35:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.