SPILDL: A Scalable and Parallel Inductive Learner in Description Logic

• URL: http://arxiv.org/abs/2412.00830v1
• Date: Sun, 01 Dec 2024 14:33:37 GMT
• Title: SPILDL: A Scalable and Parallel Inductive Learner in Description Logic
• Authors: Eyad Algahtani,
• Abstract summary: SPILDL is based on the DL-Learner (the state of the art in DL-based ILP learning) As a DL-based ILP learner, SPILDL targets the $mathcalALCQImathcal(D)$ DL language, and can learn DL hypotheses expressed as disjunctions of conjunctions. SPILDL employs a hybrid parallel approach which combines both shared-memory and distributed-memory approaches.
• Score: 0.0
• License:
• Abstract: We present SPILDL, a Scalable and Parallel Inductive Learner in Description Logic (DL). SPILDL is based on the DL-Learner (the state of the art in DL-based ILP learning). As a DL-based ILP learner, SPILDL targets the $\mathcal{ALCQI}^{\mathcal{(D)}}$ DL language, and can learn DL hypotheses expressed as disjunctions of conjunctions (using the $\sqcup$ operator). Moreover, SPILDL's hypothesis language also incorporates the use of string concrete roles (also known as string data properties in the Web Ontology Language, OWL); As a result, this incorporation of powerful DL constructs, enables SPILDL to learn powerful DL-based hypotheses for describing many real-world complex concepts. SPILDL employs a hybrid parallel approach which combines both shared-memory and distributed-memory approaches, to accelerates ILP learning (for both hypothesis search and evaluation). According to experimental results, SPILDL's parallel search improved performance by up to $\sim$27.3 folds (best case). For hypothesis evaluation, SPILDL improved evaluation performance through HT-HEDL (our multi-core CPU + multi-GPU hypothesis evaluation engine), by up to 38 folds (best case). By combining both parallel search and evaluation, SPILDL improved performance by up to $\sim$560 folds (best case). In terms of worst case scenario, SPILDL's parallel search doesn't provide consistent speedups on all datasets, and is highly dependent on the search space nature of the ILP dataset. For some datasets, increasing the number of parallel search threads result in reduced performance, similar or worse than baseline. Some ILP datasets benefit from parallel search, while others don't (or the performance gains are negligible). In terms of parallel evaluation, on small datasets, parallel evaluation provide similar or worse performance than baseline.

Related papers

• Accelerating Large Language Model Training with 4D Parallelism and Memory Consumption Estimator [4.953653137620666]
  In large language model (LLM) training, several parallelization strategies, including Parallelism (TP), Pipeline Parallelism (PP), Data Parallelism (DP), are employed. We provide precise formulas to estimate the memory consumed by parameters, gradients, states, and activations for 4D parallel training (DP, TP, PP, CP) in the Llama architecture. Results indicate that when the estimated memory usage is below 80% of the available GPU memory, the training never encounters out-of-memory errors.
  arXiv  Detail & Related papers  (2024-11-10T13:45:08Z)
• DARG: Dynamic Evaluation of Large Language Models via Adaptive Reasoning Graph [70.79413606968814]
  We introduce Dynamic Evaluation of LLMs via Adaptive Reasoning Graph Evolvement (DARG) to dynamically extend current benchmarks with controlled complexity and diversity. Specifically, we first extract the reasoning graphs of data points in current benchmarks and then perturb the reasoning graphs to generate novel testing data. Such newly generated test samples can have different levels of complexity while maintaining linguistic diversity similar to the original benchmarks.
  arXiv  Detail & Related papers  (2024-06-25T04:27:53Z)
• Multimodal Learned Sparse Retrieval with Probabilistic Expansion Control [66.78146440275093]
  Learned retrieval (LSR) is a family of neural methods that encode queries and documents into sparse lexical vectors. We explore the application of LSR to the multi-modal domain, with a focus on text-image retrieval. Current approaches like LexLIP and STAIR require complex multi-step training on massive datasets. Our proposed approach efficiently transforms dense vectors from a frozen dense model into sparse lexical vectors.
  arXiv  Detail & Related papers  (2024-02-27T14:21:56Z)
• LLM Performance Predictors are good initializers for Architecture Search [28.251129134057035]
  We construct Performance Predictors (PP) that estimate the performance of specific deep neural network architectures on downstream tasks. In machine translation (MT) tasks, GPT-4 with our PP prompts (LLM-PP) achieves a SoTA mean absolute error and a slight degradation in rank correlation coefficient compared to baseline predictors. For Neural Architecture Search (NAS), we introduce a Hybrid-Search algorithm (HS-NAS) employing LLM-Distill-PP for the initial search stages and reverting to the baseline predictor later.
  arXiv  Detail & Related papers  (2023-10-25T15:34:30Z)
• Automatic Task Parallelization of Dataflow Graphs in ML/DL models [0.0]
  We present a Linear Clustering approach to exploit inherent parallel paths in ML dataflow graphs. We generate readable and executable parallel Pytorch+Python code from input ML models in ONNX format. Preliminary results on several ML graphs demonstrate up to 1.9$times$ speedup over serial execution.
  arXiv  Detail & Related papers  (2023-08-22T04:54:30Z)
• Parallel $Q$-Learning: Scaling Off-policy Reinforcement Learning under Massively Parallel Simulation [17.827002299991285]
  Reinforcement learning is time-consuming for complex tasks due to the need for large amounts of training data. Recent advances in GPU-based simulation, such as Isaac Gym, have sped up data collection thousands of times on a commodity GPU. This paper presents a Parallel $Q$-Learning scheme that outperforms PPO in wall-clock time.
  arXiv  Detail & Related papers  (2023-07-24T17:59:37Z)
• PARTIME: Scalable and Parallel Processing Over Time with Deep Neural Networks [68.96484488899901]
  We present PARTIME, a library designed to speed up neural networks whenever data is continuously streamed over time. PARTIME starts processing each data sample at the time in which it becomes available from the stream. Experiments are performed in order to empirically compare PARTIME with classic non-parallel neural computations in online learning.
  arXiv  Detail & Related papers  (2022-10-17T14:49:14Z)
• IRLI: Iterative Re-partitioning for Learning to Index [104.72641345738425]
  Methods have to trade between obtaining high accuracy while maintaining load balance and scalability in distributed settings. We propose a novel approach called IRLI, which iteratively partitions the items by learning the relevant buckets directly from the query-item relevance data. We mathematically show that IRLI retrieves the correct item with high probability under very natural assumptions and provides superior load balancing.
  arXiv  Detail & Related papers  (2021-03-17T23:13:25Z)
• Parallel Training of Deep Networks with Local Updates [84.30918922367442]
  Local parallelism is a framework which parallelizes training of individual layers in deep networks by replacing global backpropagation with truncated layer-wise backpropagation. We show results in both vision and language domains across a diverse set of architectures, and find that local parallelism is particularly effective in the high-compute regime.
  arXiv  Detail & Related papers  (2020-12-07T16:38:45Z)
• Is deep learning necessary for simple classification tasks? [3.3793659640122717]
  Automated machine learning (AutoML) and deep learning (DL) are two cutting-edge paradigms used to solve inductive learning tasks. We compare AutoML and DL in the context of binary classification on 6 well-characterized public datasets. We also evaluate a new tool for genetic programming-based AutoML that incorporates deep estimators.
  arXiv  Detail & Related papers  (2020-06-11T18:41:47Z)
• RadixSpline: A Single-Pass Learned Index [84.84747738666263]
  We introduce RadixSpline (RS), a learned index that can be built in a single pass over the data. RS achieves competitive results on all datasets, despite the fact that it only has two parameters.
  arXiv  Detail & Related papers  (2020-04-30T01:56:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.