Transformer Architecture for NetsDB

• URL: http://arxiv.org/abs/2405.04807v2
• Date: Thu, 9 May 2024 12:02:22 GMT
• Title: Transformer Architecture for NetsDB
• Authors: Subodh Kamble, Kunal Sunil Kasodekar,
• Abstract summary: We create an end-to-end implementation of a transformer for deep learning model serving in NetsDB. We load out weights from our model for distributed processing, deployment, and efficient inferencing.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Transformers models have become the backbone of the current state-of-the-art models in language, vision, and multimodal domains. These models, at their core, utilize multi-head self-attention to selectively aggregate context, generating dynamic contextual embeddings and modeling long-range dependencies for a clear contextual understanding. Lixi et al. \cite{zhou2022serving} proposed a method to use relational databases for deploying large-scale deep learning models and created an open-source implementation called NetsDB for the same. We build upon the previous work of these authors by creating an end-to-end implementation of the Encoder part of the transformer for model serving in NetsDB. Specifically, we construct a two-block encoder that includes Multi-Head Attention and its accompanying self-attention mechanism, Layer-Norm, Dropout, FeedForward Layers, and the necessary residual connections. We load out weights from our model for distributed processing, deployment, and efficient inferencing. To prove the efficacy of our implementation, we conduct a comprehensive performance analysis by comparing it with existing implementations in PyTorch, Tensorflow, Flax, and MxNet across key metrics such as inference time and model size.

Related papers

• Two are better than one: Context window extension with multi-grained self-injection [111.1376461868317]
  SharedLLM is a novel approach grounded in the design philosophy of multi-grained context compression and query-aware information retrieval. We introduce a specialized tree-style data structure to efficiently encode, store and retrieve multi-grained contextual information for text chunks.
  arXiv  Detail & Related papers  (2024-10-25T06:08:59Z)
• EmbedLLM: Learning Compact Representations of Large Language Models [28.49433308281983]
  We propose EmbedLLM, a framework designed to learn compact vector representations of Large Language Models. We introduce an encoder-decoder approach for learning such embeddings, along with a systematic framework to evaluate their effectiveness. Empirical results show that EmbedLLM outperforms prior methods in model routing both in accuracy and latency.
  arXiv  Detail & Related papers  (2024-10-03T05:43:24Z)
• ACE: All-round Creator and Editor Following Instructions via Diffusion Transformer [40.32254040909614]
  We propose ACE, an All-round Creator and Editor, for visual generation tasks. We first introduce a unified condition format termed Long-context Condition Unit (LCU) We then propose a novel Transformer-based diffusion model that uses LCU as input, aiming for joint training across various generation and editing tasks.
  arXiv  Detail & Related papers  (2024-09-30T17:56:27Z)
• SortedNet: A Scalable and Generalized Framework for Training Modular Deep Neural Networks [30.069353400127046]
  We propose SortedNet to harness the inherent modularity of deep neural networks (DNNs) SortedNet enables the training of sub-models simultaneously along with the training of the main model. It is able to train 160 sub-models at once, achieving at least 96% of the original model's performance.
  arXiv  Detail & Related papers  (2023-09-01T05:12:25Z)
• On Optimizing the Communication of Model Parallelism [74.15423270435949]
  We study a novel and important communication pattern in large-scale model-parallel deep learning (DL) In cross-mesh resharding, a sharded tensor needs to be sent from a source device mesh to a destination device mesh. We propose two contributions to address cross-mesh resharding: an efficient broadcast-based communication system, and an "overlapping-friendly" pipeline schedule.
  arXiv  Detail & Related papers  (2022-11-10T03:56:48Z)
• Slimmable Domain Adaptation [112.19652651687402]
  We introduce a simple framework, Slimmable Domain Adaptation, to improve cross-domain generalization with a weight-sharing model bank. Our framework surpasses other competing approaches by a very large margin on multiple benchmarks.
  arXiv  Detail & Related papers  (2022-06-14T06:28:04Z)
• Retrieve-and-Fill for Scenario-based Task-Oriented Semantic Parsing [110.4684789199555]
  We introduce scenario-based semantic parsing: a variant of the original task which first requires disambiguating an utterance's "scenario" This formulation enables us to isolate coarse-grained and fine-grained aspects of the task, each of which we solve with off-the-shelf neural modules. Our model is modular, differentiable, interpretable, and allows us to garner extra supervision from scenarios.
  arXiv  Detail & Related papers  (2022-02-02T08:00:21Z)
• OneFlow: Redesign the Distributed Deep Learning Framework from Scratch [17.798586916628174]
  OneFlow is a novel distributed training framework based on an SBP (split, broadcast and partial-value) abstraction and the actor model. SBP enables much easier programming of data parallelism and model parallelism than existing frameworks. OneFlow outperforms many well-known customized libraries built on top of the state-of-the-art frameworks.
  arXiv  Detail & Related papers  (2021-10-28T11:32:14Z)
• HRKD: Hierarchical Relational Knowledge Distillation for Cross-domain Language Model Compression [53.90578309960526]
  Large pre-trained language models (PLMs) have shown overwhelming performances compared with traditional neural network methods. We propose a hierarchical relational knowledge distillation (HRKD) method to capture both hierarchical and domain relational information.
  arXiv  Detail & Related papers  (2021-10-16T11:23:02Z)
• Model Fusion via Optimal Transport [64.13185244219353]
  We present a layer-wise model fusion algorithm for neural networks. We show that this can successfully yield "one-shot" knowledge transfer between neural networks trained on heterogeneous non-i.i.d. data.
  arXiv  Detail & Related papers  (2019-10-12T22:07:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.