Pixyz: a Python library for developing deep generative models

• URL: http://arxiv.org/abs/2107.13109v3
• Date: Thu, 21 Sep 2023 18:04:11 GMT
• Title: Pixyz: a Python library for developing deep generative models
• Authors: Masahiro Suzuki, Takaaki Kaneko, Yutaka Matsuo
• Abstract summary: We propose a new Python library to implement deep generative models (DGMs) called Pixyz. This library adopts a step-by-step implementation method with three APIs, which allows us to implement various DGMs more concisely and intuitively. In addition, the library introduces memoization to reduce the cost of duplicate computations in DGMs to speed up the computation.
• Score: 23.334186745540485
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: With the recent rapid progress in the study of deep generative models (DGMs), there is a need for a framework that can implement them in a simple and generic way. In this research, we focus on two features of DGMs: (1) deep neural networks are encapsulated by probability distributions, and (2) models are designed and learned based on an objective function. Taking these features into account, we propose a new Python library to implement DGMs called Pixyz. This library adopts a step-by-step implementation method with three APIs, which allows us to implement various DGMs more concisely and intuitively. In addition, the library introduces memoization to reduce the cost of duplicate computations in DGMs to speed up the computation. We demonstrate experimentally that this library is faster than existing probabilistic programming languages in training DGMs.

Related papers

• Deep Fast Machine Learning Utils: A Python Library for Streamlined Machine Learning Prototyping [0.0]
  The Deep Fast Machine Learning Utils (DFMLU) library provides tools designed to automate and enhance aspects of machine learning processes. DFMLU offers functionalities that support model development and data handling. This manuscript presents an overview of DFMLU's functionalities, providing Python examples for each tool.
  arXiv  Detail & Related papers  (2024-09-14T21:39:17Z)
• Enabling High-Sparsity Foundational Llama Models with Efficient Pretraining and Deployment [56.44025052765861]
  Large language models (LLMs) have revolutionized Natural Language Processing (NLP), but their size creates computational bottlenecks. We introduce a novel approach to create accurate, sparse foundational versions of performant LLMs. We show a total speedup on CPUs for sparse-quantized LLaMA models of up to 8.6x.
  arXiv  Detail & Related papers  (2024-05-06T16:03:32Z)
• CyNetDiff -- A Python Library for Accelerated Implementation of Network Diffusion Models [0.9831489366502302]
  CyNetDiff is a Python library with components written in Cython to provide improved performance for these computationally intensive diffusion tasks. In many research tasks, these simulations are the most computationally intensive task, so it would be desirable to have a library for these with an interface to a high-level language.
  arXiv  Detail & Related papers  (2024-04-25T21:59:55Z)
• Multimodal Learned Sparse Retrieval with Probabilistic Expansion Control [66.78146440275093]
  Learned retrieval (LSR) is a family of neural methods that encode queries and documents into sparse lexical vectors. We explore the application of LSR to the multi-modal domain, with a focus on text-image retrieval. Current approaches like LexLIP and STAIR require complex multi-step training on massive datasets. Our proposed approach efficiently transforms dense vectors from a frozen dense model into sparse lexical vectors.
  arXiv  Detail & Related papers  (2024-02-27T14:21:56Z)
• Julearn: an easy-to-use library for leakage-free evaluation and inspection of ML models [0.23301643766310373]
  We present the rationale behind julearn's design, its core features, and showcase three examples of previously-published research projects. Julearn aims to simplify the entry into the machine learning world by providing an easy-to-use environment with built in guards against some of the most common ML pitfalls.
  arXiv  Detail & Related papers  (2023-10-19T08:21:12Z)
• CodeGen2: Lessons for Training LLMs on Programming and Natural Languages [116.74407069443895]
  We unify encoder and decoder-based models into a single prefix-LM. For learning methods, we explore the claim of a "free lunch" hypothesis. For data distributions, the effect of a mixture distribution and multi-epoch training of programming and natural languages on model performance is explored.
  arXiv  Detail & Related papers  (2023-05-03T17:55:25Z)
• DeeProb-kit: a Python Library for Deep Probabilistic Modelling [0.0]
  DeeProb-kit is a unified library written in Python consisting of a collection of deep probabilistic models (DPMs) It includes efficiently implemented learning techniques, inference routines, statistical algorithms, and provides high-quality fully-documented APIs.
  arXiv  Detail & Related papers  (2022-12-08T17:02:16Z)
• Pythae: Unifying Generative Autoencoders in Python -- A Benchmarking Use Case [0.0]
  We present Pythae, a versatile open-source Python library providing straightforward, reproducible and reliable use of generative autoencoder models. We present and compare 19 generative autoencoder models representative of some of the main improvements on downstream tasks.
  arXiv  Detail & Related papers  (2022-06-16T17:11:41Z)
• Efficient Nearest Neighbor Language Models [114.40866461741795]
  Non-parametric neural language models (NLMs) learn predictive distributions of text utilizing an external datastore. We show how to achieve up to a 6x speed-up in inference speed while retaining comparable performance.
  arXiv  Detail & Related papers  (2021-09-09T12:32:28Z)
• Solving Mixed Integer Programs Using Neural Networks [57.683491412480635]
  This paper applies learning to the two key sub-tasks of a MIP solver, generating a high-quality joint variable assignment, and bounding the gap in objective value between that assignment and an optimal one. Our approach constructs two corresponding neural network-based components, Neural Diving and Neural Branching, to use in a base MIP solver such as SCIP. We evaluate our approach on six diverse real-world datasets, including two Google production datasets and MIPLIB, by training separate neural networks on each.
  arXiv  Detail & Related papers  (2020-12-23T09:33:11Z)
• Captum: A unified and generic model interpretability library for PyTorch [49.72749684393332]
  We introduce a novel, unified, open-source model interpretability library for PyTorch. The library contains generic implementations of a number of gradient and perturbation-based attribution algorithms. It can be used for both classification and non-classification models.
  arXiv  Detail & Related papers  (2020-09-16T18:57:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.