KerasCV and KerasNLP: Vision and Language Power-Ups

• URL: http://arxiv.org/abs/2405.20247v3
• Date: Wed, 5 Jun 2024 07:52:07 GMT
• Title: KerasCV and KerasNLP: Vision and Language Power-Ups
• Authors: Matthew Watson, Divyashree Shivakumar Sreepathihalli, Francois Chollet, Martin Gorner, Kiranbir Sodhia, Ramesh Sampath, Tirth Patel, Haifeng Jin, Neel Kovelamudi, Gabriel Rasskin, Samaneh Saadat, Luke Wood, Chen Qian, Jonathan Bischof, Ian Stenbit, Abheesht Sharma, Anshuman Mishra,
• Abstract summary: KerasCV and KerasNLP are extensions of the Keras API for Computer Vision and Natural Language Processing. These domain packages are designed to enable fast experimentation, with a focus on ease-of-use and performance. The libraries are fully open-source (Apache 2.0 license) and available on GitHub.
• Score: 9.395199188271254
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present the Keras domain packages KerasCV and KerasNLP, extensions of the Keras API for Computer Vision and Natural Language Processing workflows, capable of running on either JAX, TensorFlow, or PyTorch. These domain packages are designed to enable fast experimentation, with a focus on ease-of-use and performance. We adopt a modular, layered design: at the library's lowest level of abstraction, we provide building blocks for creating models and data preprocessing pipelines, and at the library's highest level of abstraction, we provide pretrained ``task" models for popular architectures such as Stable Diffusion, YOLOv8, GPT2, BERT, Mistral, CLIP, Gemma, T5, etc. Task models have built-in preprocessing, pretrained weights, and can be fine-tuned on raw inputs. To enable efficient training, we support XLA compilation for all models, and run all preprocessing via a compiled graph of TensorFlow operations using the tf.data API. The libraries are fully open-source (Apache 2.0 license) and available on GitHub.

Related papers

• Deep Fast Machine Learning Utils: A Python Library for Streamlined Machine Learning Prototyping [0.0]
  The Deep Fast Machine Learning Utils (DFMLU) library provides tools designed to automate and enhance aspects of machine learning processes. DFMLU offers functionalities that support model development and data handling. This manuscript presents an overview of DFMLU's functionalities, providing Python examples for each tool.
  arXiv  Detail & Related papers  (2024-09-14T21:39:17Z)
• pyvene: A Library for Understanding and Improving PyTorch Models via Interventions [79.72930339711478]
  $textbfpyvene$ is an open-source library that supports customizable interventions on a range of different PyTorch modules. We show how $textbfpyvene$ provides a unified framework for performing interventions on neural models and sharing the intervened upon models with others.
  arXiv  Detail & Related papers  (2024-03-12T16:46:54Z)
• LMFlow: An Extensible Toolkit for Finetuning and Inference of Large Foundation Models [31.121714473817793]
  Foundation models have demonstrated a great ability to achieve general human-level intelligence far beyond traditional approaches. A significant shortcoming of most foundation models lies in their performance in specialized-domain and task-specific applications. We introduce LMFlow, which aims to simplify the domain- and task-aware finetuning of general foundation models.
  arXiv  Detail & Related papers  (2023-06-21T17:58:25Z)
• SequeL: A Continual Learning Library in PyTorch and JAX [50.33956216274694]
  SequeL is a library for Continual Learning that supports both PyTorch and JAX frameworks. It provides a unified interface for a wide range of Continual Learning algorithms, including regularization-based approaches, replay-based approaches, and hybrid approaches. We release SequeL as an open-source library, enabling researchers and developers to easily experiment and extend the library for their own purposes.
  arXiv  Detail & Related papers  (2023-04-21T10:00:22Z)
• DINOv2: Learning Robust Visual Features without Supervision [75.42921276202522]
  This work shows that existing pretraining methods, especially self-supervised methods, can produce such features if trained on enough curated data from diverse sources. Most of the technical contributions aim at accelerating and stabilizing the training at scale. In terms of data, we propose an automatic pipeline to build a dedicated, diverse, and curated image dataset instead of uncurated data, as typically done in the self-supervised literature.
  arXiv  Detail & Related papers  (2023-04-14T15:12:19Z)
• Trieste: Efficiently Exploring The Depths of Black-box Functions with TensorFlow [50.691232400959656]
  Trieste is an open-source Python package for Bayesian optimization and active learning. Our library enables the plug-and-play of popular models within sequential decision-making loops.
  arXiv  Detail & Related papers  (2023-02-16T17:21:49Z)
• MAP: Memory-aware Automated Intra-op Parallel Training For Foundation Models [15.256207550970501]
  We introduce MAP, a compiler built upon PyTorch to implement Memory-aware Automated Parallelization. Compared with existing methods, MAP provides an easy-to-use symbolic profiler to generate memory and computing statistics of an arbitrary PyTorch model.
  arXiv  Detail & Related papers  (2023-02-06T07:22:49Z)
• Scaling Up Models and Data with $\texttt{t5x}$ and $\texttt{seqio}$ [118.04625413322827]
  $texttt5x$ and $texttseqio$ are open source software libraries for building and training language models. These libraries have been used to train models with hundreds of billions of parameters on datasets with multiple terabytes of training data.
  arXiv  Detail & Related papers  (2022-03-31T17:12:13Z)
• SOLIS -- The MLOps journey from data acquisition to actionable insights [62.997667081978825]
  In this paper we present a unified deployment pipeline and freedom-to-operate approach that supports all requirements while using basic cross-platform tensor framework and script language engines. This approach however does not supply the needed procedures and pipelines for the actual deployment of machine learning capabilities in real production grade systems.
  arXiv  Detail & Related papers  (2021-12-22T14:45:37Z)
• TensorX: Extensible API for Neural Network Model Design and Deployment [0.0]
  TensorFlowX is a Python library for prototyping, design, and deployment of complex neural network models in computation. A special emphasis is put on ease of use, performance, and API consistency.
  arXiv  Detail & Related papers  (2020-12-29T00:15:38Z)
• fastai: A Layered API for Deep Learning [1.7223564681760164]
  fastai is a deep learning library which provides practitioners with high-level components. It provides researchers with low-level components that can be mixed and matched to build new approaches.
  arXiv  Detail & Related papers  (2020-02-11T21:16:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.