Generative Model for Models: Rapid DNN Customization for Diverse Tasks
and Resource Constraints
- URL: http://arxiv.org/abs/2308.15003v1
- Date: Tue, 29 Aug 2023 03:28:14 GMT
- Title: Generative Model for Models: Rapid DNN Customization for Diverse Tasks
and Resource Constraints
- Authors: Wenxing Xu, Yuanchun Li, Jiacheng Liu, Yi Sun, Zhengyang Cao, Yixuan
Li, Hao Wen, Yunxin Liu
- Abstract summary: NN-Factory is a one-for-all framework to generate customized lightweight models for diverse edge scenarios.
The main components of NN-Factory include a modular supernet with pretrained modules that can be conditionally activated to accomplish different tasks.
NN-Factory is able to generate high-quality task- and resource-specific models within few seconds, faster than conventional model customization approaches by orders of magnitude.
- Score: 28.983470365172057
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Unlike cloud-based deep learning models that are often large and uniform,
edge-deployed models usually demand customization for domain-specific tasks and
resource-limited environments. Such customization processes can be costly and
time-consuming due to the diversity of edge scenarios and the training load for
each scenario. Although various approaches have been proposed for rapid
resource-oriented customization and task-oriented customization respectively,
achieving both of them at the same time is challenging. Drawing inspiration
from the generative AI and the modular composability of neural networks, we
introduce NN-Factory, an one-for-all framework to generate customized
lightweight models for diverse edge scenarios. The key idea is to use a
generative model to directly produce the customized models, instead of training
them. The main components of NN-Factory include a modular supernet with
pretrained modules that can be conditionally activated to accomplish different
tasks and a generative module assembler that manipulate the modules according
to task and sparsity requirements. Given an edge scenario, NN-Factory can
efficiently customize a compact model specialized in the edge task while
satisfying the edge resource constraints by searching for the optimal strategy
to assemble the modules. Based on experiments on image classification and
object detection tasks with different edge devices, NN-Factory is able to
generate high-quality task- and resource-specific models within few seconds,
faster than conventional model customization approaches by orders of magnitude.
Related papers
- A Model Is Not Built By A Single Prompt: LLM-Based Domain Modeling With Question Decomposition [4.123601037699469]
In real-world domain modeling, engineers usually decompose complex tasks into easily solvable sub-tasks.
We propose an LLM-based domain modeling approach via question decomposition, similar to developer's modeling process.
Preliminary results show that our approach outperforms the single-prompt-based prompt.
arXiv Detail & Related papers (2024-10-13T14:28:04Z) - SortedNet: A Scalable and Generalized Framework for Training Modular Deep Neural Networks [30.069353400127046]
We propose SortedNet to harness the inherent modularity of deep neural networks (DNNs)
SortedNet enables the training of sub-models simultaneously along with the training of the main model.
It is able to train 160 sub-models at once, achieving at least 96% of the original model's performance.
arXiv Detail & Related papers (2023-09-01T05:12:25Z) - OFASys: A Multi-Modal Multi-Task Learning System for Building Generalist
Models [72.8156832931841]
Generalist models are capable of performing diverse multi-modal tasks in a task-agnostic way within a single model.
We release a generalist model learning system, OFASys, built on top of a declarative task interface named multi-modal instruction.
arXiv Detail & Related papers (2022-12-08T17:07:09Z) - Deep Model Reassembly [60.6531819328247]
We explore a novel knowledge-transfer task, termed as Deep Model Reassembly (DeRy)
The goal of DeRy is to first dissect each model into distinctive building blocks, and then selectively reassemble the derived blocks to produce customized networks.
We demonstrate that on ImageNet, the best reassemble model achieves 78.6% top-1 accuracy without fine-tuning.
arXiv Detail & Related papers (2022-10-24T10:16:13Z) - muNet: Evolving Pretrained Deep Neural Networks into Scalable
Auto-tuning Multitask Systems [4.675744559395732]
Most uses of machine learning today involve training a model from scratch for a particular task, or starting with a model pretrained on a related task and then fine-tuning on a downstream task.
We propose a method that uses the layers of a pretrained deep neural network as building blocks to construct an ML system that can jointly solve an arbitrary number of tasks.
The resulting system can leverage cross tasks knowledge transfer, while being immune from common drawbacks of multitask approaches such as catastrophic forgetting, gradients interference and negative transfer.
arXiv Detail & Related papers (2022-05-22T21:54:33Z) - Task Adaptive Parameter Sharing for Multi-Task Learning [114.80350786535952]
Adaptive Task Adapting Sharing (TAPS) is a method for tuning a base model to a new task by adaptively modifying a small, task-specific subset of layers.
Compared to other methods, TAPS retains high accuracy on downstream tasks while introducing few task-specific parameters.
We evaluate our method on a suite of fine-tuning tasks and architectures (ResNet, DenseNet, ViT) and show that it achieves state-of-the-art performance while being simple to implement.
arXiv Detail & Related papers (2022-03-30T23:16:07Z) - Retrieve-and-Fill for Scenario-based Task-Oriented Semantic Parsing [110.4684789199555]
We introduce scenario-based semantic parsing: a variant of the original task which first requires disambiguating an utterance's "scenario"
This formulation enables us to isolate coarse-grained and fine-grained aspects of the task, each of which we solve with off-the-shelf neural modules.
Our model is modular, differentiable, interpretable, and allows us to garner extra supervision from scenarios.
arXiv Detail & Related papers (2022-02-02T08:00:21Z) - Data Summarization via Bilevel Optimization [48.89977988203108]
A simple yet powerful approach is to operate on small subsets of data.
In this work, we propose a generic coreset framework that formulates the coreset selection as a cardinality-constrained bilevel optimization problem.
arXiv Detail & Related papers (2021-09-26T09:08:38Z) - Conditional Generative Modeling via Learning the Latent Space [54.620761775441046]
We propose a novel framework for conditional generation in multimodal spaces.
It uses latent variables to model generalizable learning patterns.
At inference, the latent variables are optimized to find optimal solutions corresponding to multiple output modes.
arXiv Detail & Related papers (2020-10-07T03:11:34Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.