Neural Composition: Learning to Generate from Multiple Models
- URL: http://arxiv.org/abs/2007.16013v2
- Date: Mon, 9 Nov 2020 23:41:47 GMT
- Title: Neural Composition: Learning to Generate from Multiple Models
- Authors: Denis Filimonov, Ravi Teja Gadde, Ariya Rastrow
- Abstract summary: We propose a system that combines model-defined components, by learning when to activate the generation process from each individual component.
In this paper, we propose a system that combines model-defined components, by learning when to activate the generation process from each individual component.
- Score: 13.072708028188465
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Decomposing models into multiple components is critically important in many
applications such as language modeling (LM) as it enables adapting individual
components separately and biasing of some components to the user's personal
preferences. Conventionally, contextual and personalized adaptation for
language models, are achieved through class-based factorization, which requires
class-annotated data, or through biasing to individual phrases which is limited
in scale. In this paper, we propose a system that combines model-defined
components, by learning when to activate the generation process from each
individual component, and how to combine probability distributions from each
component, directly from unlabeled text data.
Related papers
- JeDi: Joint-Image Diffusion Models for Finetuning-Free Personalized Text-to-Image Generation [49.997839600988875]
Existing personalization methods rely on finetuning a text-to-image foundation model on a user's custom dataset.
We propose Joint-Image Diffusion (jedi), an effective technique for learning a finetuning-free personalization model.
Our model achieves state-of-the-art generation quality, both quantitatively and qualitatively, significantly outperforming both the prior finetuning-based and finetuning-free personalization baselines.
arXiv Detail & Related papers (2024-07-08T17:59:02Z) - Personalized Federated Learning via Sequential Layer Expansion in Representation Learning [0.0]
Federated learning ensures the privacy of clients by conducting distributed training on individual client devices and sharing only the model weights with a central server.
We propose a new representation learning-based approach that suggests decoupling the entire deep learning model into more densely divided parts with the application of suitable scheduling methods.
arXiv Detail & Related papers (2024-04-27T06:37:19Z) - Decomposing and Editing Predictions by Modeling Model Computation [75.37535202884463]
We introduce a task called component modeling.
The goal of component modeling is to decompose an ML model's prediction in terms of its components.
We present COAR, a scalable algorithm for estimating component attributions.
arXiv Detail & Related papers (2024-04-17T16:28:08Z) - Dynamic Latent Separation for Deep Learning [67.62190501599176]
A core problem in machine learning is to learn expressive latent variables for model prediction on complex data.
Here, we develop an approach that improves expressiveness, provides partial interpretation, and is not restricted to specific applications.
arXiv Detail & Related papers (2022-10-07T17:56:53Z) - Dynamic Template Initialization for Part-Aware Person Re-ID [0.640781528166787]
spatial attention-based Dynamic Part template Initialization module.
Part-level features of the backbone are used to extract the templates of diverse human body parts.
Tests on holistic, occluded, and partial Re-ID task benchmarks.
arXiv Detail & Related papers (2022-08-24T11:20:48Z) - FlexLip: A Controllable Text-to-Lip System [6.15560473113783]
We tackle a subissue of the text-to-video generation problem, by converting the text into lip landmarks.
Our system, entitled FlexLip, is split into two separate modules: text-to-speech and speech-to-lip.
We show that by using as little as 20 min of data for the audio generation component, and as little as 5 min for the speech-to-lip component, the objective measures of the generated lip landmarks are comparable with those obtained when using a larger set of training samples.
arXiv Detail & Related papers (2022-06-07T11:51:58Z) - Dependency-based Mixture Language Models [53.152011258252315]
We introduce the Dependency-based Mixture Language Models.
In detail, we first train neural language models with a novel dependency modeling objective.
We then formulate the next-token probability by mixing the previous dependency modeling probability distributions with self-attention.
arXiv Detail & Related papers (2022-03-19T06:28:30Z) - Compositional Fine-Grained Low-Shot Learning [58.53111180904687]
We develop a novel compositional generative model for zero- and few-shot learning to recognize fine-grained classes with a few or no training samples.
We propose a feature composition framework that learns to extract attribute features from training samples and combines them to construct fine-grained features for rare and unseen classes.
arXiv Detail & Related papers (2021-05-21T16:18:24Z) - Grounded Compositional Outputs for Adaptive Language Modeling [59.02706635250856]
A language model's vocabulary$-$typically selected before training and permanently fixed later$-$affects its size.
We propose a fully compositional output embedding layer for language models.
To our knowledge, the result is the first word-level language model with a size that does not depend on the training vocabulary.
arXiv Detail & Related papers (2020-09-24T07:21:14Z) - Personalized Federated Learning: A Meta-Learning Approach [28.281166755509886]
In Federated Learning, we aim to train models across multiple computing units (users)
In this paper, we study a personalized variant of the federated learning in which our goal is to find an initial shared model that current or new users can easily adapt to their local dataset by performing one or a few steps of gradient descent with respect to their own data.
arXiv Detail & Related papers (2020-02-19T01:08:46Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.