Multi-type Disentanglement without Adversarial Training
- URL: http://arxiv.org/abs/2012.08883v1
- Date: Wed, 16 Dec 2020 11:47:18 GMT
- Title: Multi-type Disentanglement without Adversarial Training
- Authors: Lei Sha, Thomas Lukasiewicz
- Abstract summary: Controlling the style of natural language by disentangling the latent space is an important step towards interpretable machine learning.
We propose a unified distribution-controlling method, which provides each specific style value with a unique representation.
We also propose multiple loss functions to achieve a style-content disentanglement as well as a disentanglement among multiple style types.
- Score: 48.51678740102892
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Controlling the style of natural language by disentangling the latent space
is an important step towards interpretable machine learning. After the latent
space is disentangled, the style of a sentence can be transformed by tuning the
style representation without affecting other features of the sentence. Previous
works usually use adversarial training to guarantee that disentangled vectors
do not affect each other. However, adversarial methods are difficult to train.
Especially when there are multiple features (e.g., sentiment, or tense, which
we call style types in this paper), each feature requires a separate
discriminator for extracting a disentangled style vector corresponding to that
feature. In this paper, we propose a unified distribution-controlling method,
which provides each specific style value (the value of style types, e.g.,
positive sentiment, or past tense) with a unique representation. This method
contributes a solid theoretical basis to avoid adversarial training in
multi-type disentanglement. We also propose multiple loss functions to achieve
a style-content disentanglement as well as a disentanglement among multiple
style types. In addition, we observe that if two different style types always
have some specific style values that occur together in the dataset, they will
affect each other when transferring the style values. We call this phenomenon
training bias, and we propose a loss function to alleviate such training bias
while disentangling multiple types. We conduct experiments on two datasets
(Yelp service reviews and Amazon product reviews) to evaluate the
style-disentangling effect and the unsupervised style transfer performance on
two style types: sentiment and tense. The experimental results show the
effectiveness of our model.
Related papers
- Reinforcement Learning with Dynamic Multi-Reward Weighting for
Multi-Style Controllable Generation [17.937198263444046]
Humans often employ multiple styles simultaneously.
We investigate various formulations of multiple style rewards for a reinforcement learning approach to controlled multi-style generation.
All code and data for RL pipelines with multiple style attributes will be publicly available.
arXiv Detail & Related papers (2024-02-21T22:02:37Z) - Amortised Invariance Learning for Contrastive Self-Supervision [11.042648980854485]
We introduce the notion of amortised invariance learning for contrastive self supervision.
We show that our amortised features provide a reliable way to learn diverse downstream tasks with different invariance requirements.
This provides an exciting perspective that opens up new horizons in the field of general purpose representation learning.
arXiv Detail & Related papers (2023-02-24T16:15:11Z) - Combining Contrastive and Non-Contrastive Losses for Fine-Tuning
Pretrained Models in Speech Analysis [25.707717591185386]
We propose a two step approach to finetuning paralinguistic properties.
First we improve the embedding space, then we train an adapter to bridge the gap from the embedding space to a classification task.
Our approach consistently outperforms baselines that are finetuned end-to-end on multiple tasks and surpasses a benchmark on state-of-the-art emotion classification.
arXiv Detail & Related papers (2022-10-21T19:58:37Z) - Domain Enhanced Arbitrary Image Style Transfer via Contrastive Learning [84.8813842101747]
Contrastive Arbitrary Style Transfer (CAST) is a new style representation learning and style transfer method via contrastive learning.
Our framework consists of three key components, i.e., a multi-layer style projector for style code encoding, a domain enhancement module for effective learning of style distribution, and a generative network for image style transfer.
arXiv Detail & Related papers (2022-05-19T13:11:24Z) - Transductive Learning for Unsupervised Text Style Transfer [60.65782243927698]
Unsupervised style transfer models are mainly based on an inductive learning approach.
We propose a novel transductive learning approach based on a retrieval-based context-aware style representation.
arXiv Detail & Related papers (2021-09-16T08:57:20Z) - Hard-label Manifolds: Unexpected Advantages of Query Efficiency for
Finding On-manifold Adversarial Examples [67.23103682776049]
Recent zeroth order hard-label attacks on image classification models have shown comparable performance to their first-order, gradient-level alternatives.
It was recently shown in the gradient-level setting that regular adversarial examples leave the data manifold, while their on-manifold counterparts are in fact generalization errors.
We propose an information-theoretic argument based on a noisy manifold distance oracle, which leaks manifold information through the adversary's gradient estimate.
arXiv Detail & Related papers (2021-03-04T20:53:06Z) - PREGAN: Pose Randomization and Estimation for Weakly Paired Image Style
Translation [11.623477199795037]
We propose a weakly-paired setting for the style translation, where the content in the two images is aligned with errors in poses.
PREGAN is validated on both simulated and real-world collected data to show the effectiveness.
arXiv Detail & Related papers (2020-10-31T16:11:11Z) - What Should Not Be Contrastive in Contrastive Learning [110.14159883496859]
We introduce a contrastive learning framework which does not require prior knowledge of specific, task-dependent invariances.
Our model learns to capture varying and invariant factors for visual representations by constructing separate embedding spaces.
We use a multi-head network with a shared backbone which captures information across each augmentation and alone outperforms all baselines on downstream tasks.
arXiv Detail & Related papers (2020-08-13T03:02:32Z) - Exploring Contextual Word-level Style Relevance for Unsupervised Style
Transfer [60.07283363509065]
Unsupervised style transfer aims to change the style of an input sentence while preserving its original content.
We propose a novel attentional sequence-to-sequence model that exploits the relevance of each output word to the target style.
Experimental results show that our proposed model achieves state-of-the-art performance in terms of both transfer accuracy and content preservation.
arXiv Detail & Related papers (2020-05-05T10:24:28Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.