Transductive Learning for Unsupervised Text Style Transfer
- URL: http://arxiv.org/abs/2109.07812v1
- Date: Thu, 16 Sep 2021 08:57:20 GMT
- Title: Transductive Learning for Unsupervised Text Style Transfer
- Authors: Fei Xiao, Liang Pang, Yanyan Lan, Yan Wang, Huawei Shen and Xueqi
Cheng
- Abstract summary: Unsupervised style transfer models are mainly based on an inductive learning approach.
We propose a novel transductive learning approach based on a retrieval-based context-aware style representation.
- Score: 60.65782243927698
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Unsupervised style transfer models are mainly based on an inductive learning
approach, which represents the style as embeddings, decoder parameters, or
discriminator parameters and directly applies these general rules to the test
cases. However, the lacking of parallel corpus hinders the ability of these
inductive learning methods on this task. As a result, it is likely to cause
severe inconsistent style expressions, like `the salad is rude`. To tackle this
problem, we propose a novel transductive learning approach in this paper, based
on a retrieval-based context-aware style representation. Specifically, an
attentional encoder-decoder with a retriever framework is utilized. It involves
top-K relevant sentences in the target style in the transfer process. In this
way, we can learn a context-aware style embedding to alleviate the above
inconsistency problem. In this paper, both sparse (BM25) and dense retrieval
functions (MIPS) are used, and two objective functions are designed to
facilitate joint learning. Experimental results show that our method
outperforms several strong baselines. The proposed transductive learning
approach is general and effective to the task of unsupervised style transfer,
and we will apply it to the other two typical methods in the future.
Related papers
- Advancing Prompt Learning through an External Layer [24.77977865016954]
We propose a paradigm called EnPrompt with a novel External Layer (EnLa)
The learnable external layer is built upon valid embeddings of pre-trained CLIP.
Four experiments demonstrate that our method outperforms the existing prompt learning method.
arXiv Detail & Related papers (2024-07-29T03:30:09Z) - SETTP: Style Extraction and Tunable Inference via Dual-level Transferable Prompt Learning [22.04285529067442]
Style Extraction and Tunable Inference via Dual-level Transferable Prompt Learning is proposed.
SETTP learns source style-level prompts containing fundamental style characteristics from high-resource style transfer.
Experiments show SETTP requires only 1/20th of the data volume to achieve performance comparable to state-of-the-art methods.
arXiv Detail & Related papers (2024-07-22T11:34:48Z) - A Unified Arbitrary Style Transfer Framework via Adaptive Contrastive
Learning [84.8813842101747]
Unified Contrastive Arbitrary Style Transfer (UCAST) is a novel style representation learning and transfer framework.
We present an adaptive contrastive learning scheme for style transfer by introducing an input-dependent temperature.
Our framework consists of three key components, i.e., a parallel contrastive learning scheme for style representation and style transfer, a domain enhancement module for effective learning of style distribution, and a generative network for style transfer.
arXiv Detail & Related papers (2023-03-09T04:35:00Z) - Learning from Bootstrapping and Stepwise Reinforcement Reward: A
Semi-Supervised Framework for Text Style Transfer [30.622772801446132]
We propose a semi-supervised framework for text style transfer.
First, the learning process is bootstrapped with supervision guided by automatically constructed pseudo-parallel pairs.
Then the model learns from unlabeled data via reinforcement rewards.
arXiv Detail & Related papers (2022-05-19T05:18:06Z) - Gradient-guided Unsupervised Text Style Transfer via Contrastive
Learning [6.799826701166569]
We propose a gradient-guided model through a contrastive paradigm for text style transfer, to explicitly gather similar semantic sentences.
Experiments on two datasets show the effectiveness of our proposed approach, as compared to the state-of-the-arts.
arXiv Detail & Related papers (2022-01-23T12:45:00Z) - Contrastive Instruction-Trajectory Learning for Vision-Language
Navigation [66.16980504844233]
A vision-language navigation (VLN) task requires an agent to reach a target with the guidance of natural language instruction.
Previous works fail to discriminate the similarities and discrepancies across instruction-trajectory pairs and ignore the temporal continuity of sub-instructions.
We propose a Contrastive Instruction-Trajectory Learning framework that explores invariance across similar data samples and variance across different ones to learn distinctive representations for robust navigation.
arXiv Detail & Related papers (2021-12-08T06:32:52Z) - Visual Transformer for Task-aware Active Learning [49.903358393660724]
We present a novel pipeline for pool-based Active Learning.
Our method exploits accessible unlabelled examples during training to estimate their co-relation with the labelled examples.
Visual Transformer models non-local visual concept dependency between labelled and unlabelled examples.
arXiv Detail & Related papers (2021-06-07T17:13:59Z) - Aligning Pretraining for Detection via Object-Level Contrastive Learning [57.845286545603415]
Image-level contrastive representation learning has proven to be highly effective as a generic model for transfer learning.
We argue that this could be sub-optimal and thus advocate a design principle which encourages alignment between the self-supervised pretext task and the downstream task.
Our method, called Selective Object COntrastive learning (SoCo), achieves state-of-the-art results for transfer performance on COCO detection.
arXiv Detail & Related papers (2021-06-04T17:59:52Z) - On Learning Text Style Transfer with Direct Rewards [101.97136885111037]
Lack of parallel corpora makes it impossible to directly train supervised models for the text style transfer task.
We leverage semantic similarity metrics originally used for fine-tuning neural machine translation models.
Our model provides significant gains in both automatic and human evaluation over strong baselines.
arXiv Detail & Related papers (2020-10-24T04:30:02Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.