Balancing Effect of Training Dataset Distribution of Multiple Styles for
Multi-Style Text Transfer
- URL: http://arxiv.org/abs/2305.15582v1
- Date: Wed, 24 May 2023 21:36:15 GMT
- Title: Balancing Effect of Training Dataset Distribution of Multiple Styles for
Multi-Style Text Transfer
- Authors: Debarati Das, David Ma, Dongyeop Kang
- Abstract summary: This paper explores the impact of training data input diversity on the quality of the generated text from the multi-style transfer model.
We construct a pseudo-parallel dataset by devisings to adjust the style distribution in the training samples.
We observe that a balanced dataset produces more effective control effects over multiple styles than an imbalanced or skewed one.
- Score: 8.305622604531074
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Text style transfer is an exciting task within the field of natural language
generation that is often plagued by the need for high-quality paired datasets.
Furthermore, training a model for multi-attribute text style transfer requires
datasets with sufficient support across all combinations of the considered
stylistic attributes, adding to the challenges of training a style transfer
model. This paper explores the impact of training data input diversity on the
quality of the generated text from the multi-style transfer model. We construct
a pseudo-parallel dataset by devising heuristics to adjust the style
distribution in the training samples. We balance our training dataset using
marginal and joint distributions to train our style transfer models. We observe
that a balanced dataset produces more effective control effects over multiple
styles than an imbalanced or skewed one. Through quantitative analysis, we
explore the impact of multiple style distributions in training data on
style-transferred output. These findings will better inform the design of
style-transfer datasets.
Related papers
- G-DIG: Towards Gradient-based Diverse and High-quality Instruction Data Selection for Machine Translation [21.506844286376275]
We propose a novel gradient-based method to automatically select high-quality and diverse instruction finetuning data for machine translation.
Our key innovation centers around analyzing how individual training examples influence the model during training.
arXiv Detail & Related papers (2024-05-21T16:38:13Z) - Diffusion-based Neural Network Weights Generation [85.6725307453325]
We propose an efficient and adaptive transfer learning scheme through dataset-conditioned pretrained weights sampling.
Specifically, we use a latent diffusion model with a variational autoencoder that can reconstruct the neural network weights.
arXiv Detail & Related papers (2024-02-28T08:34:23Z) - Towards Large-scale 3D Representation Learning with Multi-dataset Point Prompt Training [44.790636524264]
Point Prompt Training is a novel framework for multi-dataset synergistic learning in the context of 3D representation learning.
It can overcome the negative transfer associated with synergistic learning and produce generalizable representations.
It achieves state-of-the-art performance on each dataset using a single weight-shared model with supervised multi-dataset training.
arXiv Detail & Related papers (2023-08-18T17:59:57Z) - Fine-grained Text Style Transfer with Diffusion-Based Language Models [50.02698074338317]
We trained a diffusion-based model on StylePTB dataset, the standard benchmark for fine-grained text style transfers.
Our model was able to achieve state-of-the-art performance on both individual and compositional transfers.
arXiv Detail & Related papers (2023-05-31T02:51:26Z) - Unified Model Learning for Various Neural Machine Translation [63.320005222549646]
Existing machine translation (NMT) studies mainly focus on developing dataset-specific models.
We propose a versatile'' model, i.e., the Unified Model Learning for NMT (UMLNMT) that works with data from different tasks.
OurNMT results in substantial improvements over dataset-specific models with significantly reduced model deployment costs.
arXiv Detail & Related papers (2023-05-04T12:21:52Z) - Tackling Data Bias in Painting Classification with Style Transfer [12.88476464580968]
We propose a system to handle data bias in small paintings datasets like the Kaokore dataset.
Our system consists of two stages which are style transfer and classification.
arXiv Detail & Related papers (2023-01-06T14:33:53Z) - Curriculum-Based Self-Training Makes Better Few-Shot Learners for
Data-to-Text Generation [56.98033565736974]
We propose Curriculum-Based Self-Training (CBST) to leverage unlabeled data in a rearranged order determined by the difficulty of text generation.
Our method can outperform fine-tuning and task-adaptive pre-training methods, and achieve state-of-the-art performance in the few-shot setting of data-to-text generation.
arXiv Detail & Related papers (2022-06-06T16:11:58Z) - Non-Parallel Text Style Transfer with Self-Parallel Supervision [19.441780035577352]
We propose LaMer, a novel text style transfer framework based on large-scale language models.
LaMer first mines the roughly parallel expressions in the non-parallel datasets with scene graphs, and then employs MLE training, followed by imitation learning refinement, to leverage the intrinsic parallelism within the data.
On two benchmark tasks (sentiment & formality transfer) and a newly proposed challenging task (political stance transfer), our model achieves qualitative advances in transfer accuracy, content preservation, and fluency.
arXiv Detail & Related papers (2022-04-18T01:38:35Z) - Generating More Pertinent Captions by Leveraging Semantics and Style on
Multi-Source Datasets [56.018551958004814]
This paper addresses the task of generating fluent descriptions by training on a non-uniform combination of data sources.
Large-scale datasets with noisy image-text pairs provide a sub-optimal source of supervision.
We propose to leverage and separate semantics and descriptive style through the incorporation of a style token and keywords extracted through a retrieval component.
arXiv Detail & Related papers (2021-11-24T19:00:05Z) - Improving Zero and Few-Shot Abstractive Summarization with Intermediate
Fine-tuning and Data Augmentation [101.26235068460551]
Models pretrained with self-supervised objectives on large text corpora achieve state-of-the-art performance on English text summarization tasks.
Models are typically fine-tuned on hundreds of thousands of data points, an infeasible requirement when applying summarization to new, niche domains.
We introduce a novel and generalizable method, called WikiTransfer, for fine-tuning pretrained models for summarization in an unsupervised, dataset-specific manner.
arXiv Detail & Related papers (2020-10-24T08:36:49Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.