A Causal Lens for Controllable Text Generation
- URL: http://arxiv.org/abs/2201.09119v1
- Date: Sat, 22 Jan 2022 19:31:43 GMT
- Title: A Causal Lens for Controllable Text Generation
- Authors: Zhiting Hu, Li Erran Li
- Abstract summary: This paper proposes to formulate controllable text generation from a principled causal perspective.
A direct advantage of the causal formulation is the use of rich causality tools to mitigate generation biases and improve control.
Experiments show significant superiority of the causal approach over previous conditional models for improved control accuracy and reduced bias.
- Score: 36.26478600135344
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Controllable text generation concerns two fundamental tasks of wide
applications, namely generating text of given attributes (i.e.,
attribute-conditional generation), and minimally editing existing text to
possess desired attributes (i.e., text attribute transfer). Extensive prior
work has largely studied the two problems separately, and developed different
conditional models which, however, are prone to producing biased text (e.g.,
various gender stereotypes). This paper proposes to formulate controllable text
generation from a principled causal perspective which models the two tasks with
a unified framework. A direct advantage of the causal formulation is the use of
rich causality tools to mitigate generation biases and improve control. We
treat the two tasks as interventional and counterfactual causal inference based
on a structural causal model, respectively. We then apply the framework to the
challenging practical setting where confounding factors (that induce spurious
correlations) are observable only on a small fraction of data. Experiments show
significant superiority of the causal approach over previous conditional models
for improved control accuracy and reduced bias.
Related papers
- How Well Do Text Embedding Models Understand Syntax? [50.440590035493074]
The ability of text embedding models to generalize across a wide range of syntactic contexts remains under-explored.
Our findings reveal that existing text embedding models have not sufficiently addressed these syntactic understanding challenges.
We propose strategies to augment the generalization ability of text embedding models in diverse syntactic scenarios.
arXiv Detail & Related papers (2023-11-14T08:51:00Z) - FacTool: Factuality Detection in Generative AI -- A Tool Augmented
Framework for Multi-Task and Multi-Domain Scenarios [87.12753459582116]
A wider range of tasks now face an increasing risk of containing factual errors when handled by generative models.
We propose FacTool, a task and domain agnostic framework for detecting factual errors of texts generated by large language models.
arXiv Detail & Related papers (2023-07-25T14:20:51Z) - Advancing Precise Outline-Conditioned Text Generation with Task Duality
and Explicit Outline Control [15.881568820009797]
We introduce a novel text generation task called Precise Outline-conditioned Generation.
This task requires generating stories based on specific, sentence-level outlines.
We propose an explicit outline utilization control approach and a novel framework that leverages the task duality between summarization and generation.
arXiv Detail & Related papers (2023-05-23T18:33:52Z) - Debiasing Stance Detection Models with Counterfactual Reasoning and
Adversarial Bias Learning [15.68462203989933]
Stance detection models tend to rely on dataset bias in the text part as a shortcut.
We propose an adversarial bias learning module to model the bias more accurately.
arXiv Detail & Related papers (2022-12-20T16:20:56Z) - A Causal Framework to Quantify the Robustness of Mathematical Reasoning
with Language Models [81.15974174627785]
We study the behavior of language models in terms of robustness and sensitivity to direct interventions in the input space.
Our analysis shows that robustness does not appear to continuously improve as a function of size, but the GPT-3 Davinci models (175B) achieve a dramatic improvement in both robustness and sensitivity compared to all other GPT variants.
arXiv Detail & Related papers (2022-10-21T15:12:37Z) - FAST: Improving Controllability for Text Generation with Feedback Aware
Self-Training [25.75982440355576]
Controllable text generation systems often leverage control codes to direct various properties of the output like style and length.
Inspired by recent work on causal inference for NLP, this paper reveals a previously overlooked flaw in these control code-based conditional text generation algorithms.
We propose two simple techniques to reduce these correlations in training sets.
arXiv Detail & Related papers (2022-10-06T19:00:51Z) - Text Generation with Text-Editing Models [78.03750739936956]
This tutorial provides a comprehensive overview of text-editing models and current state-of-the-art approaches.
We discuss challenges related to productionization and how these models can be used to mitigate hallucination and bias.
arXiv Detail & Related papers (2022-06-14T17:58:17Z) - Textual Explanations and Critiques in Recommendation Systems [8.406549970145846]
dissertation focuses on two fundamental challenges of addressing this need.
The first involves explanation generation in a scalable and data-driven manner.
The second challenge consists in making explanations actionable, and we refer to it as critiquing.
arXiv Detail & Related papers (2022-05-15T11:59:23Z) - Guided Generation of Cause and Effect [52.44584102429394]
We present a conditional text generation framework that posits sentential expressions of possible causes and effects.
This framework depends on two novel resources: a large-scale collection of English sentences expressing causal patterns CausalBank and a refinement over previous work on constructing large lexical causal knowledge graphs Cause Effect Graph.
arXiv Detail & Related papers (2021-07-21T02:32:47Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.