Learning to Generate Equitable Text in Dialogue from Biased Training Data

• URL: http://arxiv.org/abs/2307.04303v1
• Date: Mon, 10 Jul 2023 01:44:13 GMT
• Title: Learning to Generate Equitable Text in Dialogue from Biased Training Data
• Authors: Anthony Sicilia and Malihe Alikhani
• Abstract summary: A dialogue system's decision-making process and generated responses are crucial for user engagement, satisfaction, and task achievement. We use theories of computational learning to study this problem.
• Score: 15.102346715690755
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The ingrained principles of fairness in a dialogue system's decision-making process and generated responses are crucial for user engagement, satisfaction, and task achievement. Absence of equitable and inclusive principles can hinder the formation of common ground, which in turn negatively impacts the overall performance of the system. For example, misusing pronouns in a user interaction may cause ambiguity about the intended subject. Yet, there is no comprehensive study of equitable text generation in dialogue. Aptly, in this work, we use theories of computational learning to study this problem. We provide formal definitions of equity in text generation, and further, prove formal connections between learning human-likeness and learning equity: algorithms for improving equity ultimately reduce to algorithms for improving human-likeness (on augmented data). With this insight, we also formulate reasonable conditions under which text generation algorithms can learn to generate equitable text without any modifications to the biased training data on which they learn. To exemplify our theory in practice, we look at a group of algorithms for the GuessWhat?! visual dialogue game and, using this example, test our theory empirically. Our theory accurately predicts relative-performance of multiple algorithms in generating equitable text as measured by both human and automated evaluation.

Related papers

• A Human-Centered Approach for Improving Supervised Learning [0.44378250612683995]
  This paper shows how we can strike a balance between performance, time, and resource constraints. Another goal of this research is to make Ensembles more explainable and intelligible using the Human-Centered approach.
  arXiv  Detail & Related papers  (2024-10-14T10:27:14Z)
• LLM Critics Help Catch Bugs in Mathematics: Towards a Better Mathematical Verifier with Natural Language Feedback [71.95402654982095]
  We propose Math-Minos, a natural language feedback-enhanced verifier. Our experiments reveal that a small set of natural language feedback can significantly boost the performance of the verifier.
  arXiv  Detail & Related papers  (2024-06-20T06:42:27Z)
• Improving Language Models Meaning Understanding and Consistency by Learning Conceptual Roles from Dictionary [65.268245109828]
  Non-human-like behaviour of contemporary pre-trained language models (PLMs) is a leading cause undermining their trustworthiness. A striking phenomenon is the generation of inconsistent predictions, which produces contradictory results. We propose a practical approach that alleviates the inconsistent behaviour issue by improving PLM awareness.
  arXiv  Detail & Related papers  (2023-10-24T06:15:15Z)
• Human Inspired Progressive Alignment and Comparative Learning for Grounded Word Acquisition [6.47452771256903]
  We take inspiration from how human babies acquire their first language, and developed a computational process for word acquisition through comparative learning. Motivated by cognitive findings, we generated a small dataset that enables the computation models to compare the similarities and differences of various attributes. We frame the acquisition of words as not only the information filtration process, but also as representation-symbol mapping.
  arXiv  Detail & Related papers  (2023-07-05T19:38:04Z)
• Algorithmic failure as a humanities methodology: machine learning's mispredictions identify rich cases for qualitative analysis [0.0]
  I trained a simple machine learning algorithm to predict whether or not an action was active or passive using only information about fictional characters. The results thus support Munk et al.'s theory that failed predictions can be productively used to identify rich cases for qualitative analysis. Further research is needed to develop an understanding of what kinds of data the method is useful for and which kinds of machine learning are most generative.
  arXiv  Detail & Related papers  (2023-05-19T13:24:32Z)
• LEATHER: A Framework for Learning to Generate Human-like Text in Dialogue [15.102346715690755]
  We propose a new theoretical framework for learning to generate text in dialogue. Compared to existing theories of learning, our framework allows for analysis of the multi-faceted goals inherent to text-generation.
  arXiv  Detail & Related papers  (2022-10-14T13:05:11Z)
• The Whole Truth and Nothing But the Truth: Faithful and Controllable Dialogue Response Generation with Dataflow Transduction and Constrained Decoding [65.34601470417967]
  We describe a hybrid architecture for dialogue response generation that combines the strengths of neural language modeling and rule-based generation. Our experiments show that this system outperforms both rule-based and learned approaches in human evaluations of fluency, relevance, and truthfulness.
  arXiv  Detail & Related papers  (2022-09-16T09:00:49Z)
• Human-Algorithm Collaboration: Achieving Complementarity and Avoiding Unfairness [92.26039686430204]
  We show that even in carefully-designed systems, complementary performance can be elusive. First, we provide a theoretical framework for modeling simple human-algorithm systems. Next, we use this model to prove conditions where complementarity is impossible.
  arXiv  Detail & Related papers  (2022-02-17T18:44:41Z)
• Can Active Learning Preemptively Mitigate Fairness Issues? [66.84854430781097]
  dataset bias is one of the prevailing causes of unfairness in machine learning. We study whether models trained with uncertainty-based ALs are fairer in their decisions with respect to a protected class. We also explore the interaction of algorithmic fairness methods such as gradient reversal (GRAD) and BALD.
  arXiv  Detail & Related papers  (2021-04-14T14:20:22Z)
• Representative & Fair Synthetic Data [68.8204255655161]
  We present a framework to incorporate fairness constraints into the self-supervised learning process. We generate a representative as well as fair version of the UCI Adult census data set. We consider representative & fair synthetic data a promising future building block to teach algorithms not on historic worlds, but rather on the worlds that we strive to live in.
  arXiv  Detail & Related papers  (2021-04-07T09:19:46Z)
• Mitigating Gender Bias in Machine Learning Data Sets [5.075506385456811]
  Gender bias has been identified in the context of employment advertising and recruitment tools. This paper proposes a framework for the identification of gender bias in training data for machine learning.
  arXiv  Detail & Related papers  (2020-05-14T12:06:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.