MGR: Multi-generator Based Rationalization
- URL: http://arxiv.org/abs/2305.04492v8
- Date: Sun, 23 Jul 2023 08:54:43 GMT
- Title: MGR: Multi-generator Based Rationalization
- Authors: Wei Liu, Haozhao Wang, Jun Wang, Ruixuan Li, Xinyang Li, Yuankai
Zhang, Yang Qiu
- Abstract summary: Rationalization is to employ a generator and a predictor to construct a self-explaining NLP model.
In this paper, we propose a simple yet effective method named MGR to simultaneously solve the two problems.
We show that MGR improves the F1 score by up to 20.9% as compared to state-of-the-art methods.
- Score: 14.745836934156427
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Rationalization is to employ a generator and a predictor to construct a
self-explaining NLP model in which the generator selects a subset of
human-intelligible pieces of the input text to the following predictor.
However, rationalization suffers from two key challenges, i.e., spurious
correlation and degeneration, where the predictor overfits the spurious or
meaningless pieces solely selected by the not-yet well-trained generator and in
turn deteriorates the generator. Although many studies have been proposed to
address the two challenges, they are usually designed separately and do not
take both of them into account. In this paper, we propose a simple yet
effective method named MGR to simultaneously solve the two problems. The key
idea of MGR is to employ multiple generators such that the occurrence stability
of real pieces is improved and more meaningful pieces are delivered to the
predictor. Empirically, we show that MGR improves the F1 score by up to 20.9%
as compared to state-of-the-art methods. Codes are available at
https://github.com/jugechengzi/Rationalization-MGR .
Related papers
- Prompt Optimization via Adversarial In-Context Learning [51.18075178593142]
adv-ICL is implemented as a two-player game between a generator and a discriminator.
The generator tries to generate realistic enough output to fool the discriminator.
We show that adv-ICL results in significant improvements over state-of-the-art prompt optimization techniques.
arXiv Detail & Related papers (2023-12-05T09:44:45Z) - Exploring Equation as a Better Intermediate Meaning Representation for
Numerical Reasoning [53.2491163874712]
We use equations as IMRs to solve the numerical reasoning task.
We present a method called Boosting Numerical Reasontextbfing by Decomposing the Generation of Equations (Bridge)
Our method improves the performance by 2.2%, 0.9%, and 1.7% on GSM8K, SVAMP, and Algebra datasets.
arXiv Detail & Related papers (2023-08-21T09:35:33Z) - Decoupled Rationalization with Asymmetric Learning Rates: A Flexible
Lipschitz Restraint [16.54547887989801]
Self-explaining rationalization model is generally constructed by a cooperative game where a generator selects the most human-intelligible pieces from the input text as rationales, followed by a predictor that makes predictions based on the selected rationales.
Such a cooperative game may incur the degeneration problem where the predictor overfits to the uninformative pieces generated by a not yet well-trained generator and in turn, leads the generator to converge to a sub-optimal model that tends to select senseless pieces.
We empirically propose a simple but effective method named DR, which can naturally and flexibly restrain the Lipschitz constant of the
arXiv Detail & Related papers (2023-05-23T02:01:13Z) - Gaussian-Bernoulli RBMs Without Tears [113.62579223055958]
We propose a novel Gibbs-Langevin sampling algorithm that outperforms existing methods like Gibbs sampling.
We propose a modified contrastive divergence (CD) algorithm so that one can generate images with GRBMs starting from noise.
arXiv Detail & Related papers (2022-10-19T06:22:55Z) - FR: Folded Rationalization with a Unified Encoder [14.899075910719189]
We propose Folded Rationalization (FR) that folds the two phases of the rationale model into one from the perspective of text semantic extraction.
We show that FR improves the F1 score by up to 10.3% as compared to state-of-the-art methods.
arXiv Detail & Related papers (2022-09-17T08:49:45Z) - Joint Generator-Ranker Learning for Natural Language Generation [99.16268050116717]
JGR is a novel joint training algorithm that integrates the generator and the ranker in a single framework.
By iteratively updating the generator and the ranker, JGR can effectively harmonize their learning and enhance their quality jointly.
arXiv Detail & Related papers (2022-06-28T12:58:30Z) - KGR^4: Retrieval, Retrospect, Refine and Rethink for Commonsense
Generation [36.78998964614422]
We propose a Knowledge-enhanced Commonsense Generation framework, termed KGR4, consisting of four stages: Retrieval, Retrospect, Refine, Rethink.
KGR4 obtains 33.56 SPICE points in the official leaderboard, outperforming the previously-reported best result by 2.49 SPICE points.
arXiv Detail & Related papers (2021-12-15T17:00:11Z) - Understanding Interlocking Dynamics of Cooperative Rationalization [90.6863969334526]
Selective rationalization explains the prediction of complex neural networks by finding a small subset of the input that is sufficient to predict the neural model output.
We reveal a major problem with such cooperative rationalization paradigm -- model interlocking.
We propose a new rationalization framework, called A2R, which introduces a third component into the architecture, a predictor driven by soft attention as opposed to selection.
arXiv Detail & Related papers (2021-10-26T17:39:18Z) - Unsupervised Controllable Generation with Self-Training [90.04287577605723]
controllable generation with GANs remains a challenging research problem.
We propose an unsupervised framework to learn a distribution of latent codes that control the generator through self-training.
Our framework exhibits better disentanglement compared to other variants such as the variational autoencoder.
arXiv Detail & Related papers (2020-07-17T21:50:35Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.