PerturbScore: Connecting Discrete and Continuous Perturbations in NLP
- URL: http://arxiv.org/abs/2310.08889v1
- Date: Fri, 13 Oct 2023 06:50:15 GMT
- Title: PerturbScore: Connecting Discrete and Continuous Perturbations in NLP
- Authors: Linyang Li, Ke Ren, Yunfan Shao, Pengyu Wang, Xipeng Qiu
- Abstract summary: In this paper, we aim to connect discrete perturbations with continuous perturbations.
We design a regression task as a PerturbScore to learn the correlation automatically.
We find that we can build a connection between discrete and continuous perturbations and use the proposed PerturbScore to learn such correlation.
- Score: 64.28423650146877
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: With the rapid development of neural network applications in NLP, model
robustness problem is gaining more attention. Different from computer vision,
the discrete nature of texts makes it more challenging to explore robustness in
NLP. Therefore, in this paper, we aim to connect discrete perturbations with
continuous perturbations, therefore we can use such connections as a bridge to
help understand discrete perturbations in NLP models. Specifically, we first
explore how to connect and measure the correlation between discrete
perturbations and continuous perturbations. Then we design a regression task as
a PerturbScore to learn the correlation automatically. Through experimental
results, we find that we can build a connection between discrete and continuous
perturbations and use the proposed PerturbScore to learn such correlation,
surpassing previous methods used in discrete perturbation measuring. Further,
the proposed PerturbScore can be well generalized to different datasets,
perturbation methods, indicating that we can use it as a powerful tool to study
model robustness in NLP.
Related papers
- Cross-modulated Attention Transformer for RGBT Tracking [35.1700920590541]
We propose a novel approach called Cross-modulated Attention Transformer (CAFormer) for RGBT tracking.
In particular, we first independently generate correlation maps for each modality and feed them into the designed Correlation Modulated Enhancement module.
Experiments on five public RGBT tracking benchmarks show the outstanding performance of the proposed CAFormer against state-of-the-art methods.
arXiv Detail & Related papers (2024-08-05T03:54:40Z) - Causal Reasoning in the Presence of Latent Confounders via Neural ADMG
Learning [8.649109147825985]
Latent confounding has been a long-standing obstacle for causal reasoning from observational data.
We propose a novel neural causal model based on autoregressive flows for ADMG learning.
arXiv Detail & Related papers (2023-03-22T16:45:54Z) - Integrating Random Effects in Deep Neural Networks [4.860671253873579]
We propose to use the mixed models framework to handle correlated data in deep neural networks.
By treating the effects underlying the correlation structure as random effects, mixed models are able to avoid overfitted parameter estimates.
Our approach which we call LMMNN is demonstrated to improve performance over natural competitors in various correlation scenarios.
arXiv Detail & Related papers (2022-06-07T14:02:24Z) - Generalization of Neural Combinatorial Solvers Through the Lens of
Adversarial Robustness [68.97830259849086]
Most datasets only capture a simpler subproblem and likely suffer from spurious features.
We study adversarial robustness - a local generalization property - to reveal hard, model-specific instances and spurious features.
Unlike in other applications, where perturbation models are designed around subjective notions of imperceptibility, our perturbation models are efficient and sound.
Surprisingly, with such perturbations, a sufficiently expressive neural solver does not suffer from the limitations of the accuracy-robustness trade-off common in supervised learning.
arXiv Detail & Related papers (2021-10-21T07:28:11Z) - Identifying and Mitigating Spurious Correlations for Improving
Robustness in NLP Models [19.21465581259624]
Many problems can be attributed to models exploiting spurious correlations, or shortcuts between the training data and the task labels.
In this paper, we aim to automatically identify such spurious correlations in NLP models at scale.
We show that our proposed method can effectively and efficiently identify a scalable set of "shortcuts", and mitigating these leads to more robust models in multiple applications.
arXiv Detail & Related papers (2021-10-14T21:40:03Z) - Contrastive Self-supervised Sequential Recommendation with Robust
Augmentation [101.25762166231904]
Sequential Recommendationdescribes a set of techniques to model dynamic user behavior in order to predict future interactions in sequential user data.
Old and new issues remain, including data-sparsity and noisy data.
We propose Contrastive Self-Supervised Learning for sequential Recommendation (CoSeRec)
arXiv Detail & Related papers (2021-08-14T07:15:25Z) - Non-Singular Adversarial Robustness of Neural Networks [58.731070632586594]
Adrial robustness has become an emerging challenge for neural network owing to its over-sensitivity to small input perturbations.
We formalize the notion of non-singular adversarial robustness for neural networks through the lens of joint perturbations to data inputs as well as model weights.
arXiv Detail & Related papers (2021-02-23T20:59:30Z) - Online neural connectivity estimation with ensemble stimulation [5.156484100374058]
We propose a method based on noisy group testing that drastically increases the efficiency of this process in sparse networks.
We show that it is possible to recover binarized network connectivity with a number of tests that grows only logarithmically with population size.
We also demonstrate the feasibility of inferring connectivity for networks of up to tens of thousands of neurons online.
arXiv Detail & Related papers (2020-07-27T23:47:03Z) - Belief Propagation Reloaded: Learning BP-Layers for Labeling Problems [83.98774574197613]
We take one of the simplest inference methods, a truncated max-product Belief propagation, and add what is necessary to make it a proper component of a deep learning model.
This BP-Layer can be used as the final or an intermediate block in convolutional neural networks (CNNs)
The model is applicable to a range of dense prediction problems, is well-trainable and provides parameter-efficient and robust solutions in stereo, optical flow and semantic segmentation.
arXiv Detail & Related papers (2020-03-13T13:11:35Z) - Estimating the Effects of Continuous-valued Interventions using
Generative Adversarial Networks [103.14809802212535]
We build on the generative adversarial networks (GANs) framework to address the problem of estimating the effect of continuous-valued interventions.
Our model, SCIGAN, is flexible and capable of simultaneously estimating counterfactual outcomes for several different continuous interventions.
To address the challenges presented by shifting to continuous interventions, we propose a novel architecture for our discriminator.
arXiv Detail & Related papers (2020-02-27T18:46:21Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.