Related papers: Few-Shot NLU with Vector Projection Distance and Abstract Triangular CRF

Few-Shot NLU with Vector Projection Distance and Abstract Triangular CRF

URL: http://arxiv.org/abs/2112.04999v1
Date: Thu, 9 Dec 2021 15:46:15 GMT
Title: Few-Shot NLU with Vector Projection Distance and Abstract Triangular CRF
Authors: Su Zhu, Lu Chen, Ruisheng Cao, Zhi Chen, Qingliang Miao, and Kai Yu
Abstract summary: Data sparsity problem is a key challenge of Natural Language Understanding (NLU) We propose to improve prototypical networks with vector projection distance and triangular Conditional Random Field (CRF) for the few-shot NLU. Our approach can achieve a new state-of-the-art on two few-shot NLU benchmarks (Few-Joint and SNIPS) in Chinese and English without fine-tuning on target domains.
Score: 30.982301053976023
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Abstract: Data sparsity problem is a key challenge of Natural Language Understanding (NLU), especially for a new target domain. By training an NLU model in source domains and applying the model to an arbitrary target domain directly (even without fine-tuning), few-shot NLU becomes crucial to mitigate the data scarcity issue. In this paper, we propose to improve prototypical networks with vector projection distance and abstract triangular Conditional Random Field (CRF) for the few-shot NLU. The vector projection distance exploits projections of contextual word embeddings on label vectors as word-label similarities, which is equivalent to a normalized linear model. The abstract triangular CRF learns domain-agnostic label transitions for joint intent classification and slot filling tasks. Extensive experiments demonstrate that our proposed methods can significantly surpass strong baselines. Specifically, our approach can achieve a new state-of-the-art on two few-shot NLU benchmarks (Few-Joint and SNIPS) in Chinese and English without fine-tuning on target domains.

Related papers

Exploiting Low-confidence Pseudo-labels for Source-free Object Detection [54.98300313452037]
Source-free object detection (SFOD) aims to adapt a source-trained detector to an unlabeled target domain without access to the labeled source data. Current SFOD methods utilize a threshold-based pseudo-label approach in the adaptation phase. We propose a new approach to take full advantage of pseudo-labels by introducing high and low confidence thresholds.
arXiv Detail & Related papers (2023-10-19T12:59:55Z)
Unsupervised Adaptation of Polyp Segmentation Models via Coarse-to-Fine Self-Supervision [16.027843524655516]
We study a practical problem of Source-Free Domain Adaptation (SFDA), which eliminates the reliance on annotated source data. Current SFDA methods focus on extracting domain knowledge from the source-trained model but neglects the intrinsic structure of the target domain. We propose a new SFDA framework, called Region-to-Pixel Adaptation Network(RPANet), which learns the region-level and pixel-level discriminative representations through coarse-to-fine self-supervision.
arXiv Detail & Related papers (2023-08-13T02:37:08Z)
Neural Vector Fields: Implicit Representation by Explicit Learning [63.337294707047036]
We propose a novel 3D representation method, Neural Vector Fields (NVF) It not only adopts the explicit learning process to manipulate meshes directly, but also the implicit representation of unsigned distance functions (UDFs) Our method first predicts displacement queries towards the surface and models shapes as text reconstructions.
arXiv Detail & Related papers (2023-03-08T02:36:09Z)
Robust Target Training for Multi-Source Domain Adaptation [110.77704026569499]
We propose a novel Bi-level Optimization based Robust Target Training (BORT$2$) method for MSDA. Our proposed method achieves the state of the art performance on three MSDA benchmarks, including the large-scale DomainNet dataset.
arXiv Detail & Related papers (2022-10-04T15:20:01Z)
PC-RGNN: Point Cloud Completion and Graph Neural Network for 3D Object Detection [57.49788100647103]
LiDAR-based 3D object detection is an important task for autonomous driving. Current approaches suffer from sparse and partial point clouds of distant and occluded objects. In this paper, we propose a novel two-stage approach, namely PC-RGNN, dealing with such challenges by two specific solutions.
arXiv Detail & Related papers (2020-12-18T18:06:43Z)
Complete & Label: A Domain Adaptation Approach to Semantic Segmentation of LiDAR Point Clouds [49.47017280475232]
We study an unsupervised domain adaptation problem for the semantic labeling of 3D point clouds. We take a Complete and Label approach to recover the underlying surfaces before passing them to a segmentation network. The recovered 3D surfaces serve as a canonical domain, from which semantic labels can transfer across different LiDAR sensors.
arXiv Detail & Related papers (2020-07-16T17:42:05Z)
Bridging the Theoretical Bound and Deep Algorithms for Open Set Domain Adaptation [40.95099721257058]
In the unsupervised open set domain adaptation (UOSDA), the target domain contains unknown classes that are not observed in the source domain. We propose a new upper bound of target-domain risk for UOSDA, which includes four terms: source-domain risk, $epsilon$-open set difference ($Delta_epsilon$), a distributional discrepancy between domains, and a constant. Specifically, source-domain risk and $Delta_epsilon$ are minimized by gradient descent, and the distributional discrepancy is minimized via a novel open-set conditional adversarial training strategy.
arXiv Detail & Related papers (2020-06-23T14:01:06Z)
L-Vector: Neural Label Embedding for Domain Adaptation [62.112885747045766]
We propose a novel neural label embedding (NLE) scheme for the domain adaptation of a deep neural network (DNN) acoustic model with unpaired data samples. NLE achieves up to 14.1% relative word error rate reduction over direct re-training with one-hot labels.
arXiv Detail & Related papers (2020-04-25T06:40:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.