Retrieve-and-Fill for Scenario-based Task-Oriented Semantic Parsing
- URL: http://arxiv.org/abs/2202.00901v1
- Date: Wed, 2 Feb 2022 08:00:21 GMT
- Title: Retrieve-and-Fill for Scenario-based Task-Oriented Semantic Parsing
- Authors: Akshat Shrivastava, Shrey Desai, Anchit Gupta, Ali Elkahky, Aleksandr
Livshits, Alexander Zotov, Ahmed Aly
- Abstract summary: We introduce scenario-based semantic parsing: a variant of the original task which first requires disambiguating an utterance's "scenario"
This formulation enables us to isolate coarse-grained and fine-grained aspects of the task, each of which we solve with off-the-shelf neural modules.
Our model is modular, differentiable, interpretable, and allows us to garner extra supervision from scenarios.
- Score: 110.4684789199555
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Task-oriented semantic parsing models have achieved strong results in recent
years, but unfortunately do not strike an appealing balance between model size,
runtime latency, and cross-domain generalizability. We tackle this problem by
introducing scenario-based semantic parsing: a variant of the original task
which first requires disambiguating an utterance's "scenario" (an intent-slot
template with variable leaf spans) before generating its frame, complete with
ontology and utterance tokens. This formulation enables us to isolate
coarse-grained and fine-grained aspects of the task, each of which we solve
with off-the-shelf neural modules, also optimizing for the axes outlined above.
Concretely, we create a Retrieve-and-Fill (RAF) architecture comprised of (1) a
retrieval module which ranks the best scenario given an utterance and (2) a
filling module which imputes spans into the scenario to create the frame. Our
model is modular, differentiable, interpretable, and allows us to garner extra
supervision from scenarios. RAF achieves strong results in high-resource,
low-resource, and multilingual settings, outperforming recent approaches by
wide margins despite, using base pre-trained encoders, small sequence lengths,
and parallel decoding.
Related papers
- Transformer Architecture for NetsDB [0.0]
We create an end-to-end implementation of a transformer for deep learning model serving in NetsDB.
We load out weights from our model for distributed processing, deployment, and efficient inferencing.
arXiv Detail & Related papers (2024-05-08T04:38:36Z) - Pre-Trained Model Recommendation for Downstream Fine-tuning [22.343011779348682]
Model selection aims to rank off-the-shelf pre-trained models and select the most suitable one for the new target task.
Existing model selection techniques are often constrained in their scope and tend to overlook the nuanced relationships between models and tasks.
We present a pragmatic framework textbfFennec, delving into a diverse, large-scale model repository.
arXiv Detail & Related papers (2024-03-11T02:24:32Z) - Generative Model for Models: Rapid DNN Customization for Diverse Tasks
and Resource Constraints [28.983470365172057]
NN-Factory is a one-for-all framework to generate customized lightweight models for diverse edge scenarios.
The main components of NN-Factory include a modular supernet with pretrained modules that can be conditionally activated to accomplish different tasks.
NN-Factory is able to generate high-quality task- and resource-specific models within few seconds, faster than conventional model customization approaches by orders of magnitude.
arXiv Detail & Related papers (2023-08-29T03:28:14Z) - Deep Model Reassembly [60.6531819328247]
We explore a novel knowledge-transfer task, termed as Deep Model Reassembly (DeRy)
The goal of DeRy is to first dissect each model into distinctive building blocks, and then selectively reassemble the derived blocks to produce customized networks.
We demonstrate that on ImageNet, the best reassemble model achieves 78.6% top-1 accuracy without fine-tuning.
arXiv Detail & Related papers (2022-10-24T10:16:13Z) - Guiding the PLMs with Semantic Anchors as Intermediate Supervision:
Towards Interpretable Semantic Parsing [57.11806632758607]
We propose to incorporate the current pretrained language models with a hierarchical decoder network.
By taking the first-principle structures as the semantic anchors, we propose two novel intermediate supervision tasks.
We conduct intensive experiments on several semantic parsing benchmarks and demonstrate that our approach can consistently outperform the baselines.
arXiv Detail & Related papers (2022-10-04T07:27:29Z) - X2Parser: Cross-Lingual and Cross-Domain Framework for Task-Oriented
Compositional Semantic Parsing [51.81533991497547]
Task-oriented compositional semantic parsing (TCSP) handles complex nested user queries.
We present X2 compared a transferable Cross-lingual and Cross-domain for TCSP.
We propose to predict flattened intents and slots representations separately and cast both prediction tasks into sequence labeling problems.
arXiv Detail & Related papers (2021-06-07T16:40:05Z) - Unsupervised Paraphrasing with Pretrained Language Models [85.03373221588707]
We propose a training pipeline that enables pre-trained language models to generate high-quality paraphrases in an unsupervised setting.
Our recipe consists of task-adaptation, self-supervision, and a novel decoding algorithm named Dynamic Blocking.
We show with automatic and human evaluations that our approach achieves state-of-the-art performance on both the Quora Question Pair and the ParaNMT datasets.
arXiv Detail & Related papers (2020-10-24T11:55:28Z) - Set Based Stochastic Subsampling [85.5331107565578]
We propose a set-based two-stage end-to-end neural subsampling model that is jointly optimized with an textitarbitrary downstream task network.
We show that it outperforms the relevant baselines under low subsampling rates on a variety of tasks including image classification, image reconstruction, function reconstruction and few-shot classification.
arXiv Detail & Related papers (2020-06-25T07:36:47Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.