RexUniNLU: Recursive Method with Explicit Schema Instructor for Universal NLU
- URL: http://arxiv.org/abs/2409.05275v1
- Date: Mon, 9 Sep 2024 01:59:29 GMT
- Title: RexUniNLU: Recursive Method with Explicit Schema Instructor for Universal NLU
- Authors: Chengyuan Liu, Shihang Wang, Fubang Zhao, Kun Kuang, Yangyang Kang, Weiming Lu, Changlong Sun, Fei Wu,
- Abstract summary: There is no existing encoder-based model that can unify and CLS tasks from this perspective.
We have proposed a Recursive Method with Explicit Instructor for Universal NLU.
We introduce RexUniNLU, an universal NLU solution that employs explicit schema constraints for IE and CLS.
- Score: 45.85420719025781
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Information Extraction (IE) and Text Classification (CLS) serve as the fundamental pillars of NLU, with both disciplines relying on analyzing input sequences to categorize outputs into pre-established schemas. However, there is no existing encoder-based model that can unify IE and CLS tasks from this perspective. To fully explore the foundation shared within NLU tasks, we have proposed a Recursive Method with Explicit Schema Instructor for Universal NLU. Specifically, we firstly redefine the true universal information extraction (UIE) with a formal formulation that covers almost all extraction schemas, including quadruples and quintuples which remain unsolved for previous UIE models. Then, we expands the formulation to all CLS and multi-modal NLU tasks. Based on that, we introduce RexUniNLU, an universal NLU solution that employs explicit schema constraints for IE and CLS, which encompasses all IE and CLS tasks and prevent incorrect connections between schema and input sequence. To avoid interference between different schemas, we reset the position ids and attention mask matrices. Extensive experiments are conducted on IE, CLS in both English and Chinese, and multi-modality, revealing the effectiveness and superiority. Our codes are publicly released.
Related papers
- BioMistral-NLU: Towards More Generalizable Medical Language Understanding through Instruction Tuning [19.027921909970832]
We develop BioMistral-NLU, a generalizable medical NLU model, through fine-tuning BioMistral on MNLU-Instruct.
Our experiments show that our BioMistral-NLU outperforms the original BioMistral.
Our dataset-agnostic prompting strategy and instruction tuning step over diverse NLU tasks enhance LLMs' generalizability across diverse medical NLU tasks.
arXiv Detail & Related papers (2024-10-24T17:53:53Z) - RUIE: Retrieval-based Unified Information Extraction using Large Language Model [6.788855739199981]
Unified information extraction aims to complete all information extraction tasks using a single model or framework.
We propose RUIE (Retrieval-based Unified Information Extraction), a framework that leverages in-context learning to enable rapid generalization.
Experimental results on 8 held-out datasets demonstrate RUIE's effectiveness in generalizing to unseen tasks.
arXiv Detail & Related papers (2024-09-18T03:20:04Z) - Large Language Models are Interpretable Learners [53.56735770834617]
In this paper, we show a combination of Large Language Models (LLMs) and symbolic programs can bridge the gap between expressiveness and interpretability.
The pretrained LLM with natural language prompts provides a massive set of interpretable modules that can transform raw input into natural language concepts.
As the knowledge learned by LSP is a combination of natural language descriptions and symbolic rules, it is easily transferable to humans (interpretable) and other LLMs.
arXiv Detail & Related papers (2024-06-25T02:18:15Z) - PIVOINE: Instruction Tuning for Open-world Information Extraction [53.98073623222221]
We consider the problem of Open-world Information Extraction (Open-world IE), which extracts comprehensive entity profiles from unstructured texts.
We develop a large language model (LLM) that is able to perform Open-world IE to extract desirable entity profiles characterized by (possibly fine-grained) natural language instructions.
In particular, we construct INSTRUCTOPENWIKI, a substantial instruction tuning dataset for Open-world IE enriched with a comprehensive corpus, extensive annotations, and diverse instructions.
arXiv Detail & Related papers (2023-05-24T08:52:08Z) - RexUIE: A Recursive Method with Explicit Schema Instructor for Universal
Information Extraction [47.89362854989252]
Universal Information Extraction is an area of interest due to the challenges posed by varying targets, heterogeneous structures, and demand-specific schemas.
Previous works have only achieved limited success by unifying a few tasks, such as Named Entity Recognition (NER) and Relation Extraction (RE)
In this paper, we redefine the authentic UIE with a formal formulation that encompasses almost all extraction schemas.
arXiv Detail & Related papers (2023-04-28T11:28:56Z) - Token-level Sequence Labeling for Spoken Language Understanding using
Compositional End-to-End Models [94.30953696090758]
We build compositional end-to-end spoken language understanding systems.
By relying on intermediate decoders trained for ASR, our end-to-end systems transform the input modality from speech to token-level representations.
Our models outperform both cascaded and direct end-to-end models on a labeling task of named entity recognition.
arXiv Detail & Related papers (2022-10-27T19:33:18Z) - NLU++: A Multi-Label, Slot-Rich, Generalisable Dataset for Natural
Language Understanding in Task-Oriented Dialogue [53.54788957697192]
NLU++ is a novel dataset for natural language understanding (NLU) in task-oriented dialogue (ToD) systems.
NLU++ is divided into two domains (BANKING and HOTELS) and brings several crucial improvements over current commonly used NLU datasets.
arXiv Detail & Related papers (2022-04-27T16:00:23Z) - Dual Learning for Semi-Supervised Natural Language Understanding [29.692288627633374]
Natural language understanding (NLU) converts sentences into structured semantic forms.
We introduce a dual task of NLU, semantic-to-sentence generation (SSG)
We propose a new framework for semi-supervised NLU with the corresponding dual model.
arXiv Detail & Related papers (2020-04-26T07:17:48Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.