Unified Semantic Typing with Meaningful Label Inference

• URL: http://arxiv.org/abs/2205.01826v1
• Date: Wed, 4 May 2022 00:28:17 GMT
• Title: Unified Semantic Typing with Meaningful Label Inference
• Authors: James Y. Huang, Bangzheng Li, Jiashu Xu, Muhao Chen
• Abstract summary: We present UniST, a unified framework for semantic typing. UniST captures label semantics by projecting both inputs and labels into a joint semantic embedding space. Our experiments demonstrate that UniST achieves strong performance across three semantic typing tasks.
• Score: 19.308286513235934
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Semantic typing aims at classifying tokens or spans of interest in a textual context into semantic categories such as relations, entity types, and event types. The inferred labels of semantic categories meaningfully interpret how machines understand components of text. In this paper, we present UniST, a unified framework for semantic typing that captures label semantics by projecting both inputs and labels into a joint semantic embedding space. To formulate different lexical and relational semantic typing tasks as a unified task, we incorporate task descriptions to be jointly encoded with the input, allowing UniST to be adapted to different tasks without introducing task-specific model components. UniST optimizes a margin ranking loss such that the semantic relatedness of the input and labels is reflected from their embedding similarity. Our experiments demonstrate that UniST achieves strong performance across three semantic typing tasks: entity typing, relation classification and event typing. Meanwhile, UniST effectively transfers semantic knowledge of labels and substantially improves generalizability on inferring rarely seen and unseen types. In addition, multiple semantic typing tasks can be jointly trained within the unified framework, leading to a single compact multi-tasking model that performs comparably to dedicated single-task models, while offering even better transferability.

Related papers

• Seed-Guided Fine-Grained Entity Typing in Science and Engineering Domains [51.02035914828596]
  We study the task of seed-guided fine-grained entity typing in science and engineering domains. We propose SEType which first enriches the weak supervision by finding more entities for each seen type from an unlabeled corpus. It then matches the enriched entities to unlabeled text to get pseudo-labeled samples and trains a textual entailment model that can make inferences for both seen and unseen types.
  arXiv  Detail & Related papers  (2024-01-23T22:36:03Z)
• Matching of Descriptive Labels to Glossary Descriptions [4.030805205247758]
  We propose a framework to leverage an existing semantic text similarity measurement (STS) and augment it using semantic label enrichment and set-based collective contextualization. We performed an experiment on two datasets derived from publicly available data sources.
  arXiv  Detail & Related papers  (2023-10-27T07:09:04Z)
• Exploring Structured Semantic Prior for Multi Label Recognition with Incomplete Labels [60.675714333081466]
  Multi-label recognition (MLR) with incomplete labels is very challenging. Recent works strive to explore the image-to-label correspondence in the vision-language model, ie, CLIP, to compensate for insufficient annotations. We advocate remedying the deficiency of label supervision for the MLR with incomplete labels by deriving a structured semantic prior.
  arXiv  Detail & Related papers  (2023-03-23T12:39:20Z)
• Prototype-based Embedding Network for Scene Graph Generation [105.97836135784794]
  Current Scene Graph Generation (SGG) methods explore contextual information to predict relationships among entity pairs. Due to the diverse visual appearance of numerous possible subject-object combinations, there is a large intra-class variation within each predicate category. Prototype-based Embedding Network (PE-Net) models entities/predicates with prototype-aligned compact and distinctive representations. PL is introduced to help PE-Net efficiently learn such entitypredicate matching, and Prototype Regularization (PR) is devised to relieve the ambiguous entity-predicate matching.
  arXiv  Detail & Related papers  (2023-03-13T13:30:59Z)
• IDEA: Interactive DoublE Attentions from Label Embedding for Text Classification [4.342189319523322]
  We propose a novel model structure via siamese BERT and interactive double attentions named IDEA to capture the information exchange of text and label names. Our proposed method outperforms the state-of-the-art methods using label texts significantly with more stable results.
  arXiv  Detail & Related papers  (2022-09-23T04:50:47Z)
• Multi-grained Label Refinement Network with Dependency Structures for Joint Intent Detection and Slot Filling [13.963083174197164]
  intent and semantic components of a utterance are dependent on the syntactic elements of a sentence. In this paper, we investigate a multi-grained label refinement network, which utilizes dependency structures and label semantic embeddings. Considering to enhance syntactic representations, we introduce the dependency structures of sentences into our model by graph attention layer.
  arXiv  Detail & Related papers  (2022-09-09T07:27:38Z)
• Improving Multi-task Generalization Ability for Neural Text Matching via Prompt Learning [54.66399120084227]
  Recent state-of-the-art neural text matching models (PLMs) are hard to generalize to different tasks. We adopt a specialization-generalization training strategy and refer to it as Match-Prompt. In specialization stage, descriptions of different matching tasks are mapped to only a few prompt tokens. In generalization stage, text matching model explores the essential matching signals by being trained on diverse multiple matching tasks.
  arXiv  Detail & Related papers  (2022-04-06T11:01:08Z)
• Dynamic Semantic Matching and Aggregation Network for Few-shot Intent Detection [69.2370349274216]
  Few-shot Intent Detection is challenging due to the scarcity of available annotated utterances. Semantic components are distilled from utterances via multi-head self-attention. Our method provides a comprehensive matching measure to enhance representations of both labeled and unlabeled instances.
  arXiv  Detail & Related papers  (2020-10-06T05:16:38Z)
• Few-shot Slot Tagging with Collapsed Dependency Transfer and Label-enhanced Task-adaptive Projection Network [61.94394163309688]
  We propose a Label-enhanced Task-Adaptive Projection Network (L-TapNet) based on the state-of-the-art few-shot classification model -- TapNet. Experimental results show that our model significantly outperforms the strongest few-shot learning baseline by 14.64 F1 scores in the one-shot setting.
  arXiv  Detail & Related papers  (2020-06-10T07:50:44Z)
• E2EET: From Pipeline to End-to-end Entity Typing via Transformer-Based Embeddings [7.431445082017672]
  We propose a new type of entity typing called Entity Typing (ET) ET involves labelling each entity mention with one or more class labels. We propose to incorporate context using transformer-based embeddings for a mention-level model, and an end-to-end model using a Bi-GRU. An extensive ablative study demonstrates the effectiveness of contextualised embeddings for mention-level models and the competitiveness of our end-to-end model for entity typing.
  arXiv  Detail & Related papers  (2020-03-23T06:46:28Z)
• Description Based Text Classification with Reinforcement Learning [34.18824470728299]
  We propose a new framework for text classification, in which each category label is associated with a category description. We observe significant performance boosts over strong baselines on a wide range of text classification tasks.
  arXiv  Detail & Related papers  (2020-02-08T02:14:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.