HILL: Hierarchy-aware Information Lossless Contrastive Learning for Hierarchical Text Classification

• URL: http://arxiv.org/abs/2403.17307v1
• Date: Tue, 26 Mar 2024 01:29:17 GMT
• Title: HILL: Hierarchy-aware Information Lossless Contrastive Learning for Hierarchical Text Classification
• Authors: He Zhu, Junran Wu, Ruomei Liu, Yue Hou, Ze Yuan, Shangzhe Li, Yicheng Pan, Ke Xu,
• Abstract summary: This paper investigates the feasibility of a contrastive learning scheme in which the semantic and syntactic information inherent in the input sample is adequately reserved. A structure encoder takes the document embedding as input, extracts the essential syntactic information inherent in the label hierarchy, and injects the syntactic information into the text representation. Experiments on three common datasets are conducted to verify the superiority of HILL.
• Score: 12.930158528823524
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Existing self-supervised methods in natural language processing (NLP), especially hierarchical text classification (HTC), mainly focus on self-supervised contrastive learning, extremely relying on human-designed augmentation rules to generate contrastive samples, which can potentially corrupt or distort the original information. In this paper, we tend to investigate the feasibility of a contrastive learning scheme in which the semantic and syntactic information inherent in the input sample is adequately reserved in the contrastive samples and fused during the learning process. Specifically, we propose an information lossless contrastive learning strategy for HTC, namely \textbf{H}ierarchy-aware \textbf{I}nformation \textbf{L}ossless contrastive \textbf{L}earning (HILL), which consists of a text encoder representing the input document, and a structure encoder directly generating the positive sample. The structure encoder takes the document embedding as input, extracts the essential syntactic information inherent in the label hierarchy with the principle of structural entropy minimization, and injects the syntactic information into the text representation via hierarchical representation learning. Experiments on three common datasets are conducted to verify the superiority of HILL.

Related papers

• SE-GCL: An Event-Based Simple and Effective Graph Contrastive Learning for Text Representation [23.60337935010744]
  We present an event-based, simple, and effective graph contrastive learning (SE-GCL) for text representation. Precisely, we extract event blocks from text and construct internal relation graphs to represent inter-semantic interconnections. In particular, we introduce the concept of an event skeleton for core representation semantics and simplify the typically complex data augmentation techniques.
  arXiv  Detail & Related papers  (2024-12-16T10:53:24Z)
• Boosting Semi-Supervised Scene Text Recognition via Viewing and Summarizing [71.29488677105127]
  Existing scene text recognition (STR) methods struggle to recognize challenging texts, especially for artistic and severely distorted characters. We propose a contrastive learning-based STR framework by leveraging synthetic and real unlabeled data without any human cost. Our method achieves SOTA performance (94.7% and 70.9% average accuracy on common benchmarks and Union14M-Benchmark.
  arXiv  Detail & Related papers  (2024-11-23T15:24:47Z)
• Neural Sequence-to-Sequence Modeling with Attention by Leveraging Deep Learning Architectures for Enhanced Contextual Understanding in Abstractive Text Summarization [0.0]
  This paper presents a novel framework for abstractive TS of single documents. It integrates three dominant aspects: structure, semantic, and neural-based approaches. Results indicate significant improvements in handling rare and OOV words.
  arXiv  Detail & Related papers  (2024-04-08T18:33:59Z)
• Text2Data: Low-Resource Data Generation with Textual Control [100.5970757736845]
  Text2Data is a novel approach that utilizes unlabeled data to understand the underlying data distribution. It undergoes finetuning via a novel constraint optimization-based learning objective that ensures controllability and effectively counteracts catastrophic forgetting.
  arXiv  Detail & Related papers  (2024-02-08T03:41:39Z)
• Improving Mandarin Prosodic Structure Prediction with Multi-level Contextual Information [68.89000132126536]
  This work proposes to use inter-utterance linguistic information to improve the performance of prosodic structure prediction (PSP) Our method achieves better F1 scores in predicting prosodic word (PW), prosodic phrase (PPH) and intonational phrase (IPH)
  arXiv  Detail & Related papers  (2023-08-31T09:19:15Z)
• An xAI Approach for Data-to-Text Processing with ASP [39.58317527488534]
  This paper presents a framework that is compliant with xAI requirements. The text description is hierarchically organized, in a top-down structure where text is enriched with further details. The generation of natural language descriptions' structure is also managed by logic rules.
  arXiv  Detail & Related papers  (2023-08-30T09:09:09Z)
• Relational Contrastive Learning for Scene Text Recognition [22.131554868199782]
  We argue that prior contextual information can be interpreted as relations of textual primitives due to the heterogeneous text and background. We propose to enrich the textual relations via rearrangement, hierarchy and interaction, and design a unified framework called RCLSTR: Contrastive Learning for Scene Text Recognition.
  arXiv  Detail & Related papers  (2023-08-01T12:46:58Z)
• Informative Text Generation from Knowledge Triples [56.939571343797304]
  We propose a novel memory augmented generator that employs a memory network to memorize the useful knowledge learned during the training. We derive a dataset from WebNLG for our new setting and conduct extensive experiments to investigate the effectiveness of our model.
  arXiv  Detail & Related papers  (2022-09-26T14:35:57Z)
• Self-Supervised Visual Representation Learning with Semantic Grouping [50.14703605659837]
  We tackle the problem of learning visual representations from unlabeled scene-centric data. We propose contrastive learning from data-driven semantic slots, namely SlotCon, for joint semantic grouping and representation learning.
  arXiv  Detail & Related papers  (2022-05-30T17:50:59Z)
• CRIS: CLIP-Driven Referring Image Segmentation [71.56466057776086]
  We propose an end-to-end CLIP-Driven Referring Image framework (CRIS) CRIS resorts to vision-language decoding and contrastive learning for achieving the text-to-pixel alignment. Our proposed framework significantly outperforms the state-of-the-art performance without any post-processing.
  arXiv  Detail & Related papers  (2021-11-30T07:29:08Z)
• Learning Syntactic and Dynamic Selective Encoding for Document Summarization [17.666036645395845]
  We propose a novel neural architecture for document summarization. We incorporate syntactic information such as constituency parsing trees into the encoding sequence. We propose a dynamic gate network to select the salient information based on the context of the decoder state.
  arXiv  Detail & Related papers  (2020-03-25T01:29:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.