Graph-tree Fusion Model with Bidirectional Information Propagation for Long Document Classification

• URL: http://arxiv.org/abs/2410.02930v1
• Date: Thu, 3 Oct 2024 19:25:01 GMT
• Title: Graph-tree Fusion Model with Bidirectional Information Propagation for Long Document Classification
• Authors: Sudipta Singha Roy, Xindi Wang, Robert E. Mercer, Frank Rudzicz,
• Abstract summary: Long document classification presents challenges due to their extensive content and complex structure. Existing methods often struggle with token limits and fail to adequately model hierarchical relationships within documents. Our approach integrates syntax trees for sentence encodings and document graphs for document encodings, which capture fine-grained syntactic relationships and broader document contexts.
• Score: 20.434941308959786
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Long document classification presents challenges in capturing both local and global dependencies due to their extensive content and complex structure. Existing methods often struggle with token limits and fail to adequately model hierarchical relationships within documents. To address these constraints, we propose a novel model leveraging a graph-tree structure. Our approach integrates syntax trees for sentence encodings and document graphs for document encodings, which capture fine-grained syntactic relationships and broader document contexts, respectively. We use Tree Transformers to generate sentence encodings, while a graph attention network models inter- and intra-sentence dependencies. During training, we implement bidirectional information propagation from word-to-sentence-to-document and vice versa, which enriches the contextual representation. Our proposed method enables a comprehensive understanding of content at all hierarchical levels and effectively handles arbitrarily long contexts without token limit constraints. Experimental results demonstrate the effectiveness of our approach in all types of long document classification tasks.

Related papers

• Contextual Document Embeddings [77.22328616983417]
  We propose two complementary methods for contextualized document embeddings. First, an alternative contrastive learning objective that explicitly incorporates the document neighbors into the intra-batch contextual loss. Second, a new contextual architecture that explicitly encodes neighbor document information into the encoded representation.
  arXiv  Detail & Related papers  (2024-10-03T14:33:34Z)
• Hierarchical Attention Graph for Scientific Document Summarization in Global and Local Level [3.7651378994837104]
  Long input hinders simultaneous modeling of both global high-order relations between sentences and local intra-sentence relations. We propose HAESum, a novel approach utilizing graph neural networks to model documents based on their hierarchical discourse structure. We validate our approach on two benchmark datasets, and the experimental results demonstrate the effectiveness of HAESum.
  arXiv  Detail & Related papers  (2024-05-16T15:46:30Z)
• GraphKD: Exploring Knowledge Distillation Towards Document Object Detection with Structured Graph Creation [14.511401955827875]
  Object detection in documents is a key step to automate the structural elements identification process. We present a graph-based knowledge distillation framework to correctly identify and localize the document objects in a document image.
  arXiv  Detail & Related papers  (2024-02-17T23:08:32Z)
• Document-level Relation Extraction with Cross-sentence Reasoning Graph [14.106582119686635]
  Relation extraction (RE) has recently moved from the sentence-level to document-level. We propose a novel document-level RE model with a GRaph information Aggregation and Cross-sentence Reasoning network (GRACR) Experimental results show GRACR achieves excellent performance on two public datasets of document-level RE.
  arXiv  Detail & Related papers  (2023-03-07T14:14:12Z)
• Long Document Summarization with Top-down and Bottom-up Inference [113.29319668246407]
  We propose a principled inference framework to improve summarization models on two aspects. Our framework assumes a hierarchical latent structure of a document where the top-level captures the long range dependency. We demonstrate the effectiveness of the proposed framework on a diverse set of summarization datasets.
  arXiv  Detail & Related papers  (2022-03-15T01:24:51Z)
• Unsupervised Matching of Data and Text [6.2520079463149205]
  We introduce a framework that supports matching textual content and structured data in an unsupervised setting. Our method builds a fine-grained graph over the content of the corpora and derives word embeddings to represent the objects to match in a low dimensional space. Experiments on real use cases and public datasets show that our framework produces embeddings that outperform word embeddings and fine-tuned language models.
  arXiv  Detail & Related papers  (2021-12-16T10:40:48Z)
• BASS: Boosting Abstractive Summarization with Unified Semantic Graph [49.48925904426591]
  BASS is a framework for Boosting Abstractive Summarization based on a unified Semantic graph. A graph-based encoder-decoder model is proposed to improve both the document representation and summary generation process. Empirical results show that the proposed architecture brings substantial improvements for both long-document and multi-document summarization tasks.
  arXiv  Detail & Related papers  (2021-05-25T16:20:48Z)
• Mention-centered Graph Neural Network for Document-level Relation Extraction [2.724649366608364]
  We build cross-sentence dependencies by inferring compositional relations between inter-sentence mentions. Experiments show the connections between different mentions are crucial to document-level relation extraction.
  arXiv  Detail & Related papers  (2021-03-15T08:19:44Z)
• Heterogeneous Graph Neural Networks for Extractive Document Summarization [101.17980994606836]
  Cross-sentence relations are a crucial step in extractive document summarization. We present a graph-based neural network for extractive summarization (HeterSumGraph) We introduce different types of nodes into graph-based neural networks for extractive document summarization.
  arXiv  Detail & Related papers  (2020-04-26T14:38:11Z)
• Fine-grained Video-Text Retrieval with Hierarchical Graph Reasoning [72.52804406378023]
  Cross-modal retrieval between videos and texts has attracted growing attentions due to the rapid emergence of videos on the web. To improve fine-grained video-text retrieval, we propose a Hierarchical Graph Reasoning model, which decomposes video-text matching into global-to-local levels.
  arXiv  Detail & Related papers  (2020-03-01T03:44:19Z)
• Learning to Select Bi-Aspect Information for Document-Scale Text Content Manipulation [50.01708049531156]
  We focus on a new practical task, document-scale text content manipulation, which is the opposite of text style transfer. In detail, the input is a set of structured records and a reference text for describing another recordset. The output is a summary that accurately describes the partial content in the source recordset with the same writing style of the reference.
  arXiv  Detail & Related papers  (2020-02-24T12:52:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.