Deep Learning for Technical Document Classification

• URL: http://arxiv.org/abs/2106.14269v1
• Date: Sun, 27 Jun 2021 16:12:47 GMT
• Title: Deep Learning for Technical Document Classification
• Authors: Shuo Jiang, Jianxi Luo, Jie Hu, Christopher L. Magee
• Abstract summary: This paper describes a novel multimodal deep learning architecture, called TechDoc, for technical document classification. The trained model can potentially be scaled to millions of real-world technical documents with both text and figures.
• Score: 6.787004826008753
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In large technology companies, the requirements for managing and organizing technical documents created by engineers and managers in supporting relevant decision making have increased dramatically in recent years, which has led to a higher demand for more scalable, accurate, and automated document classification. Prior studies have primarily focused on processing text for classification and small-scale databases. This paper describes a novel multimodal deep learning architecture, called TechDoc, for technical document classification, which utilizes both natural language and descriptive images to train hierarchical classifiers. The architecture synthesizes convolutional neural networks and recurrent neural networks through an integrated training process. We applied the architecture to a large multimodal technical document database and trained the model for classifying documents based on the hierarchical International Patent Classification system. Our results show that the trained neural network presents a greater classification accuracy than those using a single modality and several earlier text classification methods. The trained model can potentially be scaled to millions of real-world technical documents with both text and figures, which is useful for data and knowledge management in large technology companies and organizations.

Related papers

• Are Large Language Models Good Classifiers? A Study on Edit Intent Classification in Scientific Document Revisions [62.12545440385489]
  Large language models (LLMs) have brought substantial advancements in text generation, but their potential for enhancing classification tasks remains underexplored. We propose a framework for thoroughly investigating fine-tuning LLMs for classification, including both generation- and encoding-based approaches. We instantiate this framework in edit intent classification (EIC), a challenging and underexplored classification task.
  arXiv  Detail & Related papers  (2024-10-02T20:48:28Z)
• HDT: Hierarchical Document Transformer [70.2271469410557]
  HDT exploits document structure by introducing auxiliary anchor tokens and redesigning the attention mechanism into a sparse multi-level hierarchy. We develop a novel sparse attention kernel that considers the hierarchical structure of documents.
  arXiv  Detail & Related papers  (2024-07-11T09:28:04Z)
• Incremental hierarchical text clustering methods: a review [49.32130498861987]
  This study aims to analyze various hierarchical and incremental clustering techniques. The main contribution of this research is the organization and comparison of the techniques used by studies published between 2010 and 2018 that aimed to texts documents clustering.
  arXiv  Detail & Related papers  (2023-12-12T22:27:29Z)
• Data Efficient Training of a U-Net Based Architecture for Structured Documents Localization [0.0]
  We propose SDL-Net: a novel U-Net like encoder-decoder architecture for the localization of structured documents. Our approach allows pre-training the encoder of SDL-Net on a generic dataset containing samples of various document classes.
  arXiv  Detail & Related papers  (2023-10-02T07:05:19Z)
• Text Classification: A Perspective of Deep Learning Methods [0.0679877553227375]
  This paper introduces deep learning-based text classification algorithms, including important steps required for text classification tasks. At the end of the article, different deep learning text classification methods are compared and summarized.
  arXiv  Detail & Related papers  (2023-09-24T21:49:51Z)
• Minimally-Supervised Structure-Rich Text Categorization via Learning on Text-Rich Networks [61.23408995934415]
  We propose a novel framework for minimally supervised categorization by learning from the text-rich network. Specifically, we jointly train two modules with different inductive biases -- a text analysis module for text understanding and a network learning module for class-discriminative, scalable network learning. Our experiments show that given only three seed documents per category, our framework can achieve an accuracy of about 92%.
  arXiv  Detail & Related papers  (2021-02-23T04:14:34Z)
• Hierarchical Metadata-Aware Document Categorization under Weak Supervision [32.80303008934164]
  We develop HiMeCat, an embedding-based generative framework for our task. We propose a novel joint representation learning module that allows simultaneous modeling of category dependencies. We introduce a data augmentation module that hierarchically synthesizes training documents to complement the original, small-scale training set.
  arXiv  Detail & Related papers  (2020-10-26T13:07:56Z)
• End to End Binarized Neural Networks for Text Classification [4.046236197219608]
  We propose an end to end binarized neural network architecture for the intent classification task. The proposed architecture achieves comparable to the state-of-the-art results on standard intent classification datasets.
  arXiv  Detail & Related papers  (2020-10-11T11:21:53Z)
• Automated Search for Resource-Efficient Branched Multi-Task Networks [81.48051635183916]
  We propose a principled approach, rooted in differentiable neural architecture search, to automatically define branching structures in a multi-task neural network. We show that our approach consistently finds high-performing branching structures within limited resource budgets.
  arXiv  Detail & Related papers  (2020-08-24T09:49:19Z)
• Light-Weighted CNN for Text Classification [0.0]
  We introduce a new architecture based on separable convolution. The idea of separable convolution already exists in the field of image classification. With the help of this architecture, we can achieve a drastic reduction in trainable parameters.
  arXiv  Detail & Related papers  (2020-04-16T20:23:52Z)
• SPECTER: Document-level Representation Learning using Citation-informed Transformers [51.048515757909215]
  SPECTER generates document-level embedding of scientific documents based on pretraining a Transformer language model. We introduce SciDocs, a new evaluation benchmark consisting of seven document-level tasks ranging from citation prediction to document classification and recommendation.
  arXiv  Detail & Related papers  (2020-04-15T16:05:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.