The Text Classification Pipeline: Starting Shallow going Deeper

• URL: http://arxiv.org/abs/2501.00174v2
• Date: Thu, 20 Mar 2025 19:18:07 GMT
• Title: The Text Classification Pipeline: Starting Shallow going Deeper
• Authors: Marco Siino, Ilenia Tinnirello, Marco La Cascia,
• Abstract summary: The past decade has seen deep learning revolutionize text classification.<n>English is the predominant language of focus, despite studies involving Arabic, Chinese, Hindi, and others.<n>This work integrates traditional and contemporary text mining methodologies, fostering a holistic understanding of text classification.
• Score: 4.97309503788908
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Text classification stands as a cornerstone within the realm of Natural Language Processing (NLP), particularly when viewed through computer science and engineering. The past decade has seen deep learning revolutionize text classification, propelling advancements in text retrieval, categorization, information extraction, and summarization. The scholarly literature includes datasets, models, and evaluation criteria, with English being the predominant language of focus, despite studies involving Arabic, Chinese, Hindi, and others. The efficacy of text classification models relies heavily on their ability to capture intricate textual relationships and non-linear correlations, necessitating a comprehensive examination of the entire text classification pipeline. In the NLP domain, a plethora of text representation techniques and model architectures have emerged, with Large Language Models (LLMs) and Generative Pre-trained Transformers (GPTs) at the forefront. These models are adept at transforming extensive textual data into meaningful vector representations encapsulating semantic information. The multidisciplinary nature of text classification, encompassing data mining, linguistics, and information retrieval, highlights the importance of collaborative research to advance the field. This work integrates traditional and contemporary text mining methodologies, fostering a holistic understanding of text classification.

Related papers

• Comprehensive Implementation of TextCNN for Enhanced Collaboration between Natural Language Processing and System Recommendation [1.7692743931394748]
  This paper analyzes the application status of deep learning in the three core tasks of NLP. It takes into account the challenges posed by adversarial techniques in text generation, text classification, and semantic parsing. An empirical study on text classification tasks demonstrates the effectiveness of interactive integration training.
  arXiv  Detail & Related papers  (2024-03-12T07:25:53Z)
• Surveying the Dead Minds: Historical-Psychological Text Analysis with Contextualized Construct Representation (CCR) for Classical Chinese [4.772998830872483]
  We develop a pipeline for historical-psychological text analysis in classical Chinese. The pipeline combines expert knowledge in psychometrics with text representations generated via transformer-based language models. Considering the scarcity of available data, we propose an indirect supervised contrastive learning approach.
  arXiv  Detail & Related papers  (2024-03-01T13:14:45Z)
• A Systematic Review of Data-to-Text NLG [2.4769539696439677]
  Methods for producing high-quality text are explored, addressing the challenge of hallucinations in data-to-text generation. Despite advancements in text quality, the review emphasizes the importance of research in low-resourced languages.
  arXiv  Detail & Related papers  (2024-02-13T14:51:45Z)
• Automatic and Human-AI Interactive Text Generation [27.05024520190722]
  This tutorial aims to provide an overview of the state-of-the-art natural language generation research. Text-to-text generation tasks are more constrained in terms of semantic consistency and targeted language styles.
  arXiv  Detail & Related papers  (2023-10-05T20:26:15Z)
• Text Classification: A Perspective of Deep Learning Methods [0.0679877553227375]
  This paper introduces deep learning-based text classification algorithms, including important steps required for text classification tasks. At the end of the article, different deep learning text classification methods are compared and summarized.
  arXiv  Detail & Related papers  (2023-09-24T21:49:51Z)
• Multi-Dimensional Evaluation of Text Summarization with In-Context Learning [79.02280189976562]
  In this paper, we study the efficacy of large language models as multi-dimensional evaluators using in-context learning. Our experiments show that in-context learning-based evaluators are competitive with learned evaluation frameworks for the task of text summarization. We then analyze the effects of factors such as the selection and number of in-context examples on performance.
  arXiv  Detail & Related papers  (2023-06-01T23:27:49Z)
• OCRBench: On the Hidden Mystery of OCR in Large Multimodal Models [122.27878464009181]
  We conducted a comprehensive evaluation of Large Multimodal Models, such as GPT4V and Gemini, in various text-related visual tasks. OCRBench contains 29 datasets, making it the most comprehensive OCR evaluation benchmark available.
  arXiv  Detail & Related papers  (2023-05-13T11:28:37Z)
• Classifying text using machine learning models and determining conversation drift [4.785406121053965]
  An analysis of various types of texts is invaluable to understanding both their semantic meaning, as well as their relevance. Text classification is a method of categorising documents. It combines computer text classification and natural language processing to analyse text in aggregate.
  arXiv  Detail & Related papers  (2022-11-15T18:09:45Z)
• An Inclusive Notion of Text [69.36678873492373]
  We argue that clarity on the notion of text is crucial for reproducible and generalizable NLP. We introduce a two-tier taxonomy of linguistic and non-linguistic elements that are available in textual sources and can be used in NLP modeling.
  arXiv  Detail & Related papers  (2022-11-10T14:26:43Z)
• Pre-training Language Model Incorporating Domain-specific Heterogeneous Knowledge into A Unified Representation [49.89831914386982]
  We propose a unified pre-trained language model (PLM) for all forms of text, including unstructured text, semi-structured text, and well-structured text. Our approach outperforms the pre-training of plain text using only 1/4 of the data.
  arXiv  Detail & Related papers  (2021-09-02T16:05:24Z)
• Deep Learning Schema-based Event Extraction: Literature Review and Current Trends [60.29289298349322]
  Event extraction technology based on deep learning has become a research hotspot. This paper fills the gap by reviewing the state-of-the-art approaches, focusing on deep learning-based models.
  arXiv  Detail & Related papers  (2021-07-05T16:32:45Z)
• Exploring Context Generalizability in Citywide Crowd Mobility Prediction: An Analytic Framework and Benchmark [8.32409186492201]
  We present a unified analytic framework and a large-scale benchmark for evaluating context generalizability.<n>We conduct experiments in several crowd mobility prediction tasks such as bike flow, metro passenger flow, and electric vehicle charging demand.<n>Using more contextual features may not always result in better prediction with existing context modeling techniques.<n>In context modeling techniques, using a gated unit to incorporate raw contextual features into the deep prediction model has good generalizability.
  arXiv  Detail & Related papers  (2021-06-30T13:19:41Z)
• Deep Learning for Text Style Transfer: A Survey [71.8870854396927]
  Text style transfer is an important task in natural language generation, which aims to control certain attributes in the generated text. We present a systematic survey of the research on neural text style transfer, spanning over 100 representative articles since the first neural text style transfer work in 2017. We discuss the task formulation, existing datasets and subtasks, evaluation, as well as the rich methodologies in the presence of parallel and non-parallel data.
  arXiv  Detail & Related papers  (2020-11-01T04:04:43Z)
• A Survey on Text Classification: From Shallow to Deep Learning [83.47804123133719]
  The last decade has seen a surge of research in this area due to the unprecedented success of deep learning. This paper fills the gap by reviewing the state-of-the-art approaches from 1961 to 2021. We create a taxonomy for text classification according to the text involved and the models used for feature extraction and classification.
  arXiv  Detail & Related papers  (2020-08-02T00:09:03Z)
• Deep Learning Based Text Classification: A Comprehensive Review [75.8403533775179]
  We provide a review of more than 150 deep learning based models for text classification developed in recent years. We also provide a summary of more than 40 popular datasets widely used for text classification.
  arXiv  Detail & Related papers  (2020-04-06T02:00:30Z)
• Deep Learning for Hindi Text Classification: A Comparison [6.8629257716723]
  The research in the classification of morphologically rich and low resource Hindi language written in Devanagari script has been limited due to the absence of large labeled corpus. In this work, we used translated versions of English data-sets to evaluate models based on CNN, LSTM and Attention. The paper also serves as a tutorial for popular text classification techniques.
  arXiv  Detail & Related papers  (2020-01-19T09:29:12Z)
• Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer [64.22926988297685]
  Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP) In this paper, we explore the landscape of introducing transfer learning techniques for NLP by a unified framework that converts all text-based language problems into a text-to-text format.
  arXiv  Detail & Related papers  (2019-10-23T17:37:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.