Enhancing Depressive Post Detection in Bangla: A Comparative Study of TF-IDF, BERT and FastText Embeddings

• URL: http://arxiv.org/abs/2407.09187v1
• Date: Fri, 12 Jul 2024 11:40:17 GMT
• Title: Enhancing Depressive Post Detection in Bangla: A Comparative Study of TF-IDF, BERT and FastText Embeddings
• Authors: Saad Ahmed Sazan, Mahdi H. Miraz, A B M Muntasir Rahman,
• Abstract summary: This study introduces a well-grounded approach to identify depressive social media posts in Bangla. The dataset used in this work, annotated by domain experts, includes both depressive and non-depressive posts. To address the issue of class imbalance, we utilised random oversampling for the minority class.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Due to massive adoption of social media, detection of users' depression through social media analytics bears significant importance, particularly for underrepresented languages, such as Bangla. This study introduces a well-grounded approach to identify depressive social media posts in Bangla, by employing advanced natural language processing techniques. The dataset used in this work, annotated by domain experts, includes both depressive and non-depressive posts, ensuring high-quality data for model training and evaluation. To address the prevalent issue of class imbalance, we utilised random oversampling for the minority class, thereby enhancing the model's ability to accurately detect depressive posts. We explored various numerical representation techniques, including Term Frequency-Inverse Document Frequency (TF-IDF), Bidirectional Encoder Representations from Transformers (BERT) embedding and FastText embedding, by integrating them with a deep learning-based Convolutional Neural Network-Bidirectional Long Short-Term Memory (CNN-BiLSTM) model. The results obtained through extensive experimentation, indicate that the BERT approach performed better the others, achieving a F1-score of 84%. This indicates that BERT, in combination with the CNN-BiLSTM architecture, effectively recognises the nuances of Bangla texts relevant to depressive contents. Comparative analysis with the existing state-of-the-art methods demonstrates that our approach with BERT embedding performs better than others in terms of evaluation metrics and the reliability of dataset annotations. Our research significantly contribution to the development of reliable tools for detecting depressive posts in the Bangla language. By highlighting the efficacy of different embedding techniques and deep learning models, this study paves the way for improved mental health monitoring through social media platforms.

Related papers

• Improving Sampling Methods for Fine-tuning SentenceBERT in Text Streams [49.3179290313959]
  This study explores the efficacy of seven text sampling methods designed to selectively fine-tune language models. We precisely assess the impact of these methods on fine-tuning the SBERT model using four different loss functions. Our findings indicate that Softmax loss and Batch All Triplets loss are particularly effective for text stream classification.
  arXiv  Detail & Related papers  (2024-03-18T23:41:52Z)
• Detecting mental disorder on social media: a ChatGPT-augmented explainable approach [1.7999333451993955]
  In the digital era, the prevalence of depressive symptoms expressed on social media has raised serious concerns. This paper proposes a novel methodology that effectively combines Large Language Models (LLMs) with eXplainable Artificial Intelligence (XAI) and conversational agents like ChatGPT. explanations are achieved by integrating BERTweet, a Twitter-specific variant of BERT, into a novel self-explanatory model, namely BERT-XDD. The interpretability is further enhanced using ChatGPT to transform technical explanations into human-readable commentaries.
  arXiv  Detail & Related papers  (2024-01-30T22:22:55Z)
• Harnessing Large Language Models Over Transformer Models for Detecting Bengali Depressive Social Media Text: A Comprehensive Study [0.0]
  This work focuses on early detection of depression using LLMs such as GPT 3.5, GPT 4 and our proposed GPT 3.5 fine-tuned model DepGPT. The study categorized Reddit and X datasets into "Depressive" and "Non-Depressive" segments, translated into Bengali by native speakers with expertise in mental health. Our work provides full architecture details for each model and a methodical way to assess their performance in Bengali depressive text categorization.
  arXiv  Detail & Related papers  (2024-01-14T15:15:58Z)
• Calibration of Transformer-based Models for Identifying Stress and Depression in Social Media [0.0]
  We present the first study in the task of depression and stress detection in social media, which injects extra linguistic information in transformer-based models. Specifically, the proposed approach employs a Multimodal Adaptation Gate for creating the combined embeddings, which are given as input to a BERT (or MentalBERT) model. We test our proposed approaches in three publicly available datasets and demonstrate that the integration of linguistic features into transformer-based models presents a surge in the performance.
  arXiv  Detail & Related papers  (2023-05-26T10:19:04Z)
• Depression detection in social media posts using affective and social norm features [84.12658971655253]
  We propose a deep architecture for depression detection from social media posts. We incorporate profanity and morality features of posts and words in our architecture using a late fusion scheme. The inclusion of the proposed features yields state-of-the-art results in both settings.
  arXiv  Detail & Related papers  (2023-03-24T21:26:27Z)
• Semantic Similarity Models for Depression Severity Estimation [53.72188878602294]
  This paper presents an efficient semantic pipeline to study depression severity in individuals based on their social media writings. We use test user sentences for producing semantic rankings over an index of representative training sentences corresponding to depressive symptoms and severity levels. We evaluate our methods on two Reddit-based benchmarks, achieving 30% improvement over state of the art in terms of measuring depression severity.
  arXiv  Detail & Related papers  (2022-11-14T18:47:26Z)
• Phrase-level Adversarial Example Generation for Neural Machine Translation [75.01476479100569]
  We propose a phrase-level adversarial example generation (PAEG) method to enhance the robustness of the model. We verify our method on three benchmarks, including LDC Chinese-English, IWSLT14 German-English, and WMT14 English-German tasks.
  arXiv  Detail & Related papers  (2022-01-06T11:00:49Z)
• Improving BERT Model Using Contrastive Learning for Biomedical Relation Extraction [13.354066085659198]
  Contrastive learning is not widely utilized in natural language processing due to the lack of a general method of data augmentation for text data. In this work, we explore the method of employing contrastive learning to improve the text representation from the BERT model for relation extraction. The experimental results on three relation extraction benchmark datasets demonstrate that our method can improve the BERT model representation and achieve state-of-the-art performance.
  arXiv  Detail & Related papers  (2021-04-28T17:50:24Z)
• An Interpretable End-to-end Fine-tuning Approach for Long Clinical Text [72.62848911347466]
  Unstructured clinical text in EHRs contains crucial information for applications including decision support, trial matching, and retrospective research. Recent work has applied BERT-based models to clinical information extraction and text classification, given these models' state-of-the-art performance in other NLP domains. In this work, we propose a novel fine-tuning approach called SnipBERT. Instead of using entire notes, SnipBERT identifies crucial snippets and feeds them into a truncated BERT-based model in a hierarchical manner.
  arXiv  Detail & Related papers  (2020-11-12T17:14:32Z)
• InfoBERT: Improving Robustness of Language Models from An Information Theoretic Perspective [84.78604733927887]
  Large-scale language models such as BERT have achieved state-of-the-art performance across a wide range of NLP tasks. Recent studies show that such BERT-based models are vulnerable facing the threats of textual adversarial attacks. We propose InfoBERT, a novel learning framework for robust fine-tuning of pre-trained language models.
  arXiv  Detail & Related papers  (2020-10-05T20:49:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.