A new training approach for text classification in Mental Health: LatentGLoss

• URL: http://arxiv.org/abs/2504.07245v1
• Date: Wed, 09 Apr 2025 19:34:31 GMT
• Title: A new training approach for text classification in Mental Health: LatentGLoss
• Authors: Korhan Sevinç,
• Abstract summary: This study presents a multi-stage approach to mental health classification by leveraging machine learning algorithms, deep learning architectures, and transformer-based models.<n>The core contribution of this study lies in a novel training strategy involving a dual-model architecture composed of a teacher and a student network.<n>The experimental results highlight the effectiveness of each modeling stage and demonstrate that the proposed loss function and teacher-student interaction significantly enhance the model's learning capacity in mental health prediction tasks.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: This study presents a multi-stage approach to mental health classification by leveraging traditional machine learning algorithms, deep learning architectures, and transformer-based models. A novel data set was curated and utilized to evaluate the performance of various methods, starting with conventional classifiers and advancing through neural networks. To broaden the architectural scope, recurrent neural networks (RNNs) such as LSTM and GRU were also evaluated to explore their effectiveness in modeling sequential patterns in the data. Subsequently, transformer models such as BERT were fine-tuned to assess the impact of contextual embeddings in this domain. Beyond these baseline evaluations, the core contribution of this study lies in a novel training strategy involving a dual-model architecture composed of a teacher and a student network. Unlike standard distillation techniques, this method does not rely on soft label transfer; instead, it facilitates information flow through both the teacher model's output and its latent representations by modifying the loss function. The experimental results highlight the effectiveness of each modeling stage and demonstrate that the proposed loss function and teacher-student interaction significantly enhance the model's learning capacity in mental health prediction tasks.

Related papers

• Generalized Factor Neural Network Model for High-dimensional Regression [50.554377879576066]
  We tackle the challenges of modeling high-dimensional data sets with latent low-dimensional structures hidden within complex, non-linear, and noisy relationships.<n>Our approach enables a seamless integration of concepts from non-parametric regression, factor models, and neural networks for high-dimensional regression.
  arXiv  Detail & Related papers  (2025-02-16T23:13:55Z)
• An Efficient Framework for Enhancing Discriminative Models via Diffusion Techniques [12.470257882838126]
  We propose the Diffusion-Based Discriminative Model Enhancement Framework (DBMEF)<n>This framework seamlessly integrates discriminative and generative models in a training-free manner.<n>DBMEF can effectively enhance the classification accuracy and capability of discriminative models in a plug-and-play manner.
  arXiv  Detail & Related papers  (2024-12-12T08:46:22Z)
• Deep Learning Through A Telescoping Lens: A Simple Model Provides Empirical Insights On Grokking, Gradient Boosting & Beyond [61.18736646013446]
  In pursuit of a deeper understanding of its surprising behaviors, we investigate the utility of a simple yet accurate model of a trained neural network. Across three case studies, we illustrate how it can be applied to derive new empirical insights on a diverse range of prominent phenomena.
  arXiv  Detail & Related papers  (2024-10-31T22:54:34Z)
• Domain-decomposed image classification algorithms using linear discriminant analysis and convolutional neural networks [0.0]
  Two different domain decomposed CNN models are experimentally compared for different image classification problems. The resulting models show improved classification accuracies compared to the corresponding, composed global CNN model. A novel decomposed LDA strategy is proposed which also relies on a localization approach and which is combined with a small neural network model.
  arXiv  Detail & Related papers  (2024-10-30T18:07:12Z)
• Revisiting Nearest Neighbor for Tabular Data: A Deep Tabular Baseline Two Decades Later [76.66498833720411]
  We introduce a differentiable version of $K$-nearest neighbors (KNN) originally designed to learn a linear projection to capture semantic similarities between instances.<n>Surprisingly, our implementation of NCA using SGD and without dimensionality reduction already achieves decent performance on tabular data.<n>We conclude our paper by analyzing the factors behind these improvements, including loss functions, prediction strategies, and deep architectures.
  arXiv  Detail & Related papers  (2024-07-03T16:38:57Z)
• Contrastive-Adversarial and Diffusion: Exploring pre-training and fine-tuning strategies for sulcal identification [3.0398616939692777]
  Techniques like adversarial learning, contrastive learning, diffusion denoising learning, and ordinary reconstruction learning have become standard. The study aims to elucidate the advantages of pre-training techniques and fine-tuning strategies to enhance the learning process of neural networks.
  arXiv  Detail & Related papers  (2024-05-29T15:44:51Z)
• Distilled Datamodel with Reverse Gradient Matching [74.75248610868685]
  We introduce an efficient framework for assessing data impact, comprising offline training and online evaluation stages. Our proposed method achieves comparable model behavior evaluation while significantly speeding up the process compared to the direct retraining method.
  arXiv  Detail & Related papers  (2024-04-22T09:16:14Z)
• SleepEGAN: A GAN-enhanced Ensemble Deep Learning Model for Imbalanced Classification of Sleep Stages [4.649202082648198]
  This paper develops a generative adversarial network (GAN)-powered ensemble deep learning model, named SleepEGAN, for the imbalanced classification of sleep stages. We show that the proposed method can improve classification accuracy compared to several existing state-of-the-art methods using three public sleep datasets.
  arXiv  Detail & Related papers  (2023-07-04T01:56:00Z)
• Regularization Through Simultaneous Learning: A Case Study on Plant Classification [0.0]
  This paper introduces Simultaneous Learning, a regularization approach drawing on principles of Transfer Learning and Multi-task Learning. We leverage auxiliary datasets with the target dataset, the UFOP-HVD, to facilitate simultaneous classification guided by a customized loss function. Remarkably, our approach demonstrates superior performance over models without regularization.
  arXiv  Detail & Related papers  (2023-05-22T19:44:57Z)
• Robust Graph Representation Learning via Predictive Coding [46.22695915912123]
  Predictive coding is a message-passing framework initially developed to model information processing in the brain. In this work, we build models that rely on the message-passing rule of predictive coding. We show that the proposed models are comparable to standard ones in terms of performance in both inductive and transductive tasks.
  arXiv  Detail & Related papers  (2022-12-09T03:58:22Z)
• Data-driven emergence of convolutional structure in neural networks [83.4920717252233]
  We show how fully-connected neural networks solving a discrimination task can learn a convolutional structure directly from their inputs. By carefully designing data models, we show that the emergence of this pattern is triggered by the non-Gaussian, higher-order local structure of the inputs.
  arXiv  Detail & Related papers  (2022-02-01T17:11:13Z)
• Improving Music Performance Assessment with Contrastive Learning [78.8942067357231]
  This study investigates contrastive learning as a potential method to improve existing MPA systems. We introduce a weighted contrastive loss suitable for regression tasks applied to a convolutional neural network. Our results show that contrastive-based methods are able to match and exceed SoTA performance for MPA regression tasks.
  arXiv  Detail & Related papers  (2021-08-03T19:24:25Z)
• Gone Fishing: Neural Active Learning with Fisher Embeddings [55.08537975896764]
  There is an increasing need for active learning algorithms that are compatible with deep neural networks. This article introduces BAIT, a practical representation of tractable, and high-performing active learning algorithm for neural networks.
  arXiv  Detail & Related papers  (2021-06-17T17:26:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.