Extended LSTM: Adaptive Feature Gating for Toxic Comment Classification

• URL: http://arxiv.org/abs/2510.17018v1
• Date: Sun, 19 Oct 2025 21:50:04 GMT
• Title: Extended LSTM: Adaptive Feature Gating for Toxic Comment Classification
• Authors: Noor Islam S. Mohammad,
• Abstract summary: xLSTM is a framework that unifies cosine-similarity gating, adaptive feature prioritization, and principled class rebalancing.<n>On the Jigsaw Toxic Comment benchmark, xLSTM attains 96.4% accuracy and 0.88 macro-F1, outperforming BERT by 33% on threat and 28% on identity_hate categories.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Toxic comment detection remains a challenging task, where transformer-based models (e.g., BERT) incur high computational costs and degrade on minority toxicity classes, while classical ensembles lack semantic adaptability. We propose xLSTM, a parameter-efficient and theoretically grounded framework that unifies cosine-similarity gating, adaptive feature prioritization, and principled class rebalancing. A learnable reference vector {v} in {R}^d modulates contextual embeddings via cosine similarity, amplifying toxic cues and attenuating benign signals to yield stronger gradients under severe class imbalance. xLSTM integrates multi-source embeddings (GloVe, FastText, BERT CLS) through a projection layer, a character-level BiLSTM for morphological cues, embedding-space SMOTE for minority augmentation, and adaptive focal loss with dynamic class weighting. On the Jigsaw Toxic Comment benchmark, xLSTM attains 96.0% accuracy and 0.88 macro-F1, outperforming BERT by 33% on threat and 28% on identity_hate categories, with 15 times fewer parameters and 50ms inference latency. Cosine gating contributes a +4.8% F1 gain in ablations. The results establish a new efficiency adaptability frontier, demonstrating that lightweight, theoretically informed architectures can surpass large pretrained models on imbalanced, domain-specific NLP tasks.

Related papers

• Opinion Mining and Analysis Using Hybrid Deep Neural Networks [0.0]
  We introduce a hybrid deep neural network model that combines a bidirectional gated recurrent unit (BGRU) and long short-term memory (LSTM) layers to improve sentiment analysis.<n>Our model exhibited a noteworthy enhancement in recall for negative sentiments, escalating from 86% (unbalanced dataset) to 96% (balanced dataset)
  arXiv  Detail & Related papers  (2025-11-16T19:04:44Z)
• Singular Value Decomposition on Kronecker Adaptation for Large Language Model [0.8747606955991707]
  Large pre-trained Transformer models achieve state-of-the-art results across diverse language and reasoning tasks.<n>Full fine-tuning incurs substantial storage, memory, and computational overhead.<n>We propose SoKA, a novel PEFT strategy that combines Kronecker-product tensor factorization with SVD-driven initialization and dynamic rank selection.
  arXiv  Detail & Related papers  (2025-06-18T08:28:53Z)
• Taming Polysemanticity in LLMs: Provable Feature Recovery via Sparse Autoencoders [50.52694757593443]
  Existing SAE training algorithms often lack rigorous mathematical guarantees and suffer from practical limitations.<n>We first propose a novel statistical framework for the feature recovery problem, which includes a new notion of feature identifiability.<n>We introduce a new SAE training algorithm based on bias adaptation'', a technique that adaptively adjusts neural network bias parameters to ensure appropriate activation sparsity.
  arXiv  Detail & Related papers  (2025-06-16T20:58:05Z)
• Taming LLMs by Scaling Learning Rates with Gradient Grouping [49.91587150497186]
  Training large language models (LLMs) poses challenges due to their massive scale and heterogeneous architectures.<n>This work introduces Scaling with Gradient Grouping (SGG), an gradient wrapper that improves adaptive learning rate estimation by dynamic grouping and group-specific scaling.
  arXiv  Detail & Related papers  (2025-06-01T15:30:37Z)
• HAFLQ: Heterogeneous Adaptive Federated LoRA Fine-tuned LLM with Quantization [55.972018549438964]
  Federated fine-tuning of pre-trained Large Language Models (LLMs) enables task-specific adaptation across diverse datasets while preserving privacy.<n>We propose HAFLQ (Heterogeneous Adaptive Federated Low-Rank Adaptation Fine-tuned LLM with Quantization), a novel framework for efficient and scalable fine-tuning of LLMs in heterogeneous environments.<n> Experimental results on the text classification task demonstrate that HAFLQ reduces memory usage by 31%, lowers communication cost by 49%, improves accuracy by 50%, and achieves faster convergence compared to the baseline method.
  arXiv  Detail & Related papers  (2024-11-10T19:59:54Z)
• Model Surgery: Modulating LLM's Behavior Via Simple Parameter Editing [63.20133320524577]
  We show that editing a small subset of parameters can effectively modulate specific behaviors of large language models (LLMs)<n>Our approach achieves reductions of up to 90.0% in toxicity on the RealToxicityPrompts dataset and 49.2% on ToxiGen.
  arXiv  Detail & Related papers  (2024-07-11T17:52:03Z)
• STAL: Spike Threshold Adaptive Learning Encoder for Classification of Pain-Related Biosignal Data [2.0738462952016232]
  This paper presents the first application of spiking neural networks (SNNs) for the classification of chronic lower back pain (CLBP) using the EmoPain dataset. We introduce Spike Threshold Adaptive Learning (STAL), a trainable encoder that effectively converts continuous biosignals into spike trains. We also propose an ensemble of Spiking Recurrent Neural Network (SRNN) classifiers for the multi-stream processing of sEMG and IMU data.
  arXiv  Detail & Related papers  (2024-07-11T10:15:52Z)
• Adaptive physics-informed neural operator for coarse-grained non-equilibrium flows [0.0]
  The framework combines dimensionality reduction and neural operators through a hierarchical and adaptive deep learning strategy. The proposed surrogate's architecture is structured as a tree, with leaf nodes representing separate neural operator blocks. In 0-D scenarios, the proposed ML framework can adaptively predict the dynamics of almost thirty species with a maximum relative error of 4.5%.
  arXiv  Detail & Related papers  (2022-10-27T23:26:57Z)
• Label-Imbalanced and Group-Sensitive Classification under Overparameterization [32.923780772605596]
  Label-imbalanced and group-sensitive classification seeks to appropriately modify standard training algorithms to optimize relevant metrics. We show that a logit-adjusted loss modification to standard empirical risk minimization might be ineffective in general. We show that our results extend naturally to binary classification with sensitive groups, thus treating the two common types of imbalances (label/group) in a unifying way.
  arXiv  Detail & Related papers  (2021-03-02T08:09:43Z)
• Efficient Semantic Image Synthesis via Class-Adaptive Normalization [116.63715955932174]
  Class-adaptive normalization (CLADE) is a lightweight but equally-effective variant that is only adaptive to semantic class. We introduce intra-class positional map encoding calculated from semantic layouts to modulate the normalization parameters of CLADE. The proposed CLADE can be generalized to different SPADE-based methods while achieving comparable generation quality compared to SPADE.
  arXiv  Detail & Related papers  (2020-12-08T18:59:32Z)
• Rethinking Spatially-Adaptive Normalization [111.13203525538496]
  Class-adaptive normalization (CLADE) is a lightweight variant that is not adaptive to spatial positions or layouts. CLADE greatly reduces the computation cost while still being able to preserve the semantic information during the generation.
  arXiv  Detail & Related papers  (2020-04-06T17:58:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.