Tiny-Toxic-Detector: A compact transformer-based model for toxic content detection

• URL: http://arxiv.org/abs/2409.02114v1
• Date: Thu, 29 Aug 2024 22:31:38 GMT
• Title: Tiny-Toxic-Detector: A compact transformer-based model for toxic content detection
• Authors: Michiel Kamphuis,
• Abstract summary: This paper presents Tiny-toxic-detector, a compact transformer-based model designed for toxic content detection. Despite having only 2.1 million parameters, Tiny-toxic-detector achieves competitive performance on benchmark datasets. Tiny-toxic-detector represents progress toward more sustainable and scalable AI-driven content moderation solutions.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: This paper presents Tiny-toxic-detector, a compact transformer-based model designed for toxic content detection. Despite having only 2.1 million parameters, Tiny-toxic-detector achieves competitive performance on benchmark datasets, with 90.97% accuracy on ToxiGen and 86.98% accuracy on the Jigsaw dataset, rivaling models over 50 times its size. This efficiency enables deployment in resource-constrained environments, addressing the need for effective content moderation tools that balance performance with computational efficiency. The model architecture features 4 transformer encoder layers, each with 2 attention heads, an embedding dimension of 64, and a feedforward dimension of 128. Trained on both public and private datasets, Tiny-toxic-detector demonstrates the potential of efficient, task-specific models for addressing online toxicity. The paper covers the model architecture, training process, performance benchmarks, and limitations, underscoring its suitability for applications such as social media monitoring and content moderation. By achieving results comparable to much larger models while significantly reducing computational demands, Tiny-toxic-detector represents progress toward more sustainable and scalable AI-driven content moderation solutions.

Related papers

• Information Capacity: Evaluating the Efficiency of Large Language Models via Text Compression [53.39128997308138]
  We introduce information capacity, a measure of model efficiency based on text compression performance.<n> Empirical evaluations on mainstream open-source models show that models of varying sizes within a series exhibit consistent information capacity.<n>A distinctive feature of information capacity is that it incorporates tokenizer efficiency, which affects both input and output token counts.
  arXiv  Detail & Related papers  (2025-11-11T10:07:32Z)
• OmniEAR: Benchmarking Agent Reasoning in Embodied Tasks [52.87238755666243]
  We present OmniEAR, a framework for evaluating how language models reason about physical interactions, tool usage, and multi-agent coordination in embodied tasks.<n>We model continuous physical properties and complex spatial relationships across 1,500 scenarios spanning household and industrial domains.<n>Our systematic evaluation reveals severe performance degradation when models must reason from constraints.
  arXiv  Detail & Related papers  (2025-08-07T17:54:15Z)
• Knowledge Grafting: A Mechanism for Optimizing AI Model Deployment in Resource-Constrained Environments [0.0]
  We introduce knowledge grafting to optimize AI models for resource-constrained environments.<n>The approach achieves an 88.54% reduction in model size.<n>It can be extended across various edge computing scenarios.
  arXiv  Detail & Related papers  (2025-07-25T13:37:45Z)
• Exploring Model Quantization in GenAI-based Image Inpainting and Detection of Arable Plants [0.0]
  We propose a framework that leverages Stable Diffusion-based inpainting to augment training data progressively in 10% increments -- up to an additional 200%. Our approach is evaluated on two state-of-the-art object detection models, YOLO11(l) and RT-DETR(l), using the mAP50 metric to assess detection performance. Deployment of the downstream models on the Jetson Orin Nano demonstrates the practical viability of our framework in resource-constrained environments.
  arXiv  Detail & Related papers  (2025-03-04T09:05:01Z)
• Low-Resolution Neural Networks [0.552480439325792]
  This study examines the impact of parameter bit precision on model performance compared to standard 32-bit models. Models analyzed include those with fully connected layers, convolutional layers, and transformer blocks. Findings suggest a potential new era for optimized neural network models with reduced memory requirements and improved computational efficiency.
  arXiv  Detail & Related papers  (2025-02-12T21:19:28Z)
• Efficient Brain Tumor Classification with Lightweight CNN Architecture: A Novel Approach [0.0]
  Brain tumor classification using MRI images is critical in medical diagnostics, where early and accurate detection significantly impacts patient outcomes. Recent advancements in deep learning (DL) have shown promise, but many models struggle with balancing accuracy and computational efficiency. We propose a novel model architecture integrating separable convolutions and squeeze and excitation (SE) blocks, designed to enhance feature extraction while maintaining computational efficiency.
  arXiv  Detail & Related papers  (2025-02-01T21:06:42Z)
• Task-Oriented Real-time Visual Inference for IoVT Systems: A Co-design Framework of Neural Networks and Edge Deployment [61.20689382879937]
  Task-oriented edge computing addresses this by shifting data analysis to the edge. Existing methods struggle to balance high model performance with low resource consumption. We propose a novel co-design framework to optimize neural network architecture.
  arXiv  Detail & Related papers  (2024-10-29T19:02:54Z)
• InfantCryNet: A Data-driven Framework for Intelligent Analysis of Infant Cries [24.06154195051215]
  We present a novel data-driven framework, "InfantCryNet," for accomplishing these tasks. We employ pre-trained audio models to incorporate prior knowledge into our model. Experiments on real-life datasets demonstrate the superior performance of the proposed framework.
  arXiv  Detail & Related papers  (2024-09-29T12:35:47Z)
• LORTSAR: Low-Rank Transformer for Skeleton-based Action Recognition [4.375744277719009]
  LORTSAR is applied to two leading Transformer-based models, "Hyperformer" and "STEP-CATFormer" Our method can reduce the number of model parameters substantially with negligible degradation or even performance increase in recognition accuracy. This confirms that SVD combined with post-compression fine-tuning can boost model efficiency, paving the way for more sustainable, lightweight, and high-performance technologies in human action recognition.
  arXiv  Detail & Related papers  (2024-07-19T20:19:41Z)
• Exploring the Effectiveness of Dataset Synthesis: An application of Apple Detection in Orchards [68.95806641664713]
  We explore the usability of Stable Diffusion 2.1-base for generating synthetic datasets of apple trees for object detection. We train a YOLOv5m object detection model to predict apples in a real-world apple detection dataset. Results demonstrate that the model trained on generated data is slightly underperforming compared to a baseline model trained on real-world images.
  arXiv  Detail & Related papers  (2023-06-20T09:46:01Z)
• Detection of Pavement Cracks by Deep Learning Models of Transformer and UNet [9.483452333312373]
  In recent years, the emergence and development of deep learning techniques have shown great potential to facilitate surface crack detection. In this study, we investigated nine promising models to evaluate their performance in pavement surface crack detection by model accuracy, computational complexity, and model stability. We find that transformer-based models generally are easier to converge during the training process and have higher accuracy, but usually exhibit more memory consumption and low processing efficiency.
  arXiv  Detail & Related papers  (2023-04-25T06:07:49Z)
• A Computer Vision Enabled damage detection model with improved YOLOv5 based on Transformer Prediction Head [0.0]
  Current state-of-the-art deep learning (DL)-based damage detection models often lack superior feature extraction capability in complex and noisy environments. DenseSPH-YOLOv5 is a real-time DL-based high-performance damage detection model where DenseNet blocks have been integrated with the backbone. DenseSPH-YOLOv5 obtains a mean average precision (mAP) value of 85.25 %, F1-score of 81.18 %, and precision (P) value of 89.51 % outperforming current state-of-the-art models.
  arXiv  Detail & Related papers  (2023-03-07T22:53:36Z)
• Fast Exploration of the Impact of Precision Reduction on Spiking Neural Networks [63.614519238823206]
  Spiking Neural Networks (SNNs) are a practical choice when the target hardware reaches the edge of computing. We employ an Interval Arithmetic (IA) model to develop an exploration methodology that takes advantage of the capability of such a model to propagate the approximation error.
  arXiv  Detail & Related papers  (2022-11-22T15:08:05Z)
• TinyDefectNet: Highly Compact Deep Neural Network Architecture for High-Throughput Manufacturing Visual Quality Inspection [72.88856890443851]
  TinyDefectNet is a highly compact deep convolutional network architecture tailored for high- throughput manufacturing visual quality inspection. TinyDefectNet was deployed on an AMD EPYC 7R32, and achieved 7.6x faster throughput using the nativeflow environment and 9x faster throughput using AMD ZenDNN accelerator library.
  arXiv  Detail & Related papers  (2021-11-29T04:19:28Z)
• MoEfication: Conditional Computation of Transformer Models for Efficient Inference [66.56994436947441]
  Transformer-based pre-trained language models can achieve superior performance on most NLP tasks due to large parameter capacity, but also lead to huge computation cost. We explore to accelerate large-model inference by conditional computation based on the sparse activation phenomenon. We propose to transform a large model into its mixture-of-experts (MoE) version with equal model size, namely MoEfication.
  arXiv  Detail & Related papers  (2021-10-05T02:14:38Z)
• Neural Network Virtual Sensors for Fuel Injection Quantities with Provable Performance Specifications [71.1911136637719]
  We show how provable guarantees can be naturally applied to other real world settings. We show how specific intervals of fuel injection quantities can be targeted to maximize robustness for certain ranges.
  arXiv  Detail & Related papers  (2020-06-30T23:33:17Z)
• ResOT: Resource-Efficient Oblique Trees for Neural Signal Classification [12.977151652608047]
  This paper introduces a machine learning model based on oblique decision trees to enable resource-efficient classification on a neural implant. We trained the resource-efficient tree with power-efficient regularization on three neural classification tasks to evaluate the performance, memory, and hardware requirements. The proposed model can enable a low-power memory-efficient implementation of classifiers for real-time neurological disease detection and motor decoding.
  arXiv  Detail & Related papers  (2020-06-14T13:29:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.