Spatial-wise Dynamic Distillation for MLP-like Efficient Visual Fault Detection of Freight Trains

• URL: http://arxiv.org/abs/2312.05832v1
• Date: Sun, 10 Dec 2023 09:18:24 GMT
• Title: Spatial-wise Dynamic Distillation for MLP-like Efficient Visual Fault Detection of Freight Trains
• Abstract summary: We present a dynamic distillation framework based on multi-layer perceptron (MLP) for fault detection of freight trains. We propose a dynamic teacher that can effectively eliminate the semantic discrepancy with the student model. Our approach outperforms the current state-of-the-art detectors and achieves the highest accuracy with real-time detection at a lower computational cost.
• Score: 11.13191969085042
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Despite the successful application of convolutional neural networks (CNNs) in object detection tasks, their efficiency in detecting faults from freight train images remains inadequate for implementation in real-world engineering scenarios. Existing modeling shortcomings of spatial invariance and pooling layers in conventional CNNs often ignore the neglect of crucial global information, resulting in error localization for fault objection tasks of freight trains. To solve these problems, we design a spatial-wise dynamic distillation framework based on multi-layer perceptron (MLP) for visual fault detection of freight trains. We initially present the axial shift strategy, which allows the MLP-like architecture to overcome the challenge of spatial invariance and effectively incorporate both local and global cues. We propose a dynamic distillation method without a pre-training teacher, including a dynamic teacher mechanism that can effectively eliminate the semantic discrepancy with the student model. Such an approach mines more abundant details from lower-level feature appearances and higher-level label semantics as the extra supervision signal, which utilizes efficient instance embedding to model the global spatial and semantic information. In addition, the proposed dynamic teacher can jointly train with students to further enhance the distillation efficiency. Extensive experiments executed on six typical fault datasets reveal that our approach outperforms the current state-of-the-art detectors and achieves the highest accuracy with real-time detection at a lower computational cost. The source code will be available at \url{https://github.com/MVME-HBUT/SDD-FTI-FDet}.

Related papers

• Enhancing Gradient Inversion Attacks in Federated Learning via Hierarchical Feature Optimization [56.95448807869383]
  Federated Learning (FL) has emerged as a compelling paradigm for privacy-preserving distributed machine learning.<n>Recent studies find that the gradients exchanged in the FL system are also vulnerable to privacy leakage.<n>We propose textbfGradient textbfInversion over textbfFeature textbfDomains (GIFD)
  arXiv  Detail & Related papers  (2026-04-01T14:32:15Z)
• Enhancing Eye Feature Estimation from Event Data Streams through Adaptive Inference State Space Modeling [68.1289208938377]
  Eye feature extraction from event-based data streams can be performed efficiently and with low energy consumption.<n>We introduce the emphadaptive inference state space model (AISSM), a novel architecture for feature extraction.<n>We also craft and evaluate a novel learning technique that improves training efficiency.
  arXiv  Detail & Related papers  (2026-03-14T18:47:08Z)
• TopoCurate:Modeling Interaction Topology for Tool-Use Agent Training [53.93696896939915]
  Training tool-use agents typically rely on Supervised Fine-Tuning (SFT) on successful trajectories and Reinforcement Learning (RL) on pass-rate-selected tasks.<n>We propose TopoCurate, an interaction-aware framework that projects multi-trial rollouts from the same task into a unified semantic quotient topology.<n>TopoCurate achieves consistent gains of 4.2% (SFT) and 6.9% (RL) over state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-03-02T10:38:54Z)
• Domain-Decomposed Graph Neural Network Surrogate Modeling for Ice Sheets [34.15484094708584]
  We develop a physics-inspired graph neural network (GNN) surrogate that operates directly on unstructured meshes.<n>We employ transfer learning to fine-tune models across velocities, accelerating training and improving accuracy in data-limited settings.<n>Our results demonstrate that graph-based DD, combined with transfer learning, provides a scalable and reliable pathway for training GNN surrogates on massive PDE-governed systems.
  arXiv  Detail & Related papers  (2025-12-01T17:10:09Z)
• Information-Theoretic Greedy Layer-wise Training for Traffic Sign Recognition [0.5024983453990065]
  layer-wise training eliminates the need for cross-entropy loss and backpropagation.<n>Most existing layer-wise training approaches have been evaluated only on relatively small datasets.<n>We propose a novel layer-wise training approach based on the recently developed deterministic information bottleneck (DIB) and the matrix-based R'enyi's $alpha$-order entropy functional.
  arXiv  Detail & Related papers  (2025-10-31T17:24:58Z)
• UTAL-GNN: Unsupervised Temporal Action Localization using Graph Neural Networks [0.0]
  Fine-grained action localization in untrimmed sports videos presents a significant challenge due to rapid and subtle motion transitions.<n>Existing supervised and weakly supervised solutions often rely on extensive datasets and high-capacity models, making them computationally intensive and less adaptable to real-world scenarios.<n>Our approach pre-trains an Attention-based Spatio-Temporal Graph Convolutional Network (ASTGCN) on a pose-sequence denoising annotated with blockwise partitions.<n>Our method achieves a mean Average Precision (mAP) of 82.66% and average latency localization of 29.09 ms on the DSV Diving dataset
  arXiv  Detail & Related papers  (2025-08-27T07:51:02Z)
• DRTA: Dynamic Reward Scaling for Reinforcement Learning in Time Series Anomaly Detection [7.185726339205792]
  Anomaly detection in time series data is important for applications in finance, healthcare, sensor networks, and industrial monitoring.<n>We propose a reinforcement learning-based framework that integrates dynamic reward shaping, Variational Autoencoder (VAE), and active learning, called DRTA.<n>Our method uses an adaptive reward mechanism that balances exploration and exploitation by dynamically scaling the effect of VAE-based reconstruction error and classification rewards.
  arXiv  Detail & Related papers  (2025-08-25T20:39:49Z)
• U-PINet: End-to-End Hierarchical Physics-Informed Learning With Sparse Graph Coupling for 3D EM Scattering Modeling [28.64166932076228]
  Electromagnetic (EM) scattering modeling is critical for radar remote sensing.<n>Traditional numerical solvers offer high accuracy, but suffer from scalability issues and substantial computational costs.<n>We propose a U-shaped Physics-Informed Network (U-PINet) to overcome these limitations.
  arXiv  Detail & Related papers  (2025-08-05T12:20:42Z)
• USDRL: Unified Skeleton-Based Dense Representation Learning with Multi-Grained Feature Decorrelation [24.90512145836643]
  We introduce a Unified Skeleton-based Dense Representation Learning framework based on feature decorrelation. We show that our approach significantly outperforms the current state-of-the-art (SOTA) approaches.
  arXiv  Detail & Related papers  (2024-12-12T12:20:27Z)
• Highly Efficient and Unsupervised Framework for Moving Object Detection in Satellite Videos [0.2023650687546586]
  We propose a highly efficient unsupervised framework for SVMOD. We show that our method can not only process 9 frames per second on 1024x images but also achieve foreground-art performance.
  arXiv  Detail & Related papers  (2024-11-24T16:06:42Z)
• Point Cloud Understanding via Attention-Driven Contrastive Learning [64.65145700121442]
  Transformer-based models have advanced point cloud understanding by leveraging self-attention mechanisms. PointACL is an attention-driven contrastive learning framework designed to address these limitations. Our method employs an attention-driven dynamic masking strategy that guides the model to focus on under-attended regions.
  arXiv  Detail & Related papers  (2024-11-22T05:41:00Z)
• 3D Multi-Object Tracking with Semi-Supervised GRU-Kalman Filter [6.13623925528906]
  3D Multi-Object Tracking (MOT) is essential for intelligent systems like autonomous driving and robotic sensing. We propose a GRU-based MOT method, which introduces a learnable Kalman filter into the motion module. This approach is able to learn object motion characteristics through data-driven learning, thereby avoiding the need for manual model design and model error.
  arXiv  Detail & Related papers  (2024-11-13T08:34:07Z)
• Efficient High-Resolution Visual Representation Learning with State Space Model for Human Pose Estimation [60.80423207808076]
  Capturing long-range dependencies while preserving high-resolution visual representations is crucial for dense prediction tasks such as human pose estimation.<n>We propose the Dynamic Visual State Space (DVSS) block, which augments visual state space models with multi-scale convolutional operations.<n>We build HRVMamba, a novel model for efficient high-resolution representation learning.
  arXiv  Detail & Related papers  (2024-10-04T06:19:29Z)
• Dynamic Sub-graph Distillation for Robust Semi-supervised Continual Learning [52.046037471678005]
  We focus on semi-supervised continual learning (SSCL), where the model progressively learns from partially labeled data with unknown categories. We propose a novel approach called Dynamic Sub-Graph Distillation (DSGD) for semi-supervised continual learning.
  arXiv  Detail & Related papers  (2023-12-27T04:40:12Z)
• Over-the-Air Federated Learning and Optimization [52.5188988624998]
  We focus on Federated learning (FL) via edge-the-air computation (AirComp) We describe the convergence of AirComp-based FedAvg (AirFedAvg) algorithms under both convex and non- convex settings. For different types of local updates that can be transmitted by edge devices (i.e., model, gradient, model difference), we reveal that transmitting in AirFedAvg may cause an aggregation error. In addition, we consider more practical signal processing schemes to improve the communication efficiency and extend the convergence analysis to different forms of model aggregation error caused by these signal processing schemes.
  arXiv  Detail & Related papers  (2023-10-16T05:49:28Z)
• Implicit Stochastic Gradient Descent for Training Physics-informed Neural Networks [51.92362217307946]
  Physics-informed neural networks (PINNs) have effectively been demonstrated in solving forward and inverse differential equation problems. PINNs are trapped in training failures when the target functions to be approximated exhibit high-frequency or multi-scale features. In this paper, we propose to employ implicit gradient descent (ISGD) method to train PINNs for improving the stability of training process.
  arXiv  Detail & Related papers  (2023-03-03T08:17:47Z)
• Magnitude Matters: Fixing SIGNSGD Through Magnitude-Aware Sparsification in the Presence of Data Heterogeneity [60.791736094073]
  Communication overhead has become one of the major bottlenecks in the distributed training of deep neural networks. We propose a magnitude-driven sparsification scheme, which addresses the non-convergence issue of SIGNSGD. The proposed scheme is validated through experiments on Fashion-MNIST, CIFAR-10, and CIFAR-100 datasets.
  arXiv  Detail & Related papers  (2023-02-19T17:42:35Z)
• Minimizing the Accumulated Trajectory Error to Improve Dataset Distillation [151.70234052015948]
  We propose a novel approach that encourages the optimization algorithm to seek a flat trajectory. We show that the weights trained on synthetic data are robust against the accumulated errors perturbations with the regularization towards the flat trajectory. Our method, called Flat Trajectory Distillation (FTD), is shown to boost the performance of gradient-matching methods by up to 4.7%.
  arXiv  Detail & Related papers  (2022-11-20T15:49:11Z)
• Evaluating the Label Efficiency of Contrastive Self-Supervised Learning for Multi-Resolution Satellite Imagery [0.0]
  Self-supervised learning has been applied in the remote sensing domain to exploit readily-available unlabeled data. In this paper, we study self-supervised visual representation learning through the lens of label efficiency.
  arXiv  Detail & Related papers  (2022-10-13T06:54:13Z)
• Physics-Inspired Temporal Learning of Quadrotor Dynamics for Accurate Model Predictive Trajectory Tracking [76.27433308688592]
  Accurately modeling quadrotor's system dynamics is critical for guaranteeing agile, safe, and stable navigation. We present a novel Physics-Inspired Temporal Convolutional Network (PI-TCN) approach to learning quadrotor's system dynamics purely from robot experience. Our approach combines the expressive power of sparse temporal convolutions and dense feed-forward connections to make accurate system predictions.
  arXiv  Detail & Related papers  (2022-06-07T13:51:35Z)
• Supervised Learning in the Presence of Concept Drift: A modelling framework [5.22609266390809]
  We present a modelling framework for the investigation of supervised learning in non-stationary environments. We model two example types of learning systems: prototype-based Learning Vector Quantization (LVQ) for classification and shallow, layered neural networks for regression tasks.
  arXiv  Detail & Related papers  (2020-05-21T09:13:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.