FISHER: A Foundation Model for Multi-Modal Industrial Signal Comprehensive Representation

• URL: http://arxiv.org/abs/2507.16696v1
• Date: Tue, 22 Jul 2025 15:31:16 GMT
• Title: FISHER: A Foundation Model for Multi-Modal Industrial Signal Comprehensive Representation
• Authors: Pingyi Fan, Anbai Jiang, Shuwei Zhang, Zhiqiang Lv, Bing Han, Xinhu Zheng, Wenrui Liang, Junjie Li, Wei-Qiang Zhang, Yanmin Qian, Xie Chen, Cheng Lu, Jia Liu,
• Abstract summary: FISHER is a Foundation model for multi-modal Industrial Signal compreHEnsive Representation.<n> FISHER takes the STFT sub-band as the modeling unit and adopts a teacher student SSL framework for pre-training.<n>Compared with top SSL models, FISHER showcases versatile and outstanding capabilities with a general performance gain up to 5.03%.
• Score: 49.48189836213443
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: With the rapid deployment of SCADA systems, how to effectively analyze industrial signals and detect abnormal states is an urgent need for the industry. Due to the significant heterogeneity of these signals, which we summarize as the M5 problem, previous works only focus on small sub-problems and employ specialized models, failing to utilize the synergies between modalities and the powerful scaling law. However, we argue that the M5 signals can be modeled in a unified manner due to the intrinsic similarity. As a result, we propose FISHER, a Foundation model for multi-modal Industrial Signal compreHEnsive Representation. To support arbitrary sampling rates, FISHER considers the increment of sampling rate as the concatenation of sub-band information. Specifically, FISHER takes the STFT sub-band as the modeling unit and adopts a teacher student SSL framework for pre-training. We also develop the RMIS benchmark, which evaluates the representations of M5 industrial signals on multiple health management tasks. Compared with top SSL models, FISHER showcases versatile and outstanding capabilities with a general performance gain up to 5.03%, along with much more efficient scaling curves. We also investigate the scaling law on downstream tasks and derive potential avenues for future works. FISHER is now open-sourced on https://github.com/jianganbai/FISHER

Related papers

• Unifying Multimodal Large Language Model Capabilities and Modalities via Model Merging [103.98582374569789]
  Model merging aims to combine multiple expert models into a single model, thereby reducing storage and serving costs.<n>Previous studies have primarily focused on merging visual classification models or Large Language Models (LLMs) for code and math tasks.<n>We introduce the model merging benchmark for MLLMs, which includes multiple tasks such as VQA, Geometry, Chart, OCR, and Grounding, providing both LoRA and full fine-tuning models.
  arXiv  Detail & Related papers  (2025-05-26T12:23:14Z)
• 6G WavesFM: A Foundation Model for Sensing, Communication, and Localization [6.70088826174291]
  This paper introduces a novel Wireless Foundation Model (WFM) framework, capable of supporting a wide array of communication, sensing, and localization tasks.<n>Our proposed architecture combines a shared Vision Transformer (ViT) backbone with task-specific multi-layer perceptron heads and incorporates Low-Rank Adaptation (LoRA) for parameter-efficient fine-tuning.<n>We show that our unified WFM can support diverse tasks and deliver significant gains in both performance and efficiency.
  arXiv  Detail & Related papers  (2025-04-18T22:51:35Z)
• Model Utility Law: Evaluating LLMs beyond Performance through Mechanism Interpretable Metric [99.56567010306807]
  Large Language Models (LLMs) have become indispensable across academia, industry, and daily applications.<n>One core challenge of evaluation in the large language model (LLM) era is the generalization issue.<n>We propose Model Utilization Index (MUI), a mechanism interpretability enhanced metric that complements traditional performance scores.
  arXiv  Detail & Related papers  (2025-04-10T04:09:47Z)
• ModServe: Scalable and Resource-Efficient Large Multimodal Model Serving [19.388562622309838]
  Large multimodal models (LMMs) demonstrate impressive capabilities in understanding images, videos, and audio beyond text.<n>We present the first comprehensive systems analysis of two prominent LMM architectures, decoder-only and cross-attention, across six representative open-source models.<n>We propose ModServe, a modular LMM serving system that decouples stages for independent optimization and adaptive scaling.
  arXiv  Detail & Related papers  (2025-02-02T22:10:40Z)
• LLaVA Steering: Visual Instruction Tuning with 500x Fewer Parameters through Modality Linear Representation-Steering [30.51487692912812]
  Multimodal Large Language Models (MLLMs) have significantly advanced visual tasks by integrating visual representations into large language models (LLMs)<n>We introduce Modality Linear Representation-Steering (MoReS) to achieve the goal.<n>MoReS effectively re-balances the intrinsic modalities throughout the model, where the key idea is to steer visual representations through linear transformations in the visual subspace across each model layer.
  arXiv  Detail & Related papers  (2024-12-16T21:14:11Z)
• EMR-Merging: Tuning-Free High-Performance Model Merging [55.03509900949149]
  We show that Elect, Mask & Rescale-Merging (EMR-Merging) shows outstanding performance compared to existing merging methods. EMR-Merging is tuning-free, thus requiring no data availability or any additional training while showing impressive performance.
  arXiv  Detail & Related papers  (2024-05-23T05:25:45Z)
• MMA-DFER: MultiModal Adaptation of unimodal models for Dynamic Facial Expression Recognition in-the-wild [81.32127423981426]
  Multimodal emotion recognition based on audio and video data is important for real-world applications. Recent methods have focused on exploiting advances of self-supervised learning (SSL) for pre-training of strong multimodal encoders. We propose a different perspective on the problem and investigate the advancement of multimodal DFER performance by adapting SSL-pre-trained disjoint unimodal encoders.
  arXiv  Detail & Related papers  (2024-04-13T13:39:26Z)
• A Generative Approach for Production-Aware Industrial Network Traffic Modeling [70.46446906513677]
  We investigate the network traffic data generated from a laser cutting machine deployed in a Trumpf factory in Germany. We analyze the traffic statistics, capture the dependencies between the internal states of the machine, and model the network traffic as a production state dependent process. We compare the performance of various generative models including variational autoencoder (VAE), conditional variational autoencoder (CVAE), and generative adversarial network (GAN)
  arXiv  Detail & Related papers  (2022-11-11T09:46:58Z)
• Long short-term memory networks and laglasso for bond yield forecasting: Peeping inside the black box [10.412912723760172]
  We conduct the first study of bond yield forecasting using long short-term memory (LSTM) networks. We calculate the LSTM signals through time, at selected locations in the memory cell, using sequence-to-sequence architectures.
  arXiv  Detail & Related papers  (2020-05-05T14:23:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.