Unsupervised Multimodal Fusion of In-process Sensor Data for Advanced Manufacturing Process Monitoring

• URL: http://arxiv.org/abs/2410.22558v1
• Date: Tue, 29 Oct 2024 21:52:04 GMT
• Title: Unsupervised Multimodal Fusion of In-process Sensor Data for Advanced Manufacturing Process Monitoring
• Authors: Matthew McKinney, Anthony Garland, Dale Cillessen, Jesse Adamczyk, Dan Bolintineanu, Michael Heiden, Elliott Fowler, Brad L. Boyce,
• Abstract summary: This paper presents a novel approach to multimodal sensor data fusion in manufacturing processes. We leverage contrastive learning techniques to correlate different data modalities without the need for labeled data. Our approach facilitates downstream tasks such as process control, anomaly detection, and quality assurance.
• Score: 0.0
• License:
• Abstract: Effective monitoring of manufacturing processes is crucial for maintaining product quality and operational efficiency. Modern manufacturing environments generate vast amounts of multimodal data, including visual imagery from various perspectives and resolutions, hyperspectral data, and machine health monitoring information such as actuator positions, accelerometer readings, and temperature measurements. However, interpreting this complex, high-dimensional data presents significant challenges, particularly when labeled datasets are unavailable. This paper presents a novel approach to multimodal sensor data fusion in manufacturing processes, inspired by the Contrastive Language-Image Pre-training (CLIP) model. We leverage contrastive learning techniques to correlate different data modalities without the need for labeled data, developing encoders for five distinct modalities: visual imagery, audio signals, laser position (x and y coordinates), and laser power measurements. By compressing these high-dimensional datasets into low-dimensional representational spaces, our approach facilitates downstream tasks such as process control, anomaly detection, and quality assurance. We evaluate the effectiveness of our approach through experiments, demonstrating its potential to enhance process monitoring capabilities in advanced manufacturing systems. This research contributes to smart manufacturing by providing a flexible, scalable framework for multimodal data fusion that can adapt to diverse manufacturing environments and sensor configurations.

Related papers

• JEMA: A Joint Embedding Framework for Scalable Co-Learning with Multimodal Alignment [0.0]
  JEMA (Joint Embedding with Multimodal Alignment) is a novel co-learning framework tailored for laser metal deposition (LMD) We report an 8% increase in performance in multimodal settings and a 1% improvement in unimodal settings compared to supervised contrastive learning. Our framework lays the foundation for integrating multisensor data with metadata, enabling diverse downstream tasks within the LMD domain and beyond.
  arXiv  Detail & Related papers  (2024-10-31T14:42:26Z)
• Scaling Wearable Foundation Models [54.93979158708164]
  We investigate the scaling properties of sensor foundation models across compute, data, and model size. Using a dataset of up to 40 million hours of in-situ heart rate, heart rate variability, electrodermal activity, accelerometer, skin temperature, and altimeter per-minute data from over 165,000 people, we create LSM. Our results establish the scaling laws of LSM for tasks such as imputation, extrapolation, both across time and sensor modalities.
  arXiv  Detail & Related papers  (2024-10-17T15:08:21Z)
• A Predictive Model Based on Transformer with Statistical Feature Embedding in Manufacturing Sensor Dataset [2.07180164747172]
  This study proposes a novel predictive model based on the Transformer, utilizing statistical feature embedding and window positional encoding. The model's performance is evaluated in two problems: fault detection and virtual metrology, showing superior results compared to baseline models. The results support the model's applicability across various manufacturing industries, demonstrating its potential for enhancing process management and yield.
  arXiv  Detail & Related papers  (2024-07-09T08:59:27Z)
• Sparse Attention-driven Quality Prediction for Production Process Optimization in Digital Twins [53.70191138561039]
  We propose to deploy a digital twin of the production line by encoding its operational logic in a data-driven approach. We adopt a quality prediction model for production process based on self-attention-enabled temporal convolutional neural networks. Our operation experiments on a specific tobacco shredding line demonstrate that the proposed digital twin-based production process optimization method fosters seamless integration between virtual and real production lines.
  arXiv  Detail & Related papers  (2024-05-20T09:28:23Z)
• A Systematic Review of Available Datasets in Additive Manufacturing [56.684125592242445]
  In-situ monitoring incorporating visual and other sensor technologies allows the collection of extensive datasets during the Additive Manufacturing process. These datasets have potential for determining the quality of the manufactured output and the detection of defects through the use of Machine Learning. This systematic review investigates the availability of open image-based datasets originating from AM processes that align with a number of pre-defined selection criteria.
  arXiv  Detail & Related papers  (2024-01-27T16:13:32Z)
• Non-contact Multimodal Indoor Human Monitoring Systems: A Survey [4.048305170077075]
  We present a comprehensive survey of multimodal approaches for indoor human monitoring systems. Our survey primarily highlights non-contact technologies, particularly cameras and radio devices. We emphasize their critical role in enhancing the quality of elderly care, offering valuable insights into the development of non-contact monitoring solutions.
  arXiv  Detail & Related papers  (2023-12-11T14:57:12Z)
• StableLLaVA: Enhanced Visual Instruction Tuning with Synthesized Image-Dialogue Data [129.92449761766025]
  We propose a novel data collection methodology that synchronously synthesizes images and dialogues for visual instruction tuning. This approach harnesses the power of generative models, marrying the abilities of ChatGPT and text-to-image generative models. Our research includes comprehensive experiments conducted on various datasets.
  arXiv  Detail & Related papers  (2023-08-20T12:43:52Z)
• DeepTimeAnomalyViz: A Tool for Visualizing and Post-processing Deep Learning Anomaly Detection Results for Industrial Time-Series [88.12892448747291]
  We introduce the DeTAVIZ interface, which is a web browser based visualization tool for quick exploration and assessment of feasibility of DL based anomaly detection in a given problem. DeTAVIZ allows the user to easily and quickly iterate through multiple post processing options and compare different models, and allows for manual optimisation towards a chosen metric.
  arXiv  Detail & Related papers  (2021-09-21T10:38:26Z)
• Auto-encoder based Model for High-dimensional Imbalanced Industrial Data [6.339700878842761]
  We introduce a variance weighted multi-headed auto-encoder classification model that fits well into the high-dimensional and highly imbalanced data. The model also simultaneously predicts multiple outputs by exploiting output-supervised representation learning and multi-task weighting.
  arXiv  Detail & Related papers  (2021-08-04T14:34:59Z)
• Modeling and Optimizing Laser-Induced Graphene [59.8912133964006]
  We provide datasets that describe the optimization of the production of laser-induced graphene. We pose three challenges based on the datasets we provide. We present illustrative results, along with the code used to generate them, as a starting point for interested users.
  arXiv  Detail & Related papers  (2021-07-29T18:08:24Z)
• Machine Learning based Indicators to Enhance Process Monitoring by Pattern Recognition [0.4893345190925177]
  We propose a novel framework for machine learning based indicators combining pattern type and intensity. In a case-study from semiconductor industry, our framework goes beyond conventional process control and achieves high quality experimental results.
  arXiv  Detail & Related papers  (2021-03-24T10:13:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.