A Modulation Layer to Increase Neural Network Robustness Against Data Quality Issues

• URL: http://arxiv.org/abs/2107.08574v4
• Date: Sat, 22 Apr 2023 19:48:08 GMT
• Title: A Modulation Layer to Increase Neural Network Robustness Against Data Quality Issues
• Authors: Mohamed Abdelhack, Jiaming Zhang, Sandhya Tripathi, Bradley A Fritz, Daniel Felsky, Michael S Avidan, Yixin Chen, Christopher R King
• Abstract summary: Data missingness and quality are common problems in machine learning, especially for high-stakes applications such as healthcare. We propose a novel neural network modification to mitigate the impacts of low quality and missing data. Our results suggest that explicitly accounting for reduced information quality with a modulating fully connected layer can enable the deployment of artificial intelligence systems in real-time applications.
• Score: 22.62510395932645
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Data missingness and quality are common problems in machine learning, especially for high-stakes applications such as healthcare. Developers often train machine learning models on carefully curated datasets using only high quality data; however, this reduces the utility of such models in production environments. We propose a novel neural network modification to mitigate the impacts of low quality and missing data which involves replacing the fixed weights of a fully-connected layer with a function of an additional input. This is inspired from neuromodulation in biological neural networks where the cortex can up- and down-regulate inputs based on their reliability and the presence of other data. In testing, with reliability scores as a modulating signal, models with modulating layers were found to be more robust against degradation of data quality, including additional missingness. These models are superior to imputation as they save on training time by completely skipping the imputation process and further allow the introduction of other data quality measures that imputation cannot handle. Our results suggest that explicitly accounting for reduced information quality with a modulating fully connected layer can enable the deployment of artificial intelligence systems in real-time applications.

Related papers

• Pseudo Replay-based Class Continual Learning for Online New Category Anomaly Detection in Additive Manufacturing [5.012204041812572]
  This paper develops a novel pseudo replay-based continual learning framework. It integrates class incremental learning and oversampling-based data generation. The effectiveness of the proposed framework is validated in three cases studies.
  arXiv  Detail & Related papers  (2023-12-05T04:43:23Z)
• Evaluation of machine learning architectures on the quantification of epistemic and aleatoric uncertainties in complex dynamical systems [0.0]
  Uncertainty Quantification (UQ) is a self assessed estimate of the model error. We examine several machine learning techniques, including both Gaussian processes and a family UQ-augmented neural networks. We evaluate UQ accuracy (distinct from model accuracy) using two metrics: the distribution of normalized residuals on validation data, and the distribution of estimated uncertainties.
  arXiv  Detail & Related papers  (2023-06-27T02:35:25Z)
• Learning From High-Dimensional Cyber-Physical Data Streams for Diagnosing Faults in Smart Grids [4.616703548353371]
  Fault diagnosis in cyber-physical power systems is affected by data quality. These systems generate massive amounts of data that overburden the system with excessive computational costs. This paper presents the effect of feature engineering on mitigating the aforementioned challenges.
  arXiv  Detail & Related papers  (2023-03-15T01:21:50Z)
• An Adversarial Active Sampling-based Data Augmentation Framework for Manufacturable Chip Design [55.62660894625669]
  Lithography modeling is a crucial problem in chip design to ensure a chip design mask is manufacturable. Recent developments in machine learning have provided alternative solutions in replacing the time-consuming lithography simulations with deep neural networks. We propose a litho-aware data augmentation framework to resolve the dilemma of limited data and improve the machine learning model performance.
  arXiv  Detail & Related papers  (2022-10-27T20:53:39Z)
• Graph Neural Networks with Trainable Adjacency Matrices for Fault Diagnosis on Multivariate Sensor Data [69.25738064847175]
  It is necessary to consider the behavior of the signals in each sensor separately, to take into account their correlation and hidden relationships with each other. The graph nodes can be represented as data from the different sensors, and the edges can display the influence of these data on each other. It was proposed to construct a graph during the training of graph neural network. This allows to train models on data where the dependencies between the sensors are not known in advance.
  arXiv  Detail & Related papers  (2022-10-20T11:03:21Z)
• Data-driven emergence of convolutional structure in neural networks [83.4920717252233]
  We show how fully-connected neural networks solving a discrimination task can learn a convolutional structure directly from their inputs. By carefully designing data models, we show that the emergence of this pattern is triggered by the non-Gaussian, higher-order local structure of the inputs.
  arXiv  Detail & Related papers  (2022-02-01T17:11:13Z)
• Reducing Catastrophic Forgetting in Self Organizing Maps with Internally-Induced Generative Replay [67.50637511633212]
  A lifelong learning agent is able to continually learn from potentially infinite streams of pattern sensory data. One major historic difficulty in building agents that adapt is that neural systems struggle to retain previously-acquired knowledge when learning from new samples. This problem is known as catastrophic forgetting (interference) and remains an unsolved problem in the domain of machine learning to this day.
  arXiv  Detail & Related papers  (2021-12-09T07:11:14Z)
• Convolutional generative adversarial imputation networks for spatio-temporal missing data in storm surge simulations [86.5302150777089]
  Generative Adversarial Imputation Nets (GANs) and GAN-based techniques have attracted attention as unsupervised machine learning methods. We name our proposed method as Con Conval Generative Adversarial Imputation Nets (Conv-GAIN)
  arXiv  Detail & Related papers  (2021-11-03T03:50:48Z)
• Non-Singular Adversarial Robustness of Neural Networks [58.731070632586594]
  Adrial robustness has become an emerging challenge for neural network owing to its over-sensitivity to small input perturbations. We formalize the notion of non-singular adversarial robustness for neural networks through the lens of joint perturbations to data inputs as well as model weights.
  arXiv  Detail & Related papers  (2021-02-23T20:59:30Z)
• STAN: Synthetic Network Traffic Generation with Generative Neural Models [10.54843182184416]
  This paper presents STAN (Synthetic network Traffic generation with Autoregressive Neural models), a tool to generate realistic synthetic network traffic datasets. Our novel neural architecture captures both temporal dependencies and dependence between attributes at any given time. We evaluate the performance of STAN in terms of the quality of data generated, by training it on both a simulated dataset and a real network traffic data set.
  arXiv  Detail & Related papers  (2020-09-27T04:20:02Z)
• On transfer learning of neural networks using bi-fidelity data for uncertainty propagation [0.0]
  We explore the application of transfer learning techniques using training data generated from both high- and low-fidelity models. In the former approach, a neural network model mapping the inputs to the outputs of interest is trained based on the low-fidelity data. The high-fidelity data is then used to adapt the parameters of the upper layer(s) of the low-fidelity network, or train a simpler neural network to map the output of the low-fidelity network to that of the high-fidelity model.
  arXiv  Detail & Related papers  (2020-02-11T15:56:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.