DIRA: Dynamic Domain Incremental Regularised Adaptation

• URL: http://arxiv.org/abs/2205.00147v5
• Date: Wed, 3 Jan 2024 01:13:25 GMT
• Title: DIRA: Dynamic Domain Incremental Regularised Adaptation
• Authors: Abanoub Ghobrial, Xuan Zheng, Darryl Hond, Hamid Asgari, Kerstin Eder
• Abstract summary: We introduce Dynamic Incremental Regularised Adaptation (DIRA) for dynamic operational domain adaptions of Deep Neural Network (DNN) DIRA improves on the problem of forgetting and achieves strong gains in performance when retraining using a few samples from the target domain. Our approach shows improvements on different image classification benchmarks aimed at evaluating robustness to distribution shifts.
• Score: 2.227417514684251
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Autonomous systems (AS) often use Deep Neural Network (DNN) classifiers to allow them to operate in complex, high-dimensional, non-linear, and dynamically changing environments. Due to the complexity of these environments, DNN classifiers may output misclassifications during operation when they face domains not identified during development. Removing a system from operation for retraining becomes impractical as the number of such AS increases. To increase AS reliability and overcome this limitation, DNN classifiers need to have the ability to adapt during operation when faced with different operational domains using a few samples (e.g. 2 to 100 samples). However, retraining DNNs on a few samples is known to cause catastrophic forgetting and poor generalisation. In this paper, we introduce Dynamic Incremental Regularised Adaptation (DIRA), an approach for dynamic operational domain adaption of DNNs using regularisation techniques. We show that DIRA improves on the problem of forgetting and achieves strong gains in performance when retraining using a few samples from the target domain. Our approach shows improvements on different image classification benchmarks aimed at evaluating robustness to distribution shifts (e.g.CIFAR-10C/100C, ImageNet-C), and produces state-of-the-art performance in comparison with other methods from the literature.

Related papers

• Evaluating the Robustness of Deep-Learning Algorithm-Selection Models by Evolving Adversarial Instances [0.16874375111244325]
  Deep convolutional networks (DNN) are increasingly being used to perform algorithm-selection in neural domains. adversarial samples are successfully generated from up to 56% of the original instances depending on the dataset. We use an evolutionary algorithm (EA) to find perturbations of instances from two existing benchmarks for online bin packing that cause trained DRNs to misclassify.
  arXiv  Detail & Related papers  (2024-06-24T12:48:44Z)
• Optimization Guarantees of Unfolded ISTA and ADMM Networks With Smooth Soft-Thresholding [57.71603937699949]
  We study optimization guarantees, i.e., achieving near-zero training loss with the increase in the number of learning epochs. We show that the threshold on the number of training samples increases with the increase in the network width.
  arXiv  Detail & Related papers  (2023-09-12T13:03:47Z)
• Adaptive Growth: Real-time CNN Layer Expansion [0.0]
  This research presents a new algorithm that allows the convolutional layer of a Convolutional Neural Network (CNN) to dynamically evolve based on data input. Instead of a rigid architecture, our approach iteratively introduces kernels to the convolutional layer, gauging its real-time response to varying data. Remarkably, our unsupervised method has outstripped its supervised counterparts across diverse datasets.
  arXiv  Detail & Related papers  (2023-09-06T14:43:58Z)
• Intelligence Processing Units Accelerate Neuromorphic Learning [52.952192990802345]
  Spiking neural networks (SNNs) have achieved orders of magnitude improvement in terms of energy consumption and latency. We present an IPU-optimized release of our custom SNN Python package, snnTorch.
  arXiv  Detail & Related papers  (2022-11-19T15:44:08Z)
• Continual BatchNorm Adaptation (CBNA) for Semantic Segmentation [39.99513327031499]
  Environment perception in autonomous driving vehicles often heavily relies on deep neural networks (DNNs) In this work, we expand a source-free UDA approach to a continual and therefore online-capable UDA on a single-image basis for semantic segmentation. Our method Continual BatchNorm Adaptation (CBNA) modifies the source domain statistics in the batch normalization layers, using target domain images in an unsupervised fashion, which yields consistent performance improvements during inference.
  arXiv  Detail & Related papers  (2022-03-02T12:55:10Z)
• TELESTO: A Graph Neural Network Model for Anomaly Classification in Cloud Services [77.454688257702]
  Machine learning (ML) and artificial intelligence (AI) are applied on IT system operation and maintenance. One direction aims at the recognition of re-occurring anomaly types to enable remediation automation. We propose a method that is invariant to dimensionality changes of given data.
  arXiv  Detail & Related papers  (2021-02-25T14:24:49Z)
• Style Normalization and Restitution for DomainGeneralization and Adaptation [88.86865069583149]
  An effective domain generalizable model is expected to learn feature representations that are both generalizable and discriminative. In this paper, we design a novel Style Normalization and Restitution module (SNR) to ensure both high generalization and discrimination capability of the networks.
  arXiv  Detail & Related papers  (2021-01-03T09:01:39Z)
• DeepRepair: Style-Guided Repairing for DNNs in the Real-world Operational Environment [27.316150020006916]
  We propose a style-guided data augmentation for repairing Deep Neural Networks (DNNs) in the operational environment. We propose a style transfer method to learn and introduce the unknown failure patterns within the failure data into the training data via data augmentation.
  arXiv  Detail & Related papers  (2020-11-19T15:09:44Z)
• SRDCNN: Strongly Regularized Deep Convolution Neural Network Architecture for Time-series Sensor Signal Classification Tasks [4.950427992960756]
  We present SRDCNN: Strongly Regularized Deep Convolution Neural Network (DCNN) based deep architecture to perform time series classification tasks. The novelty of the proposed approach is that the network weights are regularized by both L1 and L2 norm penalties.
  arXiv  Detail & Related papers  (2020-07-14T08:42:39Z)
• GraN: An Efficient Gradient-Norm Based Detector for Adversarial and Misclassified Examples [77.99182201815763]
  Deep neural networks (DNNs) are vulnerable to adversarial examples and other data perturbations. GraN is a time- and parameter-efficient method that is easily adaptable to any DNN. GraN achieves state-of-the-art performance on numerous problem set-ups.
  arXiv  Detail & Related papers  (2020-04-20T10:09:27Z)
• Unsupervised Domain Adaptation in Person re-ID via k-Reciprocal Clustering and Large-Scale Heterogeneous Environment Synthesis [76.46004354572956]
  We introduce an unsupervised domain adaptation approach for person re-identification. Experimental results show that the proposed ktCUDA and SHRED approach achieves an average improvement of +5.7 mAP in re-identification performance.
  arXiv  Detail & Related papers  (2020-01-14T17:43:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.