Multi-path Neural Networks for On-device Multi-domain Visual Classification

• URL: http://arxiv.org/abs/2010.04904v2
• Date: Fri, 8 Jan 2021 08:02:15 GMT
• Title: Multi-path Neural Networks for On-device Multi-domain Visual Classification
• Authors: Qifei Wang, Junjie Ke, Joshua Greaves, Grace Chu, Gabriel Bender, Luciano Sbaiz, Alec Go, Andrew Howard, Feng Yang, Ming-Hsuan Yang, Jeff Gilbert, and Peyman Milanfar
• Abstract summary: This paper proposes a novel approach to automatically learn a multi-path network for multi-domain visual classification on mobile devices. The proposed multi-path network is learned from neural architecture search by applying one reinforcement learning controller for each domain to select the best path in the super-network created from a MobileNetV3-like search space. The determined multi-path model selectively shares parameters across domains in shared nodes while keeping domain-specific parameters within non-shared nodes in individual domain paths.
• Score: 55.281139434736254
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Learning multiple domains/tasks with a single model is important for improving data efficiency and lowering inference cost for numerous vision tasks, especially on resource-constrained mobile devices. However, hand-crafting a multi-domain/task model can be both tedious and challenging. This paper proposes a novel approach to automatically learn a multi-path network for multi-domain visual classification on mobile devices. The proposed multi-path network is learned from neural architecture search by applying one reinforcement learning controller for each domain to select the best path in the super-network created from a MobileNetV3-like search space. An adaptive balanced domain prioritization algorithm is proposed to balance optimizing the joint model on multiple domains simultaneously. The determined multi-path model selectively shares parameters across domains in shared nodes while keeping domain-specific parameters within non-shared nodes in individual domain paths. This approach effectively reduces the total number of parameters and FLOPS, encouraging positive knowledge transfer while mitigating negative interference across domains. Extensive evaluations on the Visual Decathlon dataset demonstrate that the proposed multi-path model achieves state-of-the-art performance in terms of accuracy, model size, and FLOPS against other approaches using MobileNetV3-like architectures. Furthermore, the proposed method improves average accuracy over learning single-domain models individually, and reduces the total number of parameters and FLOPS by 78% and 32% respectively, compared to the approach that simply bundles single-domain models for multi-domain learning.

Related papers

• Investigating the potential of Sparse Mixtures-of-Experts for multi-domain neural machine translation [59.41178047749177]
  We focus on multi-domain Neural Machine Translation, with the goal of developing efficient models which can handle data from various domains seen during training and are robust to domains unseen during training. We hypothesize that Sparse Mixture-of-Experts (SMoE) models are a good fit for this task, as they enable efficient model scaling. We conduct a series of experiments aimed at validating the utility of SMoE for the multi-domain scenario, and find that a straightforward width scaling of Transformer is a simpler and surprisingly more efficient approach in practice, and reaches the same performance level as SMoE.
  arXiv  Detail & Related papers  (2024-07-01T09:45:22Z)
• M3BAT: Unsupervised Domain Adaptation for Multimodal Mobile Sensing with Multi-Branch Adversarial Training [5.128670847334003]
  multimodal mobile sensing has been used extensively for inferences regarding health and well being, behavior, and context. The distribution of data in the training set differs from the distribution of data in the real world, the deployment environment. We propose M3BAT, an unsupervised domain adaptation for multimodal mobile sensing with multi-branch adversarial training.
  arXiv  Detail & Related papers  (2024-04-26T13:09:35Z)
• SepRep-Net: Multi-source Free Domain Adaptation via Model Separation And Reparameterization [75.74369886582394]
  We propose a novel framework called SepRep-Net to tackle multi-source free domain adaptation. SepRep-Net reassembled multiple existing models to a unified network, while maintaining separate pathways (Separation) SepRep-Net is characterized by 1) effectiveness: competitive performance on the target domain, 2) efficiency: low computational costs, and 3) generalizability: maintaining more source knowledge than existing solutions.
  arXiv  Detail & Related papers  (2024-02-13T06:35:00Z)
• Multi-Domain Learning with Modulation Adapters [33.54630534228469]
  Multi-domain learning aims to handle related tasks, such as image classification across multiple domains, simultaneously. Modulation Adapters update the convolutional weights of the model in a multiplicative manner for each task. Our approach yields excellent results, with accuracies that are comparable to or better than those of existing state-of-the-art approaches.
  arXiv  Detail & Related papers  (2023-07-17T14:40:16Z)
• DynaGAN: Dynamic Few-shot Adaptation of GANs to Multiple Domains [26.95350186287616]
  Few-shot domain adaptation to multiple domains aims to learn a complex image distribution across multiple domains from a few training images. We propose DynaGAN, a novel few-shot domain-adaptation method for multiple target domains.
  arXiv  Detail & Related papers  (2022-11-26T12:46:40Z)
• Multi-Prompt Alignment for Multi-Source Unsupervised Domain Adaptation [86.02485817444216]
  We introduce Multi-Prompt Alignment (MPA), a simple yet efficient framework for multi-source UDA. MPA denoises the learned prompts through an auto-encoding process and aligns them by maximizing the agreement of all the reconstructed prompts. Experiments show that MPA achieves state-of-the-art results on three popular datasets with an impressive average accuracy of 54.1% on DomainNet.
  arXiv  Detail & Related papers  (2022-09-30T03:40:10Z)
• Multi-Source Domain Adaptation for Object Detection [52.87890831055648]
  We propose a unified Faster R-CNN based framework, termed Divide-and-Merge Spindle Network (DMSN) DMSN can simultaneously enhance domain innative and preserve discriminative power. We develop a novel pseudo learning algorithm to approximate optimal parameters of pseudo target subset.
  arXiv  Detail & Related papers  (2021-06-30T03:17:20Z)
• Not all domains are equally complex: Adaptive Multi-Domain Learning [98.25886129591974]
  We propose an adaptive parameterization approach to deep neural networks for multi-domain learning. The proposed approach performs on par with the original approach while reducing by far the number of parameters.
  arXiv  Detail & Related papers  (2020-03-25T17:16:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.