DARTS-: Robustly Stepping out of Performance Collapse Without Indicators

• URL: http://arxiv.org/abs/2009.01027v2
• Date: Fri, 15 Jan 2021 07:58:11 GMT
• Title: DARTS-: Robustly Stepping out of Performance Collapse Without Indicators
• Authors: Xiangxiang Chu, Xiaoxing Wang, Bo Zhang, Shun Lu, Xiaolin Wei, Junchi Yan
• Abstract summary: Differentiable architecture search suffers from long-standing performance instability. indicators such as Hessian eigenvalues are proposed as a signal to stop searching before the performance collapses. In this paper, we undertake a more subtle and direct approach to resolve the collapse.
• Score: 74.21019737169675
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Despite the fast development of differentiable architecture search (DARTS), it suffers from long-standing performance instability, which extremely limits its application. Existing robustifying methods draw clues from the resulting deteriorated behavior instead of finding out its causing factor. Various indicators such as Hessian eigenvalues are proposed as a signal to stop searching before the performance collapses. However, these indicator-based methods tend to easily reject good architectures if the thresholds are inappropriately set, let alone the searching is intrinsically noisy. In this paper, we undertake a more subtle and direct approach to resolve the collapse. We first demonstrate that skip connections have a clear advantage over other candidate operations, where it can easily recover from a disadvantageous state and become dominant. We conjecture that this privilege is causing degenerated performance. Therefore, we propose to factor out this benefit with an auxiliary skip connection, ensuring a fairer competition for all operations. We call this approach DARTS-. Extensive experiments on various datasets verify that it can substantially improve robustness. Our code is available at https://github.com/Meituan-AutoML/DARTS- .

Related papers

• Simple Ingredients for Offline Reinforcement Learning [86.1988266277766]
  offline reinforcement learning algorithms have proven effective on datasets highly connected to the target downstream task. We show that existing methods struggle with diverse data: their performance considerably deteriorates as data collected for related but different tasks is simply added to the offline buffer. We show that scale, more than algorithmic considerations, is the key factor influencing performance.
  arXiv  Detail & Related papers  (2024-03-19T18:57:53Z)
• A Discrepancy Aware Framework for Robust Anomaly Detection [51.710249807397695]
  We present a Discrepancy Aware Framework (DAF), which demonstrates robust performance consistently with simple and cheap strategies. Our method leverages an appearance-agnostic cue to guide the decoder in identifying defects, thereby alleviating its reliance on synthetic appearance. Under the simple synthesis strategies, it outperforms existing methods by a large margin. Furthermore, it also achieves the state-of-the-art localization performance.
  arXiv  Detail & Related papers  (2023-10-11T15:21:40Z)
• Re-Evaluating LiDAR Scene Flow for Autonomous Driving [80.37947791534985]
  Popular benchmarks for self-supervised LiDAR scene flow have unrealistic rates of dynamic motion, unrealistic correspondences, and unrealistic sampling patterns. We evaluate a suite of top methods on a suite of real-world datasets. We show that despite the emphasis placed on learning, most performance gains are caused by pre- and post-processing steps.
  arXiv  Detail & Related papers  (2023-04-04T22:45:50Z)
• Operation-level Progressive Differentiable Architecture Search [19.214462477848535]
  We propose operation-level progressive differentiable neural architecture search (OPP-DARTS) to avoid skip connections aggregation. Our method's performance on CIFAR-10 is superior to the architecture found by standard DARTS.
  arXiv  Detail & Related papers  (2023-02-11T09:18:01Z)
• ReAct: Temporal Action Detection with Relational Queries [84.76646044604055]
  This work aims at advancing temporal action detection (TAD) using an encoder-decoder framework with action queries. We first propose a relational attention mechanism in the decoder, which guides the attention among queries based on their relations. Lastly, we propose to predict the localization quality of each action query at inference in order to distinguish high-quality queries.
  arXiv  Detail & Related papers  (2022-07-14T17:46:37Z)
• Enhancing the Robustness, Efficiency, and Diversity of Differentiable Architecture Search [25.112048502327738]
  Differentiable architecture search (DARTS) has attracted much attention due to its simplicity and significant improvement in efficiency. Many works attempt to restrict the accumulation of skip connections by indicators or manual design. We suggest a more subtle and direct approach that removes skip connections from the operation space.
  arXiv  Detail & Related papers  (2022-04-10T13:25:36Z)
• Efficient Person Search: An Anchor-Free Approach [86.45858994806471]
  Person search aims to simultaneously localize and identify a query person from realistic, uncropped images. To achieve this goal, state-of-the-art models typically add a re-id branch upon two-stage detectors like Faster R-CNN. In this work, we present an anchor-free approach to efficiently tackling this challenging task, by introducing the following dedicated designs.
  arXiv  Detail & Related papers  (2021-09-01T07:01:33Z)
• Leaning Compact and Representative Features for Cross-Modality Person Re-Identification [18.06382007908855]
  This paper pays close attention to the cross-modality visible-infrared person re-identification (VI Re-ID) task. The proposed method is superior to the other most advanced methods in terms of impressive performance.
  arXiv  Detail & Related papers  (2021-03-26T01:53:16Z)
• Noisy Differentiable Architecture Search [13.154295073267367]
  Differentiable Architecture Search (DARTS) has now become one of the mainstream paradigms of neural architecture search. It largely suffers from the well-known performance collapse issue due to the aggregation of skip connections. We propose to inject unbiased random noise to impede the flow.
  arXiv  Detail & Related papers  (2020-05-07T15:53:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.