Rewrite the Stars

• URL: http://arxiv.org/abs/2403.19967v1
• Date: Fri, 29 Mar 2024 04:10:07 GMT
• Title: Rewrite the Stars
• Authors: Xu Ma, Xiyang Dai, Yue Bai, Yizhou Wang, Yun Fu,
• Abstract summary: Recent studies have drawn attention to the untapped potential of the "star operation" in network design. Our study attempts to reveal the star operation's ability to map inputs into high-dimensional, non-linear feature spaces. We introduce StarNet, a simple yet powerful prototype, demonstrating impressive performance and low latency.
• Score: 70.48224347277014
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent studies have drawn attention to the untapped potential of the "star operation" (element-wise multiplication) in network design. While intuitive explanations abound, the foundational rationale behind its application remains largely unexplored. Our study attempts to reveal the star operation's ability to map inputs into high-dimensional, non-linear feature spaces -- akin to kernel tricks -- without widening the network. We further introduce StarNet, a simple yet powerful prototype, demonstrating impressive performance and low latency under compact network structure and efficient budget. Like stars in the sky, the star operation appears unremarkable but holds a vast universe of potential. Our work encourages further exploration across tasks, with codes available at https://github.com/ma-xu/Rewrite-the-Stars.

Related papers

• Intelligent Go-Explore: Standing on the Shoulders of Giant Foundation Models [5.404186221463082]
  Go-Explore is a powerful family of algorithms designed to solve hard-exploration problems. We propose Intelligent Go-Explore (IGE) which greatly extends the scope of the original Go-Explore. IGE has a human-like ability to instinctively identify how interesting or promising any new state is.
  arXiv  Detail & Related papers  (2024-05-24T01:45:27Z)
• Rotating Features for Object Discovery [74.1465486264609]
  We present Rotating Features, a generalization of complex-valued features to higher dimensions, and a new evaluation procedure for extracting objects from distributed representations. Together, these advancements enable us to scale distributed object-centric representations from simple toy to real-world data.
  arXiv  Detail & Related papers  (2023-06-01T12:16:26Z)
• ODNet: A Convolutional Neural Network for Asteroid Occultation Detection [0.36700088931938835]
  We propose to build an algorithm that will use a Convolutional Neural Network (CNN) and observations from the Unistellar network to reliably detect asteroid occultations. The algorithm is sufficiently fast and robust so we can envision incorporating onboard the eVscopes to deliver real-time results.
  arXiv  Detail & Related papers  (2022-10-28T23:53:09Z)
• Searching a High-Performance Feature Extractor for Text Recognition Network [92.12492627169108]
  We design a domain-specific search space by exploring principles for having good feature extractors. As the space is huge and complexly structured, no existing NAS algorithms can be applied. We propose a two-stage algorithm to effectively search in the space.
  arXiv  Detail & Related papers  (2022-09-27T03:49:04Z)
• SAR Despeckling Using Overcomplete Convolutional Networks [53.99620005035804]
  despeckling is an important problem in remote sensing as speckle degrades SAR images. Recent studies show that convolutional neural networks(CNNs) outperform classical despeckling methods. This study employs an overcomplete CNN architecture to focus on learning low-level features by restricting the receptive field. We show that the proposed network improves despeckling performance compared to recent despeckling methods on synthetic and real SAR images.
  arXiv  Detail & Related papers  (2022-05-31T15:55:37Z)
• On the Power of Shallow Learning [0.0]
  Given a kernel, can one find a network that realizes it? We affirmatively answer this question for fully-connected architectures, completely characterizing the space of achievable kernels. We experimentally verify our construction and demonstrate that, by just choosing the activation function, we can design a wide shallow network that mimics the generalization performance of any wide, deep, fully-connected network.
  arXiv  Detail & Related papers  (2021-06-06T17:25:33Z)
• Towards Open World Object Detection [68.79678648726416]
  ORE: Open World Object Detector is based on contrastive clustering and energy based unknown identification. We find that identifying and characterizing unknown instances helps to reduce confusion in an incremental object detection setting.
  arXiv  Detail & Related papers  (2021-03-03T18:58:18Z)
• Inductive Graph Embeddings through Locality Encodings [0.42970700836450487]
  We look at the problem of finding inductive network embeddings in large networks without domain-dependent node/edge attributes. We propose to use a set of basic predefined local encodings as the basis of a learning algorithm. This method achieves state-of-the-art performance in tasks such as role detection, link prediction and node classification.
  arXiv  Detail & Related papers  (2020-09-26T13:09:11Z)
• StarNet: towards Weakly Supervised Few-Shot Object Detection [87.80771067891418]
  We introduce StarNet - a few-shot model featuring an end-to-end differentiable non-parametric star-model detection and classification head. Through this head, the backbone is meta-trained using only image-level labels to produce good features for jointly localizing and classifying previously unseen categories of few-shot test tasks. Being a few-shot detector, StarNet does not require any bounding box annotations, neither during pre-training nor for novel classes adaptation.
  arXiv  Detail & Related papers  (2020-03-15T11:35:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.