ViTNF: Leveraging Neural Fields to Boost Vision Transformers in Generalized Category Discovery

• URL: http://arxiv.org/abs/2506.02367v1
• Date: Tue, 03 Jun 2025 02:06:01 GMT
• Title: ViTNF: Leveraging Neural Fields to Boost Vision Transformers in Generalized Category Discovery
• Abstract summary: Generalized category discovery (GCD) is a popular task in open-world recognition.<n>By leveraging pre-training, meta-training, and fine-tuning, ViT achieves excellent few-shot learning capabilities.<n>This paper proposes a new architecture by replacing the head with a neural field-based one.
• Score: 1.1970409518725493
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Generalized category discovery (GCD) is a highly popular task in open-world recognition, aiming to identify unknown class samples using known class data. By leveraging pre-training, meta-training, and fine-tuning, ViT achieves excellent few-shot learning capabilities. Its MLP head is a feedforward network, trained synchronously with the entire network in the same process, increasing the training cost and difficulty without fully leveraging the power of the feature extractor. This paper proposes a new architecture by replacing the MLP head with a neural field-based one. We first present a new static neural field function to describe the activity distribution of the neural field and then use two static neural field functions to build an efficient few-shot classifier. This neural field-based (NF) classifier consists of two coupled static neural fields. It stores the feature information of support samples by its elementary field, the known categories by its high-level field, and the category information of support samples by its cross-field connections. We replace the MLP head with the proposed NF classifier, resulting in a novel architecture ViTNF, and simplify the three-stage training mode by pre-training the feature extractor on source tasks and training the NF classifier with support samples in meta-testing separately, significantly reducing ViT's demand for training samples and the difficulty of model training. To enhance the model's capability in identifying new categories, we provide an effective algorithm to determine the lateral interaction scale of the elementary field. Experimental results demonstrate that our model surpasses existing state-of-the-art methods on CIFAR-100, ImageNet-100, CUB-200, and Standard Cars, achieving dramatic accuracy improvements of 19\% and 16\% in new and all classes, respectively, indicating a notable advantage in GCD.

Related papers

• CATFormer: When Continual Learning Meets Spiking Transformers With Dynamic Thresholds [0.0]
  Existing spiking neural networks (SNNs) for class-incremental learning (CIL) suffer a sharp performance drop as tasks accumulate.<n>We here introduce CATFormer, a scalable framework that overcomes this limitation.<n>We observe that the key to preventing forgetting in SNNs lies not only in synaptic plasticity but also in modulating neuronal excitability.
  arXiv  Detail & Related papers  (2026-03-16T12:20:23Z)
• One-for-All Model Initialization with Frequency-Domain Knowledge [33.90471312176688]
  FRONT (FRequency dOmain kNowledge Transfer) is a novel framework that uses the Cosine Transform (DCT) to isolate the low-frequency "learngene"<n> FRONT achieves the state-of-the-art performance, accelerates convergence by up to 15 times in vision tasks, and reduces training FLOPs by an average of 40.5% in language tasks.
  arXiv  Detail & Related papers  (2026-03-08T08:25:49Z)
• Self-Supervised Learning via Flow-Guided Neural Operator on Time-Series Data [57.85958428020496]
  Flow-Guided Neural Operator (FGNO) is a novel framework combining operator learning with flow matching for SSL training.<n>FGNO learns mappings in functional spaces by using Short-Time Fourier Transform to unify different time resolutions.<n>Unlike prior generative SSL methods that use noisy inputs during inference, we propose using clean inputs for representation extraction while learning representations with noise.
  arXiv  Detail & Related papers  (2026-02-12T18:54:57Z)
• A Hybrid Multi-Well Hopfield-CNN with Feature Extraction and K-Means for MNIST Classification [0.0]
  This study presents a hybrid model for classifying handwritten digits in the MNIST dataset.<n>It combines convolutional neural networks (CNNs) with a multi-well Hopfield network.<n>The model achieves a high test accuracy of 99.2% on 10,000 MNIST images.
  arXiv  Detail & Related papers  (2025-07-11T17:26:06Z)
• Neural Field Classifiers via Target Encoding and Classification Loss [38.28073206783851]
  We propose a novel Neural Field (NFC) framework which formulates existing neural field methods as classification tasks. We demonstrate the impressive effectiveness of NFC at the nearly free extra computational costs.
  arXiv  Detail & Related papers  (2024-03-02T01:20:59Z)
• Activate and Reject: Towards Safe Domain Generalization under Category Shift [71.95548187205736]
  We study a practical problem of Domain Generalization under Category Shift (DGCS) It aims to simultaneously detect unknown-class samples and classify known-class samples in the target domains. Compared to prior DG works, we face two new challenges: 1) how to learn the concept of unknown'' during training with only source known-class samples, and 2) how to adapt the source-trained model to unseen environments.
  arXiv  Detail & Related papers  (2023-10-07T07:53:12Z)
• Exploring Learned Representations of Neural Networks with Principal Component Analysis [1.0923877073891446]
  In certain layers, as little as 20% of the intermediate feature-space variance is necessary for high-accuracy classification. We relate our findings to neural collapse and provide partial evidence for the related phenomenon of intermediate neural collapse.
  arXiv  Detail & Related papers  (2023-09-27T00:18:25Z)
• FaFCNN: A General Disease Classification Framework Based on Feature Fusion Neural Networks [4.097623533226476]
  We propose the Feature-aware Fusion Correlation Neural Network (FaFCNN), which introduces a feature-aware interaction module and a feature alignment module based on domain adversarial learning. The experimental results show that training using augmented features obtained by pre-training gradient boosting decision tree yields more performance gains than random-forest based methods.
  arXiv  Detail & Related papers  (2023-07-24T04:23:08Z)
• ELFIS: Expert Learning for Fine-grained Image Recognition Using Subsets [6.632855264705276]
  We propose ELFIS, an expert learning framework for Fine-Grained Visual Recognition. A set of neural networks-based experts are trained focusing on the meta-categories and are integrated into a multi-task framework. Experiments show improvements in the SoTA FGVR benchmarks of up to +1.3% of accuracy using both CNNs and transformer-based networks.
  arXiv  Detail & Related papers  (2023-03-16T12:45:19Z)
• Low-Resource Music Genre Classification with Cross-Modal Neural Model Reprogramming [129.4950757742912]
  We introduce a novel method for leveraging pre-trained models for low-resource (music) classification based on the concept of Neural Model Reprogramming (NMR) NMR aims at re-purposing a pre-trained model from a source domain to a target domain by modifying the input of a frozen pre-trained model. Experimental results suggest that a neural model pre-trained on large-scale datasets can successfully perform music genre classification by using this reprogramming method.
  arXiv  Detail & Related papers  (2022-11-02T17:38:33Z)
• On Feature Learning in Neural Networks with Global Convergence Guarantees [49.870593940818715]
  We study the optimization of wide neural networks (NNs) via gradient flow (GF) We show that when the input dimension is no less than the size of the training set, the training loss converges to zero at a linear rate under GF. We also show empirically that, unlike in the Neural Tangent Kernel (NTK) regime, our multi-layer model exhibits feature learning and can achieve better generalization performance than its NTK counterpart.
  arXiv  Detail & Related papers  (2022-04-22T15:56:43Z)
• Do We Really Need a Learnable Classifier at the End of Deep Neural Network? [118.18554882199676]
  We study the potential of learning a neural network for classification with the classifier randomly as an ETF and fixed during training. Our experimental results show that our method is able to achieve similar performances on image classification for balanced datasets.
  arXiv  Detail & Related papers  (2022-03-17T04:34:28Z)
• Towards Open-World Feature Extrapolation: An Inductive Graph Learning Approach [80.8446673089281]
  We propose a new learning paradigm with graph representation and learning. Our framework contains two modules: 1) a backbone network (e.g., feedforward neural nets) as a lower model takes features as input and outputs predicted labels; 2) a graph neural network as an upper model learns to extrapolate embeddings for new features via message passing over a feature-data graph built from observed data.
  arXiv  Detail & Related papers  (2021-10-09T09:02:45Z)
• No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Data [78.69828864672978]
  A central challenge in training classification models in the real-world federated system is learning with non-IID data. We propose a novel and simple algorithm called Virtual Representations (CCVR), which adjusts the classifier using virtual representations sampled from an approximated ssian mixture model. Experimental results demonstrate that CCVR state-of-the-art performance on popular federated learning benchmarks including CIFAR-10, CIFAR-100, and CINIC-10.
  arXiv  Detail & Related papers  (2021-06-09T12:02:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.