AutoFCL: Automatically Tuning Fully Connected Layers for Handling Small Dataset

• URL: http://arxiv.org/abs/2001.11951v4
• Date: Thu, 28 Jan 2021 17:05:06 GMT
• Title: AutoFCL: Automatically Tuning Fully Connected Layers for Handling Small Dataset
• Authors: S.H.Shabbeer Basha, Sravan Kumar Vinakota, Shiv Ram Dubey, Viswanath Pulabaigari, Snehasis Mukherjee
• Abstract summary: The proposed AutoFCL model attempts to learn the structure of FC layers of a CNN automatically using Bayesian optimization. Fine-tuning the newly learned (target-dependent) FC layers leads to state-of-the-art performance. The proposed AutoFCL method outperforms the existing methods over CalTech-101 and Oxford-102 Flowers datasets.
• Score: 13.909484906513102
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep Convolutional Neural Networks (CNN) have evolved as popular machine learning models for image classification during the past few years, due to their ability to learn the problem-specific features directly from the input images. The success of deep learning models solicits architecture engineering rather than hand-engineering the features. However, designing state-of-the-art CNN for a given task remains a non-trivial and challenging task, especially when training data size is less. To address this phenomena, transfer learning has been used as a popularly adopted technique. While transferring the learned knowledge from one task to another, fine-tuning with the target-dependent Fully Connected (FC) layers generally produces better results over the target task. In this paper, the proposed AutoFCL model attempts to learn the structure of FC layers of a CNN automatically using Bayesian optimization. To evaluate the performance of the proposed AutoFCL, we utilize five pre-trained CNN models such as VGG-16, ResNet, DenseNet, MobileNet, and NASNetMobile. The experiments are conducted on three benchmark datasets, namely CalTech-101, Oxford-102 Flowers, and UC Merced Land Use datasets. Fine-tuning the newly learned (target-dependent) FC layers leads to state-of-the-art performance, according to the experiments carried out in this research. The proposed AutoFCL method outperforms the existing methods over CalTech-101 and Oxford-102 Flowers datasets by achieving the accuracy of 94.38% and 98.89%, respectively. However, our method achieves comparable performance on the UC Merced Land Use dataset with 96.83% accuracy. The source codes of this research are available at https://github.com/shabbeersh/AutoFCL.

Related papers

• Dataset Quantization [72.61936019738076]
  We present dataset quantization (DQ), a new framework to compress large-scale datasets into small subsets. DQ is the first method that can successfully distill large-scale datasets such as ImageNet-1k with a state-of-the-art compression ratio.
  arXiv  Detail & Related papers  (2023-08-21T07:24:29Z)
• CoV-TI-Net: Transferred Initialization with Modified End Layer for COVID-19 Diagnosis [5.546855806629448]
  Transfer learning is a relatively new learning method that has been employed in many sectors to achieve good performance with fewer computations. In this research, the PyTorch pre-trained models (VGG19_bn and WideResNet -101) are applied in the MNIST dataset. The proposed model is developed and verified in the Kaggle notebook, and it reached the outstanding accuracy of 99.77% without taking a huge computational time.
  arXiv  Detail & Related papers  (2022-09-20T08:52:52Z)
• FiT: Parameter Efficient Few-shot Transfer Learning for Personalized and Federated Image Classification [47.24770508263431]
  We develop FiLM Transfer (FiT) which fulfills requirements in the image classification setting. FiT uses an automatically configured Naive Bayes classifier on top of a fixed backbone that has been pretrained on large image datasets. We show that FiT achieves better classification accuracy than the state-of-the-art Big Transfer (BiT) algorithm at low-shot and on the challenging VTAB-1k benchmark.
  arXiv  Detail & Related papers  (2022-06-17T10:17:20Z)
• Learning Rate Curriculum [75.98230528486401]
  We propose a novel curriculum learning approach termed Learning Rate Curriculum (LeRaC) LeRaC uses a different learning rate for each layer of a neural network to create a data-agnostic curriculum during the initial training epochs. We compare our approach with Curriculum by Smoothing (CBS), a state-of-the-art data-agnostic curriculum learning approach.
  arXiv  Detail & Related papers  (2022-05-18T18:57:36Z)
• Target Aware Network Architecture Search and Compression for Efficient Knowledge Transfer [9.434523476406424]
  We propose a two-stage framework called TASCNet which enables efficient knowledge transfer. TASCNet reduces the computational complexity of pre-trained CNNs over the target task by reducing both trainable parameters and FLOPs. Similar to computer vision tasks, we have also conducted experiments on Movie Review Sentiment Analysis task.
  arXiv  Detail & Related papers  (2022-05-12T09:11:00Z)
• Auto-Transfer: Learning to Route Transferrable Representations [77.30427535329571]
  We propose a novel adversarial multi-armed bandit approach which automatically learns to route source representations to appropriate target representations. We see upwards of 5% accuracy improvements compared with the state-of-the-art knowledge transfer methods.
  arXiv  Detail & Related papers  (2022-02-02T13:09:27Z)
• An Acceleration Method Based on Deep Learning and Multilinear Feature Space [0.0]
  This paper presents an alternative approach based on the Multilinear Feature Space (MFS) method resorting to transfer learning from large CNN architectures. The proposed method uses CNNs to generate feature maps, although it does not work as complexity reduction approach. Our method, named AMFC, uses the transfer learning from pre-trained CNN to reduce the classification time of new sample image, with minimal accuracy loss.
  arXiv  Detail & Related papers  (2021-10-16T23:49:12Z)
• Towards Efficient and Data Agnostic Image Classification Training Pipeline for Embedded Systems [0.0]
  This work is focusing on reviewing the latest augmentation and regularization methods for the image classification. We can achieve a reasonable performance on a variety of downstream image classification tasks without manual tuning of parameters to each particular task. Resulting models are computationally efficient and can be deployed to CPU using the OpenVINO toolkit.
  arXiv  Detail & Related papers  (2021-08-16T12:38:05Z)
• 2nd Place Scheme on Action Recognition Track of ECCV 2020 VIPriors Challenges: An Efficient Optical Flow Stream Guided Framework [57.847010327319964]
  We propose a data-efficient framework that can train the model from scratch on small datasets. Specifically, by introducing a 3D central difference convolution operation, we proposed a novel C3D neural network-based two-stream framework. It is proved that our method can achieve a promising result even without a pre-trained model on large scale datasets.
  arXiv  Detail & Related papers  (2020-08-10T09:50:28Z)
• RIFLE: Backpropagation in Depth for Deep Transfer Learning through Re-Initializing the Fully-connected LayEr [60.07531696857743]
  Fine-tuning the deep convolution neural network(CNN) using a pre-trained model helps transfer knowledge learned from larger datasets to the target task. We propose RIFLE - a strategy that deepens backpropagation in transfer learning settings. RIFLE brings meaningful updates to the weights of deep CNN layers and improves low-level feature learning.
  arXiv  Detail & Related papers  (2020-07-07T11:27:43Z)
• AutoTune: Automatically Tuning Convolutional Neural Networks for Improved Transfer Learning [13.909484906513102]
  We introduce a mechanism for automatically tuning the Convolutional Neural Networks (CNN) for improved transfer learning. The pre-trained CNN layers are tuned with the knowledge from target data using Bayesian Optimization. Experiments are conducted on three benchmark datasets, e.g., CalTech-101, CalTech-256, and Stanford Dogs.
  arXiv  Detail & Related papers  (2020-04-25T10:42:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.