On-Sensor Binarized Fully Convolutional Neural Network with A Pixel Processor Array

• URL: http://arxiv.org/abs/2202.00836v1
• Date: Wed, 2 Feb 2022 01:18:40 GMT
• Title: On-Sensor Binarized Fully Convolutional Neural Network with A Pixel Processor Array
• Authors: Yanan Liu, Laurie Bose, Yao Lu, Piotr Dudek, Walterio Mayol-Cuevas
• Abstract summary: This work presents a method to implement fully convolutional neural networks (FCNs) on Pixel Processor Array ( PPA) sensors. We design and train binarized FCN for both binary weights and activations using batchnorm, group convolution, and learnable threshold for binarization. We demonstrate the first implementation of an FCN on a PPA device, performing three convolution layers entirely in the pixel-level processors.
• Score: 17.4097919720973
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This work presents a method to implement fully convolutional neural networks (FCNs) on Pixel Processor Array (PPA) sensors, and demonstrates coarse segmentation and object localisation tasks. We design and train binarized FCN for both binary weights and activations using batchnorm, group convolution, and learnable threshold for binarization, producing networks small enough to be embedded on the focal plane of the PPA, with limited local memory resources, and using parallel elementary add/subtract, shifting, and bit operations only. We demonstrate the first implementation of an FCN on a PPA device, performing three convolution layers entirely in the pixel-level processors. We use this architecture to demonstrate inference generating heat maps for object segmentation and localisation at over 280 FPS using the SCAMP-5 PPA vision chip.

Related papers

• Tiled Bit Networks: Sub-Bit Neural Network Compression Through Reuse of Learnable Binary Vectors [4.95475852994362]
  We propose a new form of quantization to tile neural network layers with sequences of bits to achieve sub-bit compression of binary-weighted neural networks. We employ the approach to both fully-connected and convolutional layers, which make up the breadth of space in most neural architectures.
  arXiv  Detail & Related papers  (2024-07-16T15:55:38Z)
• BDC-Occ: Binarized Deep Convolution Unit For Binarized Occupancy Network [55.21288428359509]
  Existing 3D occupancy networks demand significant hardware resources, hindering the deployment of edge devices. We propose a novel binarized deep convolution (BDC) unit that effectively enhances performance while increasing the number of binarized convolutional layers. Our BDC-Occ model is created by applying the proposed BDC unit to binarize the existing 3D occupancy networks.
  arXiv  Detail & Related papers  (2024-05-27T10:44:05Z)
• Moving Object Proposals with Deep Learned Optical Flow for Video Object Segmentation [1.551271936792451]
  We propose a state of art architecture of neural networks to get the moving object proposals (MOP) We first train an unsupervised convolutional neural network (UnFlow) to generate optical flow estimation. Then we render the output of optical flow net to a fully convolutional SegNet model.
  arXiv  Detail & Related papers  (2024-02-14T01:13:55Z)
• Signal Processing for Implicit Neural Representations [80.38097216996164]
  Implicit Neural Representations (INRs) encode continuous multi-media data via multi-layer perceptrons. Existing works manipulate such continuous representations via processing on their discretized instance. We propose an implicit neural signal processing network, dubbed INSP-Net, via differential operators on INR.
  arXiv  Detail & Related papers  (2022-10-17T06:29:07Z)
• Bandwidth-efficient distributed neural network architectures with application to body sensor networks [73.02174868813475]
  This paper describes a conceptual design methodology to design distributed neural network architectures. We show that the proposed framework enables up to a factor 20 in bandwidth reduction with minimal loss. While the application focus of this paper is on wearable brain-computer interfaces, the proposed methodology can be applied in other sensor network-like applications as well.
  arXiv  Detail & Related papers  (2022-10-14T12:35:32Z)
• Two-Stream Networks for Object Segmentation in Videos [83.1383102535413]
  We present a Two-Stream Network (TSN) to segment the seen pixels based on their pixellevel memory retrieval. A holistic understanding of the instance is obtained with dynamic segmentation heads conditioned on the features of the target instance. The compact instance stream effectively improves the segmentation accuracy of the unseen pixels, while fusing two streams with the adaptive routing map leads to an overall performance boost.
  arXiv  Detail & Related papers  (2022-08-08T10:22:42Z)
• Trident Pyramid Networks: The importance of processing at the feature pyramid level for better object detection [50.008529403150206]
  We present a new core architecture called Trident Pyramid Network (TPN) TPN allows for a deeper design and for a better balance between communication-based processing and self-processing. We show consistent improvements when using our TPN core on the object detection benchmark, outperforming the popular BiFPN baseline by 1.5 AP.
  arXiv  Detail & Related papers  (2021-10-08T09:59:59Z)
• Quantized Neural Networks via {-1, +1} Encoding Decomposition and Acceleration [83.84684675841167]
  We propose a novel encoding scheme using -1, +1 to decompose quantized neural networks (QNNs) into multi-branch binary networks. We validate the effectiveness of our method on large-scale image classification, object detection, and semantic segmentation tasks.
  arXiv  Detail & Related papers  (2021-06-18T03:11:15Z)
• Efficient 3D Point Cloud Feature Learning for Large-Scale Place Recognition [21.818744369503197]
  We develop an efficient point cloud learning network ( EPC-Net) to form a global descriptor for visual place recognition. Our proposed method can achieve state-of-the-art performance with lower parameters, FLOPs, and runtime per frame.
  arXiv  Detail & Related papers  (2021-01-07T05:15:31Z)
• Efficient Deep Learning of Non-local Features for Hyperspectral Image Classification [28.72648031677868]
  A deep fully convolutional network (FCN) with an efficient non-local module, named ENL-FCN, is proposed for hyperspectral image (HSI) classification. The proposed framework, a deep FCN considers an entire HSI as input and extracts spectral-spatial information in a local receptive field. By using a recurrent operation, each pixel's response is aggregated from all pixels of HSI.
  arXiv  Detail & Related papers  (2020-08-02T19:13:22Z)
• Fully Embedding Fast Convolutional Networks on Pixel Processor Arrays [16.531637803429277]
  We present a novel method of CNN inference for pixel processor array ( PPA) vision sensors. Our approach can perform convolutional layers, max pooling, ReLu, and a final fully connected layer entirely upon the PPA sensor. This is the first work demonstrating CNN inference conducted entirely upon the processor array of a PPA vision sensor device, requiring no external processing.
  arXiv  Detail & Related papers  (2020-04-27T01:00:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.