Systematically designing better instance counting models on cell images with Neural Arithmetic Logic Units

• URL: http://arxiv.org/abs/2004.06674v2
• Date: Mon, 15 Jun 2020 07:44:46 GMT
• Title: Systematically designing better instance counting models on cell images with Neural Arithmetic Logic Units
• Authors: Ashish Rana, Taranveer Singh, Harpreet Singh, Neeraj Kumar and Prashant Singh Rana
• Abstract summary: We are aiming to create better generalization systems for cell counting. numerically biased units do help models to learn numeric quantities for better generalization results. Our results confirm that above stated numerically biased units does help models to learn numeric quantities for better generalization results.
• Score: 11.864159170745893
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The big problem for neural network models which are trained to count instances is that whenever test range goes high training range generalization error increases i.e. they are not good generalizers outside training range. Consider the case of automating cell counting process where more dense images with higher cell counts are commonly encountered as compared to images used in training data. By making better predictions for higher ranges of cell count we are aiming to create better generalization systems for cell counting. With architecture proposal of neural arithmetic logic units (NALU) for arithmetic operations, task of counting has become feasible for higher numeric ranges which were not included in training data with better accuracy. As a part of our study we used these units and different other activation functions for learning cell counting task with two different architectures namely Fully Convolutional Regression Network and U-Net. These numerically biased units are added in the form of residual concatenated layers to original architectures and a comparative experimental study is done with these newly proposed changes. This comparative study is described in terms of optimizing regression loss problem from these models trained with extensive data augmentation techniques. We were able to achieve better results in our experiments of cell counting tasks with introduction of these numerically biased units to already existing architectures in the form of residual layer concatenation connections. Our results confirm that above stated numerically biased units does help models to learn numeric quantities for better generalization results.

Related papers

• A Dynamical Model of Neural Scaling Laws [79.59705237659547]
  We analyze a random feature model trained with gradient descent as a solvable model of network training and generalization. Our theory shows how the gap between training and test loss can gradually build up over time due to repeated reuse of data.
  arXiv  Detail & Related papers  (2024-02-02T01:41:38Z)
• Target Variable Engineering [0.0]
  We compare the predictive performance of regression models trained to predict numeric targets vs. classifiers trained to predict their binarized counterparts. We find that regression requires significantly more computational effort to converge upon the optimal performance.
  arXiv  Detail & Related papers  (2023-10-13T23:12:21Z)
• Winner-Take-All Column Row Sampling for Memory Efficient Adaptation of Language Model [89.8764435351222]
  We propose a new family of unbiased estimators called WTA-CRS, for matrix production with reduced variance. Our work provides both theoretical and experimental evidence that, in the context of tuning transformers, our proposed estimators exhibit lower variance compared to existing ones.
  arXiv  Detail & Related papers  (2023-05-24T15:52:08Z)
• Cross-Model Comparative Loss for Enhancing Neuronal Utility in Language Understanding [82.46024259137823]
  We propose a cross-model comparative loss for a broad range of tasks. We demonstrate the universal effectiveness of comparative loss through extensive experiments on 14 datasets from 3 distinct NLU tasks.
  arXiv  Detail & Related papers  (2023-01-10T03:04:27Z)
• An Information-Theoretic Analysis of Compute-Optimal Neural Scaling Laws [24.356906682593532]
  We study the compute-optimal trade-off between model and training data set sizes for large neural networks. Our result suggests a linear relation similar to that supported by the empirical analysis of chinchilla.
  arXiv  Detail & Related papers  (2022-12-02T18:46:41Z)
• HyperImpute: Generalized Iterative Imputation with Automatic Model Selection [77.86861638371926]
  We propose a generalized iterative imputation framework for adaptively and automatically configuring column-wise models. We provide a concrete implementation with out-of-the-box learners, simulators, and interfaces.
  arXiv  Detail & Related papers  (2022-06-15T19:10:35Z)
• A Tutorial on the Mathematical Model of Single Cell Variational Inference [0.0]
  This tutorial will introduce the the mathematical model of the single cell variational inference (scVI) It was written for beginners in the simple and intuitive way with many deduction details to encourage more researchers into this field.
  arXiv  Detail & Related papers  (2021-01-03T16:02:36Z)
• Category-Learning with Context-Augmented Autoencoder [63.05016513788047]
  Finding an interpretable non-redundant representation of real-world data is one of the key problems in Machine Learning. We propose a novel method of using data augmentations when training autoencoders. We train a Variational Autoencoder in such a way, that it makes transformation outcome predictable by auxiliary network.
  arXiv  Detail & Related papers  (2020-10-10T14:04:44Z)
• Neural Execution Engines: Learning to Execute Subroutines [29.036699193820215]
  We study the generalization issues at the level of numerical subroutines that comprise common algorithms like sorting, shortest paths, and minimum spanning trees. To generalize to unseen data, we show that encoding numbers with a binary representation leads to embeddings with rich structure once trained on downstream tasks like addition or multiplication.
  arXiv  Detail & Related papers  (2020-06-15T01:51:37Z)
• iNALU: Improved Neural Arithmetic Logic Unit [2.331160520377439]
  The recently proposed Neural Arithmetic Logic Unit (NALU) is a novel neural architecture which is able to explicitly represent the mathematical relationships by the units of the network to learn operations such as summation, subtraction or multiplication. We show that our model solves stability issues and outperforms the original NALU model in means of arithmetic precision and convergence.
  arXiv  Detail & Related papers  (2020-03-17T10:37:22Z)
• Model Fusion via Optimal Transport [64.13185244219353]
  We present a layer-wise model fusion algorithm for neural networks. We show that this can successfully yield "one-shot" knowledge transfer between neural networks trained on heterogeneous non-i.i.d. data.
  arXiv  Detail & Related papers  (2019-10-12T22:07:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.