Affinity LCFCN: Learning to Segment Fish with Weak Supervision

• URL: http://arxiv.org/abs/2011.03149v1
• Date: Fri, 6 Nov 2020 00:33:20 GMT
• Title: Affinity LCFCN: Learning to Segment Fish with Weak Supervision
• Authors: Issam Laradji, Alzayat Saleh, Pau Rodriguez, Derek Nowrouzezahrai, Mostafa Rahimi Azghadi, David Vazquez
• Abstract summary: We propose an automatic segmentation model efficiently trained on images labeled with only point-level supervision. Our approach uses a fully convolutional neural network with one branch that outputs per-pixel scores and another that outputs an affinity matrix. We validate our model on the DeepFish dataset, which contains many fish habitats from the north-eastern Australian region.
• Score: 15.245008639754328
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Aquaculture industries rely on the availability of accurate fish body measurements, e.g., length, width and mass. Manual methods that rely on physical tools like rulers are time and labour intensive. Leading automatic approaches rely on fully-supervised segmentation models to acquire these measurements but these require collecting per-pixel labels -- also time consuming and laborious: i.e., it can take up to two minutes per fish to generate accurate segmentation labels, almost always requiring at least some manual intervention. We propose an automatic segmentation model efficiently trained on images labeled with only point-level supervision, where each fish is annotated with a single click. This labeling process requires significantly less manual intervention, averaging roughly one second per fish. Our approach uses a fully convolutional neural network with one branch that outputs per-pixel scores and another that outputs an affinity matrix. We aggregate these two outputs using a random walk to obtain the final, refined per-pixel segmentation output. We train the entire model end-to-end with an LCFCN loss, resulting in our A-LCFCN method. We validate our model on the DeepFish dataset, which contains many fish habitats from the north-eastern Australian region. Our experimental results confirm that A-LCFCN outperforms a fully-supervised segmentation model at fixed annotation budget. Moreover, we show that A-LCFCN achieves better segmentation results than LCFCN and a standard baseline. We have released the code at \url{https://github.com/IssamLaradji/affinity_lcfcn}.

Related papers

• AutoFish: Dataset and Benchmark for Fine-grained Analysis of Fish [19.025566399187547]
  The dataset comprises 1,500 images of 454 specimens of visually similar fish placed in various constellations on a white conveyor belt. The data was collected in a controlled environment using an RGB camera. We establish baseline instance segmentation results using two variations of the Mask2Former architecture.
  arXiv  Detail & Related papers  (2025-01-07T13:14:25Z)
• Better Call SAL: Towards Learning to Segment Anything in Lidar [63.9984147657437]
  We propose a text-promptable zero-shot model for segmenting and classifying any object in Lidar. We utilize 2D vision foundation models to generate 3D supervision for free'' using pseudo-labels. Our model achieves $91%$ in terms of class-agnostic and $54%$ in terms of zero-shot Lidar Panopticon.
  arXiv  Detail & Related papers  (2024-03-19T19:58:54Z)
• CFDP: Common Frequency Domain Pruning [0.3021678014343889]
  We introduce a novel end-to-end pipeline for model pruning via the frequency domain. We have achieved state-of-the-art results on CIFAR-10 with GoogLeNet reaching an accuracy of 95.25%, that is, +0.2% from the original model. In addition to notable performances, models produced via CFDP exhibit robustness to a variety of configurations.
  arXiv  Detail & Related papers  (2023-06-07T04:49:26Z)
• Multi-Level Contrastive Learning for Dense Prediction Task [59.591755258395594]
  We present Multi-Level Contrastive Learning for Dense Prediction Task (MCL), an efficient self-supervised method for learning region-level feature representation for dense prediction tasks. Our method is motivated by the three key factors in detection: localization, scale consistency and recognition. Our method consistently outperforms the recent state-of-the-art methods on various datasets with significant margins.
  arXiv  Detail & Related papers  (2023-04-04T17:59:04Z)
• LESS: Label-Efficient Semantic Segmentation for LiDAR Point Clouds [62.49198183539889]
  We propose a label-efficient semantic segmentation pipeline for outdoor scenes with LiDAR point clouds. Our method co-designs an efficient labeling process with semi/weakly supervised learning. Our proposed method is even highly competitive compared to the fully supervised counterpart with 100% labels.
  arXiv  Detail & Related papers  (2022-10-14T19:13:36Z)
• Exploiting Shape Cues for Weakly Supervised Semantic Segmentation [15.791415215216029]
  Weakly supervised semantic segmentation (WSSS) aims to produce pixel-wise class predictions with only image-level labels for training. We propose to exploit shape information to supplement the texture-biased property of convolutional neural networks (CNNs) We further refine the predictions in an online fashion with a novel refinement method that takes into account both the class and the color affinities.
  arXiv  Detail & Related papers  (2022-08-08T17:25:31Z)
• Transformer-based Self-Supervised Fish Segmentation in Underwater Videos [1.9249287163937976]
  We introduce a Transformer-based method that uses self-supervision for high-quality fish segmentation. We show that when trained on a set of underwater videos from one dataset, the proposed model surpasses previous CNN-based and Transformer-based self-supervised methods.
  arXiv  Detail & Related papers  (2022-06-11T01:20:48Z)
• Adaptive Context-Aware Multi-Modal Network for Depth Completion [107.15344488719322]
  We propose to adopt the graph propagation to capture the observed spatial contexts. We then apply the attention mechanism on the propagation, which encourages the network to model the contextual information adaptively. Finally, we introduce the symmetric gated fusion strategy to exploit the extracted multi-modal features effectively. Our model, named Adaptive Context-Aware Multi-Modal Network (ACMNet), achieves the state-of-the-art performance on two benchmarks.
  arXiv  Detail & Related papers  (2020-08-25T06:00:06Z)
• The Devil is in Classification: A Simple Framework for Long-tail Object Detection and Instance Segmentation [93.17367076148348]
  We investigate performance drop of the state-of-the-art two-stage instance segmentation model Mask R-CNN on the recent long-tail LVIS dataset. We unveil that a major cause is the inaccurate classification of object proposals. We propose a simple calibration framework to more effectively alleviate classification head bias with a bi-level class balanced sampling approach.
  arXiv  Detail & Related papers  (2020-07-23T12:49:07Z)
• Pre-Trained Models for Heterogeneous Information Networks [57.78194356302626]
  We propose a self-supervised pre-training and fine-tuning framework, PF-HIN, to capture the features of a heterogeneous information network. PF-HIN consistently and significantly outperforms state-of-the-art alternatives on each of these tasks, on four datasets.
  arXiv  Detail & Related papers  (2020-07-07T03:36:28Z)
• Deep Learning based Segmentation of Fish in Noisy Forward Looking MBES Images [1.5469452301122177]
  We build on recent advances in Deep Learning (DL) and Convolutional Neural Networks (CNNs) for semantic segmentation. We demonstrate an end-to-end approach for a fish/non-fish probability prediction for all range-azimuth positions projected by an imaging sonar. We show that our model proves the desired performance and has learned to harness the importance of semantic context.
  arXiv  Detail & Related papers  (2020-06-16T09:57:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.