PlankFormer: Robust Plankton Instance Segmentation via MAE-Pretrained Vision Transformers and Pseudo Community Image Generation

• URL: http://arxiv.org/abs/2604.17856v1
• Date: Mon, 20 Apr 2026 06:09:32 GMT
• Title: PlankFormer: Robust Plankton Instance Segmentation via MAE-Pretrained Vision Transformers and Pseudo Community Image Generation
• Abstract summary: Plankton is essential for assessing aquatic ecosystems but is limited by the labor-intensive nature of microscopic analysis.<n>The segmentation of plankton from crowded images is crucial, however, it faces two major challenges.<n>We propose PlankFormer, a novel framework for plankton instance segmentation.
• Score: 1.2314765641075438
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Plankton monitoring is essential for assessing aquatic ecosystems but is limited by the labor-intensive nature of manual microscopic analysis. Automating the segmentation of plankton from crowded images is crucial, however, it faces two major challenges: (i) the scarcity of pixel-level annotated datasets and (ii) the difficulty of distinguishing plankton from debris and overlapping individuals using conventional CNN-based methods. To address these issues, we propose PlankFormer, a novel framework for plankton instance segmentation. First, to overcome the data shortage, we introduce a method to generate labeled Pseudo Community Images (PCI) by synthesizing individual plankton images onto diverse backgrounds, including those created by generative models. Second, we propose a segmentation model utilizing a Vision Transformer (ViT) backbone with a Mask2Former decoder. To robustly capture the global structural features of plankton against occlusion and debris, we employ a Masked Autoencoder (MAE) for self-supervised pre-training on unlabeled individual images. Experimental results on real-world datasets demonstrate that our method significantly outperforms conventional methods, such as Mask R-CNN, particularly in challenging environments with high debris density. We demonstrate that our synthetic training strategy and MAE-based architecture enable high-precision segmentation with requiring less manual annotations for individual plankton images.

Related papers

• Cross-modal learning for plankton recognition [7.782757108520769]
  This paper considers self-supervised cross-modal coordination as a strategy enabling utilization of multiple modalities for plankton recognition.<n>We show that the proposed method achieves high recognition accuracy while requiring only a minimal number of labeled images.
  arXiv  Detail & Related papers  (2026-03-17T12:04:52Z)
• UniSH: Unifying Scene and Human Reconstruction in a Feed-Forward Pass [83.7071371474926]
  UniSH is a unified, feed-forward framework for joint metric-scale 3D scene and human reconstruction.<n>Our framework bridges strong, disparate priors from scene reconstruction and HMR.<n>Our model achieves state-of-the-art performance on human-centric scene reconstruction.
  arXiv  Detail & Related papers  (2026-01-03T16:06:27Z)
• Expose Camouflage in the Water: Underwater Camouflaged Instance Segmentation and Dataset [76.92197418745822]
  camouflaged instance segmentation (CIS) faces greater challenges in accurately segmenting objects that blend closely with their surroundings.<n>Traditional camouflaged instance segmentation methods, trained on terrestrial-dominated datasets with limited underwater samples, may exhibit inadequate performance in underwater scenes.<n>We introduce the first underwater camouflaged instance segmentation dataset, UCIS4K, which comprises 3,953 images of camouflaged marine organisms with instance-level annotations.
  arXiv  Detail & Related papers  (2025-10-20T14:34:51Z)
• Bridging Knowledge Gap Between Image Inpainting and Large-Area Visible Watermark Removal [57.84348166457113]
  We introduce a novel feature adapting framework that leverages the representation capacity of a pre-trained image inpainting model.<n>Our approach bridges the knowledge gap between image inpainting and watermark removal by fusing information of the residual background content beneath watermarks into the inpainting backbone model.<n>For relieving the dependence on high-quality watermark masks, we introduce a new training paradigm by utilizing coarse watermark masks to guide the inference process.
  arXiv  Detail & Related papers  (2025-04-07T02:37:14Z)
• Self-Supervised Pretraining for Fine-Grained Plankton Recognition [0.11309478649967238]
  Plankton recognition is an important computer vision problem due to plankton's essential role in ocean food webs and carbon capture.<n>In this work, we study large-scale self-supervised pretraining for fine-grained plankton recognition.
  arXiv  Detail & Related papers  (2025-03-14T12:15:20Z)
• Image-Based Relocalization and Alignment for Long-Term Monitoring of Dynamic Underwater Environments [57.59857784298534]
  We propose an integrated pipeline that combines Visual Place Recognition (VPR), feature matching, and image segmentation on video-derived images.<n>This method enables robust identification of revisited areas, estimation of rigid transformations, and downstream analysis of ecosystem changes.
  arXiv  Detail & Related papers  (2025-03-06T05:13:19Z)
• MPT: A Large-scale Multi-Phytoplankton Tracking Benchmark [36.37530623015916]
  We propose a benchmark dataset, Multiple Phytoplankton Tracking (MPT), which covers diverse background information and variations in motion during observation.<n>The dataset includes 27 species of phytoplankton and zooplankton, 14 different backgrounds to simulate diverse and complex underwater environments, and a total of 140 videos.<n>We introduce an additional feature extractor to predict the residuals of the standard feature extractor's output, and compute multi-scale frame-to-frame similarity based on features from different layers of the extractor.
  arXiv  Detail & Related papers  (2024-10-22T04:57:28Z)
• Pixel-Inconsistency Modeling for Image Manipulation Localization [59.968362815126326]
  Digital image forensics plays a crucial role in image authentication and manipulation localization. This paper presents a generalized and robust manipulation localization model through the analysis of pixel inconsistency artifacts. Experiments show that our method successfully extracts inherent pixel-inconsistency forgery fingerprints.
  arXiv  Detail & Related papers  (2023-09-30T02:54:51Z)
• Efficient Unsupervised Learning for Plankton Images [12.447149371717]
  Monitoring plankton populations in situ is fundamental to preserve the aquatic ecosystem. The adoption of machine learning algorithms to classify such data may be affected by the significant cost of manual annotation. We propose an efficient unsupervised learning pipeline to provide accurate classification of plankton microorganisms.
  arXiv  Detail & Related papers  (2022-09-14T15:33:16Z)
• Towards Generating Large Synthetic Phytoplankton Datasets for Efficient Monitoring of Harmful Algal Blooms [77.25251419910205]
  Harmful algal blooms (HABs) cause significant fish deaths in aquaculture farms. Currently, the standard method to enumerate harmful algae and other phytoplankton is to manually observe and count them under a microscope. We employ Generative Adversarial Networks (GANs) to generate synthetic images.
  arXiv  Detail & Related papers  (2022-08-03T20:15:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.