Tile-Based ViT Inference with Visual-Cluster Priors for Zero-Shot Multi-Species Plant Identification

• URL: http://arxiv.org/abs/2507.06093v1
• Date: Tue, 08 Jul 2025 15:35:19 GMT
• Title: Tile-Based ViT Inference with Visual-Cluster Priors for Zero-Shot Multi-Species Plant Identification
• Abstract summary: We describe DS@GT's second-place solution to the PlantCLEF 2025 challenge on multi-species plant identification in vegetation quadrat images.<n>Our pipeline combines (i) a fine-tuned Vision Transformer ViTD2PC24All for patch-level inference, (ii) a 4x4 tiling strategy that aligns patch size with the network's 518x518 receptive field, and (iii) domain-prior adaptation through PaCMAP + K-Means visual clustering and geolocation filtering.
• Score: 0.1181206257787103
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We describe DS@GT's second-place solution to the PlantCLEF 2025 challenge on multi-species plant identification in vegetation quadrat images. Our pipeline combines (i) a fine-tuned Vision Transformer ViTD2PC24All for patch-level inference, (ii) a 4x4 tiling strategy that aligns patch size with the network's 518x518 receptive field, and (iii) domain-prior adaptation through PaCMAP + K-Means visual clustering and geolocation filtering. Tile predictions are aggregated by majority vote and re-weighted with cluster-specific Bayesian priors, yielding a macro-averaged F1 of 0.348 (private leaderboard) while requiring no additional training. All code, configuration files, and reproducibility scripts are publicly available at https://github.com/dsgt-arc/plantclef-2025.

Related papers

• Cut-ViT: Task-Specific Model Pruning via Gram Anchoring Subspace Consistency [71.81785123423516]
  We propose a task-specific pruning pipeline, named Cut-ViT, for pruning visual foundation models.<n> Basis-agnostic and residual constraints are adopted to align the gram subspaces between the native and pruned DINOv3 models.<n>Experiments show that Cut-ViT requires approximately one minute on a single A100 GPU to obtainworks at various sparsity levels.
  arXiv  Detail & Related papers  (2026-08-28T11:25:36Z)
• Multi-Scale ViT Inference with Habitat-Fit Priors and kNN Retrieval for Multi-Species Plant Identification [0.04014524824655105]
  This paper describes DS@GT ARC's third-place solution to the PlantCLEF 2026 challenge on multi-species plant identification in vegetation quadrat images.<n>The pipeline is built around a fine-tuned DINOv2 ViT-L/14 classifier applied over a multi-scale tile decomposition of each quadrat.<n>The selected submission reaches a private-leaderboard macro-F1 of 0.43902 (third place; public 0.51096); an unselected configuration of the same pipeline scored above 0.45 on the private set.
  arXiv  Detail & Related papers  (2026-07-16T02:47:38Z)
• Scaling Parallel Sequence Models to Foundation-Scale Vision Encoders [123.58723804218151]
  Vision foundation models are bottlenecked by the quadratic cost of self-attention.<n>We present C-GSPN based on 2D spatial propagation propagation.
  arXiv  Detail & Related papers  (2026-05-30T14:29:43Z)
• CropVLM: A Domain-Adapted Vision-Language Model for Open-Set Crop Analysis [0.0]
  We present CropVLM, a vision-language model adapted for the agricultural domain via Domain-Specific Semantic Alignment (DSSA)<n>CropVLM is trained on 52,987 manually selected image-caption pairs covering 37 species in natural field conditions.<n>In comprehensive evaluations, CropVLM achieves 72.51% zero-shot classification accuracy, outperforming seven CLIP-style baselines.
  arXiv  Detail & Related papers  (2026-05-05T01:17:27Z)
• Zero-Shot Segmentation through Prototype-Guidance for Multi-Label Plant Species Identification [0.5249805590164902]
  This paper presents an approach developed to address the PlantClef 2025 challenge, which consists of a fine-grained multi-label species identification.<n>Our solution focused on employing class prototypes obtained from the training dataset as a proxy guidance for training a segmentation Vision Transformer (ViT) on the test set images.<n>The proposed approach enabled a domain-adaptation from multi-class identification with individual species, into multi-label classification from high-resolution vegetation plots.
  arXiv  Detail & Related papers  (2025-12-23T01:06:55Z)
• Re$^{\ ext{2}}$MaP: Macro Placement by Recursively Prototyping and Packing Tree-based Relocating [67.49674976434322]
  This work introduces the Re$text2$MaP method, which generates expert-quality macro placements.<n>We use DREAMPlace to build a mixed-size placement prototype and obtain reference positions for each macro and cluster.<n>A packing tree-based relocating procedure is then designed to jointly adjust the locations of macro groups and the macros within each group.
  arXiv  Detail & Related papers  (2025-11-11T09:56:10Z)
• Multi-Label Plant Species Prediction with Metadata-Enhanced Multi-Head Vision Transformers [0.0]
  We present a multi-head vision transformer approach for multi-label plant species prediction in vegetation plot images.<n>The task involves training models on single-species plant images while testing on multi-species quadrat images, creating a drastic domain shift.<n>Our methodology leverages a pre-trained DINOv2 Vision Transformer Base (ViT-B/14) backbone with multiple classification heads for species, genus, and family prediction.
  arXiv  Detail & Related papers  (2025-08-14T08:56:58Z)
• FTCFormer: Fuzzy Token Clustering Transformer for Image Classification [22.410199372985584]
  Transformer-based deep neural networks have achieved remarkable success across various computer vision tasks.<n>Most transformer architectures embed images into uniform, grid-based vision tokens, neglecting the underlying semantic meanings of image regions.<n>We propose Fuzzy Token Clustering Transformer (FTCFormer) to dynamically generate vision tokens based on the semantic meanings instead of spatial positions.
  arXiv  Detail & Related papers  (2025-07-14T13:49:47Z)
• Frequency-Adaptive Discrete Cosine-ViT-ResNet Architecture for Sparse-Data Vision [7.0134322436635275]
  We introduce an adaptive frequency-domain selection mechanism that learns optimal low-, mid-, and high-frequency boundaries.<n>On our self-built 50-class wildlife dataset, this approach outperforms conventional CNN and fixed-band DCT pipelines.
  arXiv  Detail & Related papers  (2025-05-28T17:39:58Z)
• The Missing Point in Vision Transformers for Universal Image Segmentation [17.571552686063335]
  We introduce ViT-P, a two-stage segmentation framework that decouples mask generation from classification.<n>ViT-P serves as a pre-training-free adapter, allowing the integration of various pre-trained vision transformers.<n>Experiments across COCO, ADE20K, and Cityscapes datasets validate the effectiveness of ViT-P.
  arXiv  Detail & Related papers  (2025-05-26T10:29:13Z)
• Tile Compression and Embeddings for Multi-Label Classification in GeoLifeCLEF 2024 [0.0]
  We solve the multi-label classification task posed by the GeoLifeCLEF 2024 competition with the DS@GT team. Our approach uses frequency-domain coefficients via the Discrete Cosine Transform (DCT) to compress and pre-compute the raw input data. Our best competition model utilized geolocation features with a leaderboard score of 0.152 and a best post-competition score of 0.161.
  arXiv  Detail & Related papers  (2024-07-08T18:44:03Z)
• Multi-Label Plant Species Classification with Self-Supervised Vision Transformers [0.0]
  We present a transfer learning approach using a self-supervised Vision Transformer (DINOv2) for the PlantCLEF 2024 competition. To address the computational challenges of the large-scale dataset, we employ Spark for distributed data processing. Our results demonstrate the efficacy of combining transfer learning with advanced data processing techniques for multi-label image classification tasks.
  arXiv  Detail & Related papers  (2024-07-08T18:07:33Z)
• ClusterFormer: Clustering As A Universal Visual Learner [80.79669078819562]
  CLUSTERFORMER is a universal vision model based on the CLUSTERing paradigm with TransFORMER. It is capable of tackling heterogeneous vision tasks with varying levels of clustering granularity. For its efficacy, we hope our work can catalyze a paradigm shift in universal models in computer vision.
  arXiv  Detail & Related papers  (2023-09-22T22:12:30Z)
• DilateFormer: Multi-Scale Dilated Transformer for Visual Recognition [62.95223898214866]
  We explore effective Vision Transformers to pursue a preferable trade-off between the computational complexity and size of the attended receptive field. With a pyramid architecture, we construct a Multi-Scale Dilated Transformer (DilateFormer) by stacking MSDA blocks at low-level stages and global multi-head self-attention blocks at high-level stages. Our experiment results show that our DilateFormer achieves state-of-the-art performance on various vision tasks.
  arXiv  Detail & Related papers  (2023-02-03T14:59:31Z)
• Vision Transformers: From Semantic Segmentation to Dense Prediction [139.15562023284187]
  We explore the global context learning potentials of vision transformers (ViTs) for dense visual prediction. Our motivation is that through learning global context at full receptive field layer by layer, ViTs may capture stronger long-range dependency information. We formulate a family of Hierarchical Local-Global (HLG) Transformers, characterized by local attention within windows and global-attention across windows in a pyramidal architecture.
  arXiv  Detail & Related papers  (2022-07-19T15:49:35Z)
• Global Context Vision Transformers [78.5346173956383]
  We propose global context vision transformer (GC ViT), a novel architecture that enhances parameter and compute utilization for computer vision. We address the lack of the inductive bias in ViTs, and propose to leverage a modified fused inverted residual blocks in our architecture. Our proposed GC ViT achieves state-of-the-art results across image classification, object detection and semantic segmentation tasks.
  arXiv  Detail & Related papers  (2022-06-20T18:42:44Z)
• CSWin Transformer: A General Vision Transformer Backbone with Cross-Shaped Windows [99.36226415086243]
  We present CSWin Transformer, an efficient and effective Transformer-based backbone for general-purpose vision tasks. A challenging issue in Transformer design is that global self-attention is very expensive to compute whereas local self-attention often limits the field of interactions of each token.
  arXiv  Detail & Related papers  (2021-07-01T17:59:56Z)
• Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions [103.03973037619532]
  This work investigates a simple backbone network useful for many dense prediction tasks without convolutions. Unlike the recently-proposed Transformer model (e.g., ViT) that is specially designed for image classification, we propose Pyramid Vision Transformer(PVT) PVT can be not only trained on dense partitions of the image to achieve high output resolution, which is important for dense predictions.
  arXiv  Detail & Related papers  (2021-02-24T08:33:55Z)
• A Deep Learning Approach Based on Graphs to Detect Plantation Lines [16.76043873454695]
  We propose a deep learning approach based on graphs to detect plantation lines in UAV-based RGB imagery. The proposed method was compared against state-of-the-art deep learning methods. It achieved superior performance with a significant margin, returning precision, recall, and F1-score of 98.7%, 91.9%, and 95.1%, respectively.
  arXiv  Detail & Related papers  (2021-02-05T14:56:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.