Annolid: Annotate, Segment, and Track Anything You Need

• URL: http://arxiv.org/abs/2403.18690v1
• Date: Wed, 27 Mar 2024 15:41:23 GMT
• Title: Annolid: Annotate, Segment, and Track Anything You Need
• Authors: Chen Yang, Thomas A. Cleland,
• Abstract summary: Annolid is a software package designed for the segmentation, labeling, and tracking of research targets within video files. Based on state-of-the-art instance segmentation methods, Annolid now harnesses the Cutie video object segmentation model. Our integration of Segment Anything and Grounding-DINO strategies additionally enables the automatic masking and segmentation of recognizable animals and objects by text command.
• Score: 2.0517786943861376
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Annolid is a deep learning-based software package designed for the segmentation, labeling, and tracking of research targets within video files, focusing primarily on animal behavior analysis. Based on state-of-the-art instance segmentation methods, Annolid now harnesses the Cutie video object segmentation model to achieve resilient, markerless tracking of multiple animals from single annotated frames, even in environments in which they may be partially or entirely concealed by environmental features or by one another. Our integration of Segment Anything and Grounding-DINO strategies additionally enables the automatic masking and segmentation of recognizable animals and objects by text command, removing the need for manual annotation. Annolid's comprehensive approach to object segmentation flexibly accommodates a broad spectrum of behavior analysis applications, enabling the classification of diverse behavioral states such as freezing, digging, pup huddling, and social interactions in addition to the tracking of animals and their body parts.

Related papers

• Appearance-Based Refinement for Object-Centric Motion Segmentation [85.2426540999329]
  We introduce an appearance-based refinement method that leverages temporal consistency in video streams to correct inaccurate flow-based proposals. Our approach involves a sequence-level selection mechanism that identifies accurate flow-predicted masks as exemplars. Its performance is evaluated on multiple video segmentation benchmarks, including DAVIS, YouTube, SegTrackv2, and FBMS-59.
  arXiv  Detail & Related papers  (2023-12-18T18:59:51Z)
• Automated Behavioral Analysis Using Instance Segmentation [2.043437148047176]
  Animal behavior analysis plays a crucial role in various fields, such as life science and biomedical research. The scarcity of available data and the high cost associated with obtaining a large number of labeled datasets pose significant challenges. We propose a novel approach that leverages instance segmentation-based transfer learning to address these issues.
  arXiv  Detail & Related papers  (2023-12-12T20:36:36Z)
• Towards Open-World Segmentation of Parts [16.056921233445784]
  We propose to explore a class-agnostic part segmentation task. We argue that models trained without part classes can better localize parts and segment them on objects unseen in training. We show notable and consistent gains by our approach, essentially a critical step towards open-world part segmentation.
  arXiv  Detail & Related papers  (2023-05-26T10:34:58Z)
• Tag-Based Attention Guided Bottom-Up Approach for Video Instance Segmentation [83.13610762450703]
  Video instance is a fundamental computer vision task that deals with segmenting and tracking object instances across a video sequence. We introduce a simple end-to-end train bottomable-up approach to achieve instance mask predictions at the pixel-level granularity, instead of the typical region-proposals-based approach. Our method provides competitive results on YouTube-VIS and DAVIS-19 datasets, and has minimum run-time compared to other contemporary state-of-the-art performance methods.
  arXiv  Detail & Related papers  (2022-04-22T15:32:46Z)
• Animal Kingdom: A Large and Diverse Dataset for Animal Behavior Understanding [4.606145900630665]
  We create a large and diverse dataset, Animal Kingdom, that provides multiple annotated tasks. Our dataset contains 50 hours of annotated videos to localize relevant animal behavior segments. We propose a Collaborative Action Recognition (CARe) model that learns general and specific features for action recognition with unseen new animals.
  arXiv  Detail & Related papers  (2022-04-18T02:05:15Z)
• The Emergence of Objectness: Learning Zero-Shot Segmentation from Videos [59.12750806239545]
  We show that a video has different views of the same scene related by moving components, and the right region segmentation and region flow would allow mutual view synthesis. Our model starts with two separate pathways: an appearance pathway that outputs feature-based region segmentation for a single image, and a motion pathway that outputs motion features for a pair of images. By training the model to minimize view synthesis errors based on segment flow, our appearance and motion pathways learn region segmentation and flow estimation automatically without building them up from low-level edges or optical flows respectively.
  arXiv  Detail & Related papers  (2021-11-11T18:59:11Z)
• Learning Visual Affordance Grounding from Demonstration Videos [76.46484684007706]
  Affordance grounding aims to segment all possible interaction regions between people and objects from an image/video. We propose a Hand-aided Affordance Grounding Network (HAGNet) that leverages the aided clues provided by the position and action of the hand in demonstration videos.
  arXiv  Detail & Related papers  (2021-08-12T11:45:38Z)
• CompFeat: Comprehensive Feature Aggregation for Video Instance Segmentation [67.17625278621134]
  Video instance segmentation is a complex task in which we need to detect, segment, and track each object for any given video. Previous approaches only utilize single-frame features for the detection, segmentation, and tracking of objects. We propose a novel comprehensive feature aggregation approach (CompFeat) to refine features at both frame-level and object-level with temporal and spatial context information.
  arXiv  Detail & Related papers  (2020-12-07T00:31:42Z)
• DyStaB: Unsupervised Object Segmentation via Dynamic-Static Bootstrapping [72.84991726271024]
  We describe an unsupervised method to detect and segment portions of images of live scenes that are seen moving as a coherent whole. Our method first partitions the motion field by minimizing the mutual information between segments. It uses the segments to learn object models that can be used for detection in a static image.
  arXiv  Detail & Related papers  (2020-08-16T22:05:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.