BU-CVKit: Extendable Computer Vision Framework for Species Independent Tracking and Analysis

• URL: http://arxiv.org/abs/2306.04736v1
• Date: Wed, 7 Jun 2023 19:12:03 GMT
• Title: BU-CVKit: Extendable Computer Vision Framework for Species Independent Tracking and Analysis
• Authors: Mahir Patel, Lucas Carstensen, Yiwen Gu, Michael E. Hasselmo, Margrit Betke
• Abstract summary: We present a computer vision framework that allows creation of research pipelines with chain Processors. The community can create plugins of their work for the framework, hence improving the reusability, accessibility, and exposure of their work. We show examples of behavioral pipelines created through the sample plugins created for our framework.
• Score: 7.036239435275302
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A major bottleneck of interdisciplinary computer vision (CV) research is the lack of a framework that eases the reuse and abstraction of state-of-the-art CV models by CV and non-CV researchers alike. We present here BU-CVKit, a computer vision framework that allows the creation of research pipelines with chainable Processors. The community can create plugins of their work for the framework, hence improving the re-usability, accessibility, and exposure of their work with minimal overhead. Furthermore, we provide MuSeqPose Kit, a user interface for the pose estimation package of BU-CVKit, which automatically scans for installed plugins and programmatically generates an interface for them based on the metadata provided by the user. It also provides software support for standard pose estimation features such as annotations, 3D reconstruction, reprojection, and camera calibration. Finally, we show examples of behavioral neuroscience pipelines created through the sample plugins created for our framework.

Related papers

• VisioFirm: Cross-Platform AI-assisted Annotation Tool for Computer Vision [1.5469452301122175]
  COCO-Firm is an open-source web application designed to streamline image labeling through AI-assisted automation.<n>Coco-Firm integrates state-of-the-art foundation models into an interface with a filtering pipeline to reduce human-in-the-loop efforts.
  arXiv  Detail & Related papers  (2025-09-04T12:54:32Z)
• Granite Vision: a lightweight, open-source multimodal model for enterprise Intelligence [88.74800617923083]
  We introduce Granite Vision, a lightweight large language model with vision capabilities. Our model is trained on a comprehensive instruction-following dataset. Granite Vision achieves strong results in standard benchmarks related to visual document understanding.
  arXiv  Detail & Related papers  (2025-02-14T05:36:32Z)
• Unveiling the Backbone-Optimizer Coupling Bias in Visual Representation Learning [54.956037293979506]
  This paper delves into the interplay between vision backbones and vision backbones and their inter-dependent phenomenon termed textittextbfbackbonetextbfoptimizer textbfcoupling textbfbias (BOCB) We observe that canonical CNNs, such as VGG and ResNet, exhibit a marked co-dependency with SGD families, while recent architectures like ViTs and ConvNeXt share a tight coupling with the adaptive learning rate ones.
  arXiv  Detail & Related papers  (2024-10-08T21:14:23Z)
• fVDB: A Deep-Learning Framework for Sparse, Large-Scale, and High-Performance Spatial Intelligence [50.417261057533786]
  fVDB is a novel framework for deep learning on large-scale 3D data. Our framework is fully integrated with PyTorch enabling interoperability with existing pipelines.
  arXiv  Detail & Related papers  (2024-07-01T20:20:33Z)
• BEHAVIOR Vision Suite: Customizable Dataset Generation via Simulation [57.40024206484446]
  We introduce the BEHAVIOR Vision Suite (BVS), a set of tools and assets to generate fully customized synthetic data for systematic evaluation of computer vision models. BVS supports a large number of adjustable parameters at the scene level. We showcase three example application scenarios.
  arXiv  Detail & Related papers  (2024-05-15T17:57:56Z)
• arcjetCV: an open-source software to analyze material ablation [44.99833362998488]
  arcjetCV is an open-source Python software designed to automate time-resolved measurements of heatshield material recession and recession rates from arcjet test video footage. ArcjetCV automates the video segmentation process using machine learning models, including a one-dimensional (1D) Convolutional Neural Network (CNN) A graphical user interface (GUI) simplifies the user experience and an application programming interface (API) allows users to call the core functions from scripts.
  arXiv  Detail & Related papers  (2024-04-17T15:47:26Z)
• Lightweight Syntactic API Usage Analysis with UCov [0.0]
  We present a novel conceptual framework designed to assist library maintainers in understanding the interactions allowed by their APIs. These customizable models enable library maintainers to improve their design ahead of release, reducing friction during evolution. We implement these models for Java libraries in a new tool UCov and demonstrate its capabilities on three libraries exhibiting diverse styles of interaction.
  arXiv  Detail & Related papers  (2024-02-19T10:33:41Z)
• ESPnet-SPK: full pipeline speaker embedding toolkit with reproducible recipes, self-supervised front-ends, and off-the-shelf models [51.35570730554632]
  ESPnet-SPK is a toolkit for training speaker embedding extractors. We provide several models, ranging from x-vector to recent SKA-TDNN. We also aspire to bridge developed models with other domains.
  arXiv  Detail & Related papers  (2024-01-30T18:18:27Z)
• What Can Human Sketches Do for Object Detection? [127.67444974452411]
  Sketches are highly expressive, inherently capturing subjective and fine-grained visual cues. A sketch-enabled object detection framework detects based on what textityou sketch -- textitthat zebra'' We show an intuitive synergy between foundation models (e.g., CLIP) and existing sketch models build for sketch-based image retrieval (SBIR) In particular, we first perform independent on both sketch branches of an encoder model to build highly generalisable sketch and photo encoders.
  arXiv  Detail & Related papers  (2023-03-27T12:33:23Z)
• Interactive Visualization of Protein RINs using NetworKit in the Cloud [57.780880387925954]
  In this paper, we consider an example from protein dynamics, specifically residue interaction networks (RINs) We use NetworKit to build a cloud-based environment that enables domain scientists to run their visualization and analysis on large compute servers. To demonstrate the versatility of this approach, we use it to build a custom Jupyter-based widget for RIN visualization.
  arXiv  Detail & Related papers  (2022-03-02T17:41:45Z)
• OdoViz: A 3D Odometry Visualization and Processing Tool [0.0]
  OdoViz is a reactive web-based tool for 3D visualization and processing of autonomous vehicle datasets. The system includes functionality for loading, inspecting, visualizing, and processing GPS/INS poses, point clouds and camera images.
  arXiv  Detail & Related papers  (2021-07-15T18:37:19Z)
• A survey on Kornia: an Open Source Differentiable Computer Vision Library for PyTorch [0.0]
  This work presents Kornia, an open source computer vision library built upon a set of differentiable routines and modules that aims to solve generic computer vision problems. The package uses PyTorch as its main backend, not only for efficiency but also to take advantage of the reverse auto-differentiation engine to define and compute the gradient of complex functions.
  arXiv  Detail & Related papers  (2020-09-21T08:48:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.