A ROS 2 Wrapper for Florence-2: Multi-Mode Local Vision-Language Inference for Robotic Systems

• URL: http://arxiv.org/abs/2604.01179v1
• Date: Wed, 01 Apr 2026 17:29:59 GMT
• Title: A ROS 2 Wrapper for Florence-2: Multi-Mode Local Vision-Language Inference for Robotic Systems
• Abstract summary: This article presents a ROS 2 wrapper for Florence-2 that exposes the model through three complementary interaction modes.<n>The wrapper is designed for local execution and supports both native installation and Docker container deployment.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Foundation vision-language models are becoming increasingly relevant to robotics because they can provide richer semantic perception than narrow task-specific pipelines. However, their practical adoption in robot software stacks still depends on reproducible middleware integrations rather than on model quality alone. Florence-2 is especially attractive in this regard because it unifies captioning, optical character recognition, open-vocabulary detection, grounding and related vision-language tasks within a comparatively manageable model size. This article presents a ROS 2 wrapper for Florence-2 that exposes the model through three complementary interaction modes: continuous topic-driven processing, synchronous service calls and asynchronous actions. The wrapper is designed for local execution and supports both native installation and Docker container deployment. It also combines generic JSON outputs with standard ROS 2 message bindings for detection-oriented tasks. A functional validation is reported together with a throughput study on several GPUs, showing that local deployment is feasible with consumer grade hardware. The repository is publicly available here: https://github.com/JEDominguezVidal/florence2_ros2_wrapper

Related papers

• Flama: a Python framework for development and deployment of production-ready APIs, machine learning, and LLM services [0.9668407688201359]
  Flama is an open-source Python framework for developing and deploying production-ready web APIs, machine learning services, and large-model (LLM) applications.<n>It offers a type-driven, async-first programming model that unifies REST API development, predictive model serving, and generative AI inference in one architecture.
  arXiv  Detail & Related papers  (2026-08-19T09:37:00Z)
• ServerlessT2I: Efficient Text-to-Image Workflow Serving on a Serverless Platform [12.291915972120902]
  Text-to-image (T2I) are increasingly deployed on serverless platforms because users often compose customized and invoke them intermittently.<n>Existing platforms typically deploy each workflow as an opaque GPU function, provisioning, placing, and scaling all constituent models in the workflow together.<n>We present ServerlessT2I, a serverless-native system that decomposes a T2I workflow into loosely coupled model functions that can be independently managed and scheduled.
  arXiv  Detail & Related papers  (2026-07-29T07:39:56Z)
• Specification-Driven DevOps for Multi-Service Environments [0.0]
  Large Language Models (LLMs) are increasingly used to generate executable software environments from repository artifacts.<n>This study investigates whether a frontier LLM can generate Dockerfiles and Docker Compose for multi-service applications using repository contents.
  arXiv  Detail & Related papers  (2026-07-27T23:22:44Z)
• COLIP-2: Olfaction-Vision-Language Embeddings [0.0]
  The Contrastive Olfaction-Language-Image Pre-training 2 (COLIP-2) model is a multimodal embeddings space that places olfaction as a first-class citizen among vision and language.<n> Molecular structure, gas-sensor readings, odor-descriptor language, and images are all trained into a single shared representation space, so that a robot can localize a detected aroma to objects in a scene probabilistically.
  arXiv  Detail & Related papers  (2026-07-20T05:08:03Z)
• A Conversational Framework for Human-Robot Collaborative Manipulation with Distributed Generative AI models [0.0]
  This paper presents a distributed conversational framework for human-robot collaborative manipulation.<n>It integrates local language and vision-language models (VLMs) with a Robot Operating System 2 (ROS 2)-based execution stack.
  arXiv  Detail & Related papers  (2026-06-04T12:00:41Z)
• GTA: Generating Long-Horizon Tasks for Web Agents at Scale [82.43869456830664]
  We introduce a scalable framework, GTA, that integrates crawling, retrieval-based seeding, in-context generation, and automated quality control.<n>We instantiate the pipeline on over 50 websites covering e-commerce, government, forums, and news, with multilingual and multi-hop coverage.<n>Our contributions are three-fold: (i) formalizing multi-hop web-agent task generation, (ii) proposing an efficient and validated pipeline for automatic data creation, and (iii) releasing a dynamic benchmark with reproducible evaluation.
  arXiv  Detail & Related papers  (2026-05-28T01:05:50Z)
• A Serverless Edge-Native Data Processing Architecture for Autonomous Driving Training [0.0]
  This paper introduces the framework, an edge-native platform that enables on-vehicle data filtering and processing through user-defined functions.<n>We evaluate the framework on an NVIDIA Jetson Orin Nano and compare it against native ROS 2 deployments.<n>Results show competitive performance, reduced latency and jitter, and confirm that Lambda-based abstractions can support real-time data processing in embedded autonomous driving systems.
  arXiv  Detail & Related papers  (2026-01-30T12:41:11Z)
• TDHook: A Lightweight Framework for Interpretability [0.0]
  Interpretability of Deep Neural Networks (DNNs) is a growing field driven by the study of vision and language models.<n>We present TDHook, an open-source, lightweight, generic interpretability framework based on $texttttensordict$.<n>It focuses on handling complex composed models which can be trained for Computer Vision, Natural Language Processing, Reinforcement Learning or any other domain.
  arXiv  Detail & Related papers  (2025-09-29T20:28:43Z)
• OmniParser V2: Structured-Points-of-Thought for Unified Visual Text Parsing and Its Generality to Multimodal Large Language Models [58.45517851437422]
  Visually-situated text parsing (VsTP) has recently seen notable advancements, driven by the growing demand for automated document understanding.<n>Existing solutions often rely on task-specific architectures and objectives for individual tasks.<n>In this paper, we introduce Omni V2, a universal model that unifies VsTP typical tasks, including text spotting, key information extraction, table recognition, and layout analysis.
  arXiv  Detail & Related papers  (2025-02-22T09:32:01Z)
• Not All Prompts Are Made Equal: Prompt-based Pruning of Text-to-Image Diffusion Models [59.16287352266203]
  We introduce Adaptive Prompt-Tailored Pruning (APTP), a novel prompt-based pruning method for text-to-image (T2I) models.<n>APTP learns to determine the required capacity for an input text prompt and routes it to an architecture code, given a total desired compute budget for prompts.<n>APTP outperforms the single-model pruning baselines in terms of FID, CLIP, and CMMD scores.
  arXiv  Detail & Related papers  (2024-06-17T19:22:04Z)
• Multiscale Vision Transformers meet Bipartite Matching for efficient single-stage Action Localization [27.472705540825316]
  Action localization is a challenging problem that combines detection and recognition tasks, which are often addressed separately. We show that a single MViTv2-S architecture trained with bipartite matching to perform both tasks surpasses the same MViTv2-S when trained with RoI align on pre-computed bounding boxes.
  arXiv  Detail & Related papers  (2023-12-29T17:08:38Z)
• MiniGPT-v2: large language model as a unified interface for vision-language multi-task learning [65.60607895153692]
  MiniGPT-v2 is a model that can be treated as a unified interface for better handling various vision-language tasks. We propose using unique identifiers for different tasks when training the model. Our results show that MiniGPT-v2 achieves strong performance on many visual question-answering and visual grounding benchmarks.
  arXiv  Detail & Related papers  (2023-10-14T03:22:07Z)
• ESPnet-ST-v2: Multipurpose Spoken Language Translation Toolkit [61.52122386938913]
  ESPnet-ST-v2 is a revamp of the open-source ESPnet-ST toolkit. This paper describes the overall design, example models for each task, and performance benchmarking behind ESPnet-ST-v2.
  arXiv  Detail & Related papers  (2023-04-10T14:05:22Z)
• X2Parser: Cross-Lingual and Cross-Domain Framework for Task-Oriented Compositional Semantic Parsing [51.81533991497547]
  Task-oriented compositional semantic parsing (TCSP) handles complex nested user queries. We present X2 compared a transferable Cross-lingual and Cross-domain for TCSP. We propose to predict flattened intents and slots representations separately and cast both prediction tasks into sequence labeling problems.
  arXiv  Detail & Related papers  (2021-06-07T16:40:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.