Language-Based Swarm Perception: Decentralized Person Re-Identification via Natural Language Descriptions

• URL: http://arxiv.org/abs/2601.12479v1
• Date: Sun, 18 Jan 2026 16:43:27 GMT
• Title: Language-Based Swarm Perception: Decentralized Person Re-Identification via Natural Language Descriptions
• Abstract summary: We introduce a method for decentralized person re-identification in robot swarms that leverages natural language as the primary representational modality.<n>Each robot locally detects and describes individuals using a vision-language model (VLM)<n>These descriptions are compared and clustered across the swarm without centralized coordination, allowing robots to collaboratively group observations of the same individual.
• Score: 11.456639842384506
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce a method for decentralized person re-identification in robot swarms that leverages natural language as the primary representational modality. Unlike traditional approaches that rely on opaque visual embeddings -- high-dimensional feature vectors extracted from images -- the proposed method uses human-readable language to represent observations. Each robot locally detects and describes individuals using a vision-language model (VLM), producing textual descriptions of appearance instead of feature vectors. These descriptions are compared and clustered across the swarm without centralized coordination, allowing robots to collaboratively group observations of the same individual. Each cluster is distilled into a representative description by a language model, providing an interpretable, concise summary of the swarm's collective perception. This approach enables natural-language querying, enhances transparency, and supports explainable swarm behavior. Preliminary experiments demonstrate competitive performance in identity consistency and interpretability compared to embedding-based methods, despite current limitations in text similarity and computational load. Ongoing work explores refined similarity metrics, semantic navigation, and the extension of language-based perception to environmental elements. This work prioritizes decentralized perception and communication, while active navigation remains an open direction for future study.

Related papers

• Conceptual Networks for Cross-Linguistic Idiomatic Expressions: A Feature-Based Graph Approach [0.0]
  We present an interpretable network-based framework for representing idiomatic and figurative meaning across eight typologically diverse languages.<n>Each expression is annotated with binary conceptual features derived from cognitive-linguistic theory, and pairwise Jaccard similarities define a weighted graph.<n>Community detection reveals that idioms cluster by conceptual schema rather than by language, producing a structure consistent with cognitive-linguistic predictions.
  arXiv  Detail & Related papers  (2026-07-10T16:25:51Z)
• A Multimodal Framework for Aligning Human Linguistic Descriptions with Visual Perceptual Data [0.0]
  We introduce a computational framework designed to model core aspects of human referential interpretation.<n>We evaluate the model on the Stanford Repeated Reference Game corpus.<n>Results suggest that relatively simple perceptual-linguistic alignment mechanisms can yield human-competitive behavior.
  arXiv  Detail & Related papers  (2026-02-23T07:20:11Z)
• Learning Human-Object Interaction as Groups [52.28258599873394]
  GroupHOI is a framework that propagates contextual information in terms of geometric proximity and semantic similarity.<n>It exhibits leading performance on the more challenging Nonverbal Interaction Detection task.
  arXiv  Detail & Related papers  (2025-10-21T07:25:10Z)
• ImaGGen: Zero-Shot Generation of Co-Speech Semantic Gestures Grounded in Language and Image Input [0.0]
  This paper tackles a core challenge in co-speech gesture synthesis: generating iconic or deictic gestures that are semantically coherent with a verbal utterance.<n>We introduce a zero-shot system that generates gestures from a given language input and additionally is informed by imagistic input, without manual annotation or human intervention.<n>Our results highlight the importance of context-aware semantic gestures for creating expressive and collaborative virtual agents or avatars.
  arXiv  Detail & Related papers  (2025-10-20T15:01:56Z)
• Identifying and interpreting non-aligned human conceptual representations using language modeling [0.0]
  We show that congenital blindness induces conceptual reorganization in both a-modal and sensory-related verbal domains. We find that blind individuals more strongly associate social and cognitive meanings to verbs related to motion. For some verbs, representations of blind and sighted are highly similar.
  arXiv  Detail & Related papers  (2024-03-10T13:02:27Z)
• Pixel Sentence Representation Learning [67.4775296225521]
  In this work, we conceptualize the learning of sentence-level textual semantics as a visual representation learning process. We employ visually-grounded text perturbation methods like typos and word order shuffling, resonating with human cognitive patterns, and enabling perturbation to be perceived as continuous. Our approach is further bolstered by large-scale unsupervised topical alignment training and natural language inference supervision.
  arXiv  Detail & Related papers  (2024-02-13T02:46:45Z)
• Decentralised Emergence of Robust and Adaptive Linguistic Conventions in Populations of Autonomous Agents Grounded in Continuous Worlds [4.63732827131233]
  This paper introduces a methodology through which a population of autonomous agents can establish a linguistic convention. The convention emerges in a decentralised manner through local communicative interactions between pairs of agents. We show that the methodology enables a population to converge on a communicatively effective, coherent and human-interpretable linguistic convention.
  arXiv  Detail & Related papers  (2024-01-16T16:11:35Z)
• Rewrite Caption Semantics: Bridging Semantic Gaps for Language-Supervised Semantic Segmentation [100.81837601210597]
  We propose Concept Curation (CoCu) to bridge the gap between visual and textual semantics in pre-training data. CoCu achieves superb zero-shot transfer performance and greatly boosts language-supervised segmentation baseline by a large margin.
  arXiv  Detail & Related papers  (2023-09-24T00:05:39Z)
• data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language [85.9019051663368]
  data2vec is a framework that uses the same learning method for either speech, NLP or computer vision. The core idea is to predict latent representations of the full input data based on a masked view of the input in a self-distillation setup. Experiments on the major benchmarks of speech recognition, image classification, and natural language understanding demonstrate a new state of the art or competitive performance.
  arXiv  Detail & Related papers  (2022-02-07T22:52:11Z)
• Understanding Synonymous Referring Expressions via Contrastive Features [105.36814858748285]
  We develop an end-to-end trainable framework to learn contrastive features on the image and object instance levels. We conduct extensive experiments to evaluate the proposed algorithm on several benchmark datasets.
  arXiv  Detail & Related papers  (2021-04-20T17:56:24Z)
• Quantifying Learnability and Describability of Visual Concepts Emerging in Representation Learning [91.58529629419135]
  We consider how to characterise visual groupings discovered automatically by deep neural networks. We introduce two concepts, visual learnability and describability, that can be used to quantify the interpretability of arbitrary image groupings.
  arXiv  Detail & Related papers  (2020-10-27T18:41:49Z)
• Generating Hierarchical Explanations on Text Classification via Feature Interaction Detection [21.02924712220406]
  We build hierarchical explanations by detecting feature interactions. Such explanations visualize how words and phrases are combined at different levels of the hierarchy. Experiments show the effectiveness of the proposed method in providing explanations both faithful to models and interpretable to humans.
  arXiv  Detail & Related papers  (2020-04-04T20:56:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.