Freehand Sketching for End-User Programming of Robot Swarms

• URL: http://arxiv.org/abs/2609.11078v1
• Date: Thu, 10 Sep 2026 04:34:02 GMT
• Title: Freehand Sketching for End-User Programming of Robot Swarms
• Abstract summary: This paper investigates freehand sketching as an end-user programming interface for specifying robot swarms.<n>Users communicate spatial intent through a drawing, while the swarm autonomously extracts target formation points.<n>The resulting sketch-to-swarm framework is evaluated through a human study examining the usability of freehand formation specification.
• Score: 5.463395053888906
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Robot swarms are increasingly used in applications where accessible interaction with non-expert users is desirable. This paper investigates freehand sketching as an end-user programming interface for specifying robot swarm geometries. Users communicate spatial intent through a drawing, while the swarm autonomously extracts target formation points, constructs a rigid formation graph, assigns robots to formation nodes, and executes distributed formation control with a guarantee against unintended reflected formations. The resulting sketch-to-swarm framework is evaluated through a human study examining the usability of freehand formation specification. Twenty participants generated $42$ geometric shapes, and the interface achieved a mean System Usability Scale score of $84.25$, which conventionally indicates high perceived usability. The results support freehand sketching as an intuitive interaction abstraction for human-swarm collaboration without requiring robotics or programming expertise.

Related papers

• Robot Learning from Human Demonstrations: Handwritten Alphabet Trajectories and Human-Likeness Evaluation [0.26763498831034044]
  This paper presents a framework for learning human-like robot motion from demonstration.<n>A dataset of 3,142 handwriting demonstrations was collected from 22 participants.<n>A user study with 21 participants evaluated the perceived human-likeness of the generated trajectories.
  arXiv  Detail & Related papers  (2026-08-06T16:12:18Z)
• Interactive Robot Programming for Surface Finishing via Task-Centric Mixed Reality Interfaces [5.67537424643785]
  Collaborative robots are rarely used for surface finishing in small-scale craft and manufacturing settings.<n>New surface segmentation algorithm incorporates human input to identify regions for processing.<n>Users receive continuous visual feedback on the robot's learned model.
  arXiv  Detail & Related papers  (2025-12-29T17:21:01Z)
• RoboPEPP: Vision-Based Robot Pose and Joint Angle Estimation through Embedding Predictive Pre-Training [27.63332596592781]
  Vision-based pose estimation of articulated robots with unknown joint angles has applications in collaborative robotics and human-robot interaction tasks.<n>Current frameworks use neural network encoders to extract image features and downstream layers to predict joint angles and robot pose.<n>We introduce RoboPEPP, a method that fuses information about the robot's physical model into the encoder using a masking-based self-supervised embedding-predictive architecture.
  arXiv  Detail & Related papers  (2024-11-26T18:26:17Z)
• Real-time Addressee Estimation: Deployment of a Deep-Learning Model on the iCub Robot [52.277579221741746]
  Addressee Estimation is a skill essential for social robots to interact smoothly with humans. Inspired by human perceptual skills, a deep-learning model for Addressee Estimation is designed, trained, and deployed on an iCub robot. The study presents the procedure of such implementation and the performance of the model deployed in real-time human-robot interaction.
  arXiv  Detail & Related papers  (2023-11-09T13:01:21Z)
• Gesture2Path: Imitation Learning for Gesture-aware Navigation [54.570943577423094]
  We present Gesture2Path, a novel social navigation approach that combines image-based imitation learning with model-predictive control. We deploy our method on real robots and showcase the effectiveness of our approach for the four gestures-navigation scenarios.
  arXiv  Detail & Related papers  (2022-09-19T23:05:36Z)
• I Know What You Draw: Learning Grasp Detection Conditioned on a Few Freehand Sketches [74.63313641583602]
  We propose a method to generate a potential grasp configuration relevant to the sketch-depicted objects. Our model is trained and tested in an end-to-end manner which is easy to be implemented in real-world applications.
  arXiv  Detail & Related papers  (2022-05-09T04:23:36Z)
• Can Foundation Models Perform Zero-Shot Task Specification For Robot Manipulation? [54.442692221567796]
  Task specification is critical for engagement of non-expert end-users and adoption of personalized robots. A widely studied approach to task specification is through goals, using either compact state vectors or goal images from the same robot scene. In this work, we explore alternate and more general forms of goal specification that are expected to be easier for humans to specify and use.
  arXiv  Detail & Related papers  (2022-04-23T19:39:49Z)
• Online Body Schema Adaptation through Cost-Sensitive Active Learning [63.84207660737483]
  The work was implemented in a simulation environment, using the 7DoF arm of the iCub robot simulator. A cost-sensitive active learning approach is used to select optimal joint configurations. The results show cost-sensitive active learning has similar accuracy to the standard active learning approach, while reducing in about half the executed movement.
  arXiv  Detail & Related papers  (2021-01-26T16:01:02Z)
• Projection Mapping Implementation: Enabling Direct Externalization of Perception Results and Action Intent to Improve Robot Explainability [62.03014078810652]
  Existing research on non-verbal cues, e.g., eye gaze or arm movement, may not accurately present a robot's internal states. Projecting the states directly onto a robot's operating environment has the advantages of being direct, accurate, and more salient.
  arXiv  Detail & Related papers  (2020-10-05T18:16:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.