Budget-Aware Keyboardless Interaction

• URL: http://arxiv.org/abs/2606.26508v1
• Date: Thu, 25 Jun 2026 01:27:13 GMT
• Title: Budget-Aware Keyboardless Interaction
• Abstract summary: This paper proposes a novel virtual keyboard system that leverages only a standard camera and a paper with a printed keyboard layout.<n>Our approach can work on standard environment using modern computer vision technologies.
• Score: 11.27056623462154
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Interacting with computers typically relies on traditional input devices such as keyboards, mice, and monitors, which can be cumbersome for users seeking greater mobility. Virtual keyboards have been explored to address these limitations, but they often involve complex setups or expensive equipment. This paper proposes a novel virtual keyboard system that leverages only a standard camera and a paper with a printed keyboard layout. Unlike previous methods requiring complex calibration or special lighting conditions, our approach can work on standard environment using modern computer vision technologies. Combining modern segmentation and detection models with traditional image processing algorithms, we efficiently identify the keyboard region. Touch detection is performed using an algorithm analyzing the color of the user's fingernail. Experiments demonstrated a promising results our proposed solution of keyboard and keystroke detection for practical applications. Participants attended our user study also found the proposed system interesting.

Related papers

• QUACK! Making the (Rubber) Ducky Talk: A Systematic Study of Keystroke Dynamics for HID Injection Detection [60.105690843777666]
  We show that robust, privacy-preserving detection is achievable using lightweight models operating solely on timing features.<n>Our analysis reveals that attacker sophistication does not monotonically translate into improved evasion.
  arXiv  Detail & Related papers  (2026-04-17T08:53:24Z)
• Doodle Your Keypoints: Sketch-Based Few-Shot Keypoint Detection [80.90808879991182]
  Keypoint detection faces challenges in few-shot learning, particularly when source data from the same distribution as the query is unavailable.<n>This gap is addressed by leveraging sketches, a popular form of human expression, providing a source-free alternative.<n>Our proposed framework overcomes these hurdles with a prototypical setup, combined with a grid-based locator and prototypical domain adaptation.
  arXiv  Detail & Related papers  (2025-07-10T17:59:49Z)
• Keystroke Verification Challenge (KVC): Biometric and Fairness Benchmark Evaluation [21.63351064421652]
  Keystroke dynamics (KD) for biometric verification has several advantages. KD is among the most discriminative behavioral traits. We present a new experimental framework to benchmark KD-based biometric verification performance and fairness.
  arXiv  Detail & Related papers  (2023-11-10T11:23:28Z)
• Typing on Any Surface: A Deep Learning-based Method for Real-Time Keystroke Detection in Augmented Reality [4.857109990499532]
  Mid-air keyboard interface, wireless keyboards or voice input, either suffer from poor ergonomic design, limited accuracy, or are simply embarrassing to use in public. This paper proposes and validates a deep-learning based approach, that enables AR applications to accurately predict keystrokes from the user perspective RGB video stream. A two-stage model, combing an off-the-shelf hand landmark extractor and a novel adaptive Convolutional Recurrent Neural Network (C-RNN) was trained.
  arXiv  Detail & Related papers  (2023-08-31T23:58:25Z)
• Agile gesture recognition for capacitive sensing devices: adapting on-the-job [55.40855017016652]
  We demonstrate a hand gesture recognition system that uses signals from capacitive sensors embedded into the etee hand controller. The controller generates real-time signals from each of the wearer five fingers. We use a machine learning technique to analyse the time series signals and identify three features that can represent 5 fingers within 500 ms.
  arXiv  Detail & Related papers  (2023-05-12T17:24:02Z)
• SiLK -- Simple Learned Keypoints [11.208547877814574]
  Keypoint detection & descriptors are foundational tech-nologies for computer vision tasks like image matching, 3D reconstruction and visual odometry. Recent learning-based methods employ a vast diversity of experimental setups and design choices. We re-design each component from first-principle and propose Simple Learned Keypoints (SiLK) that is fully-differentiable, lightweight, and flexible.
  arXiv  Detail & Related papers  (2023-04-12T23:56:00Z)
• Effective Gesture Based Framework for Capturing User Input [0.4588028371034407]
  Users of virtual keyboards can type on any surface as if it were a keyboard thanks to sensor technology and artificial intelligence. A camera is used to capture keyboard images and finger movements which subsequently acts as a virtual keyboard. A visible virtual mouse that accepts finger coordinates as input is also described in this study.
  arXiv  Detail & Related papers  (2022-08-01T14:58:17Z)
• Muscle Vision: Real Time Keypoint Based Pose Classification of Physical Exercises [52.77024349608834]
  3D human pose recognition extrapolated from video has advanced to the point of enabling real-time software applications. We propose a new machine learning pipeline and web interface that performs human pose recognition on a live video feed to detect when common exercises are performed and classify them accordingly.
  arXiv  Detail & Related papers  (2022-03-23T00:55:07Z)
• Improving Compositionality of Neural Networks by Decoding Representations to Inputs [83.97012077202882]
  We bridge the benefits of traditional and deep learning programs by jointly training a generative model to constrain neural network activations to "decode" back to inputs. We demonstrate applications of decodable representations to out-of-distribution detection, adversarial examples, calibration, and fairness.
  arXiv  Detail & Related papers  (2021-06-01T20:07:16Z)
• TypeNet: Deep Learning Keystroke Biometrics [77.80092630558305]
  We introduce TypeNet, a Recurrent Neural Network trained with a moderate number of keystrokes per identity. With 5 gallery sequences and test sequences of length 50, TypeNet achieves state-of-the-art keystroke biometric authentication performance. Our experiments demonstrate a moderate increase in error with up to 100,000 subjects, demonstrating the potential of TypeNet to operate at an Internet scale.
  arXiv  Detail & Related papers  (2021-01-14T12:49:09Z)
• Gestop : Customizable Gesture Control of Computer Systems [0.3553493344868413]
  Gestop is a framework that learns to detect gestures from demonstrations and is customizable by end-users. It enables users to interact in real-time with computers having only RGB cameras, using gestures.
  arXiv  Detail & Related papers  (2020-10-25T19:13:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.