FUTO Swipe: Layout-Agnostic Neural Swipe Decoding

• URL: http://arxiv.org/abs/2606.25247v1
• Date: Wed, 24 Jun 2026 00:11:09 GMT
• Title: FUTO Swipe: Layout-Agnostic Neural Swipe Decoding
• Abstract summary: Training neural models requires substantial data, but public swipe data is limited, particularly for non-QWERTY layouts.<n>We release swipe.futo.org, the largest MIT-licensed swipe corpus we are aware of, containing over 1M donated swipes from more than 12k donor sessions.<n>We apply geometric augmentations to both the swipe trajectory and the keyboard layout at every training step, forcing the model to make predictions based on characteristics of the swipe gesture rather than the training layout.
• Score: 45.88028371034407
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Neural swipe decoders are typically tied to the keyboard they were trained on, requiring a new corpus and training run for each layout. In this report, we document our approach toward training models that can function on any contiguous mobile keyboard layout. At each point along the swipe, our encoder predicts whether the user is indicating a character and where on the keyboard that character lies. The keyboard layout is supplied at inference time and used to map the spatial and temporal prediction to a logit at each key, rather than being learned during training. Training neural models requires substantial data, but public swipe data is limited, particularly for non-QWERTY layouts. We release swipe.futo.org, the largest MIT-licensed swipe corpus we are aware of, containing over 1M donated swipes from more than 12k donor sessions. To generalize beyond the English QWERTY layout, we apply geometric augmentations to both the swipe trajectory and the keyboard layout at every training step, forcing the model to make predictions based on characteristics of the swipe gesture rather than the training layout. The model generalizes to layouts absent from training, in some cases more accurately than the layout it was trained on. This combines the layout-flexibility of an algorithmic decoder with the accuracy of a neural model. Trained models are publicly available.

Related papers

• Device Passport: Enabling Spatio-Temporal Pretrained Models to Generalize Across Input Layouts [5.217148612121133]
  We study how different channel embedding techniques behave when pretraining layouts differ substantially from the downstream decoding layout.<n>We propose Device Passport, a new channel embedding technique that learns experts and mixture models that take each channel's functional activity and metadata as input.
  arXiv  Detail & Related papers  (2026-06-30T22:58:09Z)
• Pretraining Language Models to Ponder in Continuous Space [50.52734567589996]
  We introduce this pondering process into language models by repeatedly invoking the forward process within a single token generation step.<n>We show that the model can learn to ponder in this way through self-supervised learning, without any human annotations.
  arXiv  Detail & Related papers  (2025-05-27T03:47:33Z)
• HandReader: Advanced Techniques for Efficient Fingerspelling Recognition [75.38606213726906]
  This paper introduces HandReader, a group of three architectures designed to address the fingerspelling recognition task.<n>HandReader$_RGB$ employs the novel Adaptive Shift-Temporal Module (TSAM) to process RGB features from videos of varying lengths.<n>HandReader$_KP$ is built on the proposed Temporal Pose (TPE) operated on keypoints as tensors.<n>Each HandReader model possesses distinct advantages and achieves state-of-the-art results on the ChicagoFSWild and ChicagoFSWild+ datasets.
  arXiv  Detail & Related papers  (2025-05-15T13:18:37Z)
• TapType: Ten-finger text entry on everyday surfaces via Bayesian inference [32.33746932895968]
  TapType is a mobile text entry system for full-size typing on passive surfaces. From the inertial sensors inside a band on either wrist, TapType decodes and relates surface taps to a traditional QWERTY keyboard layout.
  arXiv  Detail & Related papers  (2024-10-08T12:58:31Z)
• Wavelet GPT: Wavelet Inspired Large Language Models [1.2328446298523066]
  Large Language Models (LLMs) have ushered in a new wave of artificial intelligence advancements.<n>This paper infuses LLMs with a traditional signal processing idea, namely wavelets, during pre-training to take advantage of the structure.<n>We achieve the same pre-training performance almost twice as fast in text, audio, and images.
  arXiv  Detail & Related papers  (2024-09-04T03:17:19Z)
• Typing on Any Surface: A Deep Learning-based Method for Real-Time Keystroke Detection in Augmented Reality [4.857109990499532]
  Mid-air keyboard interface, wireless keyboards or voice input, either suffer from poor ergonomic design, limited accuracy, or are simply embarrassing to use in public. This paper proposes and validates a deep-learning based approach, that enables AR applications to accurately predict keystrokes from the user perspective RGB video stream. A two-stage model, combing an off-the-shelf hand landmark extractor and a novel adaptive Convolutional Recurrent Neural Network (C-RNN) was trained.
  arXiv  Detail & Related papers  (2023-08-31T23:58:25Z)
• EfficientTrain: Exploring Generalized Curriculum Learning for Training Visual Backbones [80.662250618795]
  This paper presents a new curriculum learning approach for the efficient training of visual backbones (e.g., vision Transformers) As an off-the-shelf method, it reduces the wall-time training cost of a wide variety of popular models by >1.5x on ImageNet-1K/22K without sacrificing accuracy.
  arXiv  Detail & Related papers  (2022-11-17T17:38:55Z)
• ERNIE-Layout: Layout Knowledge Enhanced Pre-training for Visually-rich Document Understanding [52.3895498789521]
  We propose ERNIE, a novel document pre-training solution with layout knowledge enhancement. We first rearrange input sequences in the serialization stage, then present a correlative pre-training task, reading order prediction, and learn the proper reading order of documents. Experimental results show ERNIE achieves superior performance on various downstream tasks, setting new state-of-the-art on key information, and document question answering.
  arXiv  Detail & Related papers  (2022-10-12T12:59:24Z)
• Learning Rate Curriculum [75.98230528486401]
  We propose a novel curriculum learning approach termed Learning Rate Curriculum (LeRaC) LeRaC uses a different learning rate for each layer of a neural network to create a data-agnostic curriculum during the initial training epochs. We compare our approach with Curriculum by Smoothing (CBS), a state-of-the-art data-agnostic curriculum learning approach.
  arXiv  Detail & Related papers  (2022-05-18T18:57:36Z)
• Variational Information Bottleneck Model for Accurate Indoor Position Recognition [0.0]
  We propose a Variational Information Bottleneck model for accurate indoor positioning. The proposed model consists of an encoder structure and a predictor structure. We conduct the validation experiments on a real-world dataset.
  arXiv  Detail & Related papers  (2021-01-26T09:29:53Z)
• Neural networks behave as hash encoders: An empirical study [79.38436088982283]
  The input space of a neural network with ReLU-like activations is partitioned into multiple linear regions. We demonstrate that this partition exhibits the following encoding properties across a variety of deep learning models. Simple algorithms, such as $K$-Means, $K$-NN, and logistic regression, can achieve fairly good performance on both training and test data.
  arXiv  Detail & Related papers  (2021-01-14T07:50:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.