DECKER: Domain-invariant Embedding for Cross-Keyboard Extraction and Recognition

• URL: http://arxiv.org/abs/2605.03384v1
• Date: Tue, 05 May 2026 05:43:24 GMT
• Title: DECKER: Domain-invariant Embedding for Cross-Keyboard Extraction and Recognition
• Abstract summary: Acoustic side-channel attacks (ASCA) on keyboards pose a significant security risk.<n>HEAR is a dataset designed to study ASCA along three axes: keyboard generalization, noise adaptation, and user bias.<n>We propose DECKER, a domain-invariant keystroke inference framework with four stages.
• Score: 7.244731714427563
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Acoustic side-channel attacks (ASCA) on keyboards pose a significant security risk, as keystrokes can be inferred from typing acoustics, revealing sensitive information. Prior ASCA studies are limited by small-scale datasets with restricted diversity in users, keyboards, and environments, constraining analysis across devices, microphones, and noise conditions. We introduce HEAR, a dataset designed to study ASCA along three axes: keyboard generalization, noise adaptation, and user bias. HEAR contains recordings from 53 participants using 37 laptop keyboards, collected in three realistic settings: (1) external microphone capture, (2) device microphone capture without network noise, and (3) VoIP-based streaming capture. This enables controlled evaluation across users, keyboards, and environments. On HEAR, we establish an ASCA benchmark spanning conventional features and pre-trained representations from raw audio and spectrograms in unimodal and multimodal settings. We propose DECKER, a domain-invariant keystroke inference framework with four stages: (1) Keyboard Signature Normalization to reduce device coloration, (2) domain-adversarial disentanglement to suppress keyboard identity, (3) supervised cross-keyboard contrastive alignment to enforce key consistency, and (4) Acoustic Style Randomization to synthesize unseen keyboard responses. We further explore sentence-level inference using an LLM-based post-processing layer to refine keystroke sequences via linguistic context. Results on HEAR show DECKER improves keystroke identification over strong baselines, particularly in cross-keyboard and cross-user settings, with further gains from language-model rectification. These findings highlight that ASCA remains effective across diverse users, devices, and noisy environments, underscoring its practical security risk.

Related papers

• Transforming Keystroke Noise to Text: Self-Supervised Acoustic Eavesdropping Attacks on Keyboards [5.107513187852866]
  We present a self-supervised acoustic eavesdropping attack that reconstructs typed text solely from keystroke sounds.<n>The proposed attack enables stealthy eavesdropping in two real-world scenarios-physical spaces and online meetings.
  arXiv  Detail & Related papers  (2026-07-24T08:43:16Z)
• RadKey: An LLM-Guided RF Backscatter System for Through-Wall Keystroke Inference [7.054093620465402]
  RadKey is an RF backscatter system for covert, long-range, through-wall keystroke eavesdropping.<n>Tag captures keystroke-induced vibrations and acoustic signals, modulating them onto the frequency shift of its backscattered RF signal.<n>RF reader demodulates the backscattered RF signal to infer typed content.
  arXiv  Detail & Related papers  (2026-06-08T20:26:41Z)
• Still Between Us? Evaluating and Improving Voice Assistant Robustness to Third-Party Interruptions [52.45054413627452]
  TPI-Train is a dataset of 88K instances designed with speaker-aware hard negatives to enforce acoustic cue prioritization for interruption handling.<n>TPI-Bench is a comprehensive evaluation framework designed to rigorously measure the interruption-handling strategy.
  arXiv  Detail & Related papers  (2026-04-19T10:03:42Z)
• DecepGPT: Schema-Driven Deception Detection with Multicultural Datasets and Robust Multimodal Learning [64.33887406863899]
  Multimodal deception detection aims to identify deceptive behavior by analyzing audiovisual cues for forensics and security.<n>Existing benchmarks provide only binary labels without intermediate reasoning cues.<n>We construct reasoning datasets with structured cue-level descriptions and reasoning chains.<n>With 1695 samples, it is the largest non-laboratory deception detection dataset.
  arXiv  Detail & Related papers  (2026-03-25T04:06:36Z)
• An Agent-Based Modeling Approach to Free-Text Keyboard Dynamics for Continuous Authentication [0.0]
  Continuous authentication systems leveraging free-text keyboard dynamics offer a promising additional layer of security in a multifactor authentication setup.<n>This study investigates the efficacy of behavioral biometrics by employing an Agent-Based Model (ABM) to simulate diverse typing profiles across mechanical and membrane keyboards.
  arXiv  Detail & Related papers  (2025-05-08T07:42:05Z)
• Acoustic Side Channel Attack on Keyboards Based on Typing Patterns [0.0]
  Side-channel attacks on keyboards can bypass security measures in many systems that use keyboards as one of the input devices. This paper proposes an applicable method that takes into account the user's typing pattern in a realistic environment. Our method achieved an average success rate of 43% across all our case studies when considering real-world scenarios.
  arXiv  Detail & Related papers  (2024-03-13T17:44:15Z)
• Keystroke Verification Challenge (KVC): Biometric and Fairness Benchmark Evaluation [21.63351064421652]
  Keystroke dynamics (KD) for biometric verification has several advantages. KD is among the most discriminative behavioral traits. We present a new experimental framework to benchmark KD-based biometric verification performance and fairness.
  arXiv  Detail & Related papers  (2023-11-10T11:23:28Z)
• OverHear: Headphone based Multi-sensor Keystroke Inference [1.9915929143641455]
  We develop a keystroke inference framework that leverages both acoustic and accelerometer data from headphones. We achieve top-5 key prediction accuracy of around 80% for mechanical keyboards and around 60% for membrane keyboards. Results highlight the effectiveness and limitations of our approach in the context of real-world scenarios.
  arXiv  Detail & Related papers  (2023-11-04T00:48:20Z)
• Sign Language Recognition via Skeleton-Aware Multi-Model Ensemble [71.97020373520922]
  Sign language is commonly used by deaf or mute people to communicate. We propose a novel Multi-modal Framework with a Global Ensemble Model (GEM) for isolated Sign Language Recognition ( SLR) Our proposed SAM- SLR-v2 framework is exceedingly effective and achieves state-of-the-art performance with significant margins.
  arXiv  Detail & Related papers  (2021-10-12T16:57:18Z)
• Speech Enhancement for Wake-Up-Word detection in Voice Assistants [60.103753056973815]
  Keywords spotting and in particular Wake-Up-Word (WUW) detection is a very important task for voice assistants. This paper proposes a Speech Enhancement model adapted to the task of WUW detection. It aims at increasing the recognition rate and reducing the false alarms in the presence of these types of noises.
  arXiv  Detail & Related papers  (2021-01-29T18:44:05Z)
• SpEx: Multi-Scale Time Domain Speaker Extraction Network [89.00319878262005]
  Speaker extraction aims to mimic humans' selective auditory attention by extracting a target speaker's voice from a multi-talker environment. It is common to perform the extraction in frequency-domain, and reconstruct the time-domain signal from the extracted magnitude and estimated phase spectra. We propose a time-domain speaker extraction network (SpEx) that converts the mixture speech into multi-scale embedding coefficients instead of decomposing the speech signal into magnitude and phase spectra.
  arXiv  Detail & Related papers  (2020-04-17T16:13:06Z)
• End-to-End Automatic Speech Recognition Integrated With CTC-Based Voice Activity Detection [48.80449801938696]
  This paper integrates a voice activity detection function with end-to-end automatic speech recognition. We focus on connectionist temporal classification ( CTC) and its extension ofsynchronous/attention. We use the labels as a cue for detecting speech segments with simple thresholding.
  arXiv  Detail & Related papers  (2020-02-03T03:36:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.