AnySimLite: A Lightweight Few-Shot Similarity Encoder for On-Device Speech-Adjacent Classification

• URL: http://arxiv.org/abs/2606.26452v1
• Date: Wed, 24 Jun 2026 23:25:28 GMT
• Title: AnySimLite: A Lightweight Few-Shot Similarity Encoder for On-Device Speech-Adjacent Classification
• Abstract summary: We show that AnySimLite can consistently achieve state-of-the-art (SOTA) or SOTA-competitive performance in few-shot settings.<n>Even in the worst case, the performance drop remains below 7% while using $frac1250mathrmth$ of the model size of the SOTA qLLaMA_LoRA-7B baseline.
• Score: 2.5057127311354885
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: To minimize privacy concerns and inference latency on edge devices like smartphones, lightweight on-device models remain important for end-user applications. Many of these applications involve natural language classification, but deploying multiple specialized models creates a memory footprint challenge. We investigate: Can a single lightweight architecture solve multiple Speech-Adjacent (SA) classification tasks through reduction to a nuanced text similarity formulation? We propose AnySimLite, a lightweight similarity encoder that combines word-level and character-level channels. Together with a dataset transformation strategy, we evaluate AnySimLite across multiple SA classification tasks and show that it consistently achieves state-of-the-art (SOTA) or SOTA-competitive performance in few-shot settings while maintaining a low memory footprint. Even in the worst case, the performance drop remains below 7% while using $<\frac{1}{250}^{\mathrm{th}}$ of the model size of the SOTA qLLaMA_LoRA-7B baseline.

Related papers

• Speech Emotion Recognition using Attention-based LSTM-Network with Residual Connection [0.0]
  Speech emotion recognition is an important component of modern human-computer interaction systems.<n>This paper proposes ResLSTM-SA, a lightweight architecture that integrates residual connections with soft attention.<n>The best-performing variant (ResLSTM-SA-h64) achieves a maximum UAR of 0.6517 with only 46.8k trainable parameters.
  arXiv  Detail & Related papers  (2026-06-02T09:08:59Z)
• Resource-Efficient Iterative LLM-Based NAS with Feedback Memory [49.44875022114861]
  Neural Architecture Search (NAS) automates network design, but conventional methods demand substantial computational resources.<n>We propose a closed-loop pipeline leveraging large language models (LLMs) to iteratively generate, evaluate, and refine convolutional neural network architectures.
  arXiv  Detail & Related papers  (2026-03-12T16:00:22Z)
• Beyond Short-Horizon: VQ-Memory for Robust Long-Horizon Manipulation in Non-Markovian Simulation Benchmarks [96.60530830276281]
  RuleSafe is a new articulated manipulation benchmark built upon a scalable LLM-aided simulation framework.<n>VQ-Memory is a compact and structured temporal representation that uses vector-quantized variational autoencoders.
  arXiv  Detail & Related papers  (2026-03-10T11:13:54Z)
• AHELM: A Holistic Evaluation of Audio-Language Models [78.20477815156484]
  multimodal audio-language models (ALMs) take interleaved audio and text as input and output text.<n>AHELM is a benchmark that aggregates various datasets -- including 2 new synthetic audio-text datasets called PARADE and CoRe-Bench.<n>We also standardize the prompts, inference parameters, and evaluation metrics to ensure equitable comparisons across models.
  arXiv  Detail & Related papers  (2025-08-29T07:40:39Z)
• AeroLite: Tag-Guided Lightweight Generation of Aerial Image Captions [5.67477841586604]
  textbfAeroLite is a tag-guided captioning framework for remote sensing images.<n>textbfAeroLite leverages GPT-4o to generate a large-scale, semantically rich pseudo-caption dataset.<n>We propose a novel bridging multilayer perceptron (MLP) architecture, aligning semantic tags with visual embeddings.
  arXiv  Detail & Related papers  (2025-04-13T11:29:31Z)
• Efficient Multi-Instance Generation with Janus-Pro-Dirven Prompt Parsing [53.295515505026096]
  Janus-Pro-driven Prompt Parsing is a prompt- parsing module that bridges text understanding and layout generation.<n>MIGLoRA is a parameter-efficient plug-in integrating Low-Rank Adaptation into UNet (SD1.5) and DiT (SD3) backbones.<n>The proposed method achieves state-of-the-art performance on COCO and LVIS benchmarks while maintaining parameter efficiency.
  arXiv  Detail & Related papers  (2025-03-27T00:59:14Z)
• AC-Lite : A Lightweight Image Captioning Model for Low-Resource Assamese Language [3.7440657375085986]
  This work presents AC-Lite, a computationally efficient model for image captioning in low-resource Assamese language.<n>A combination of ShuffleNetv2x1.5 with GRU based language decoder along with bilinear attention is found to provide the best performance with minimum compute.<n>AC-Lite was observed to achieve an 82.3 CIDEr score on the COCO-AC dataset with 2.45 GFLOPs and 22.87M parameters.
  arXiv  Detail & Related papers  (2025-03-03T12:07:52Z)
• Cross-Speaker Encoding Network for Multi-Talker Speech Recognition [74.97576062152709]
  Cross-MixSpeaker. Network addresses limitations of SIMO models by aggregating cross-speaker representations. Network is integrated with SOT to leverage both the advantages of SIMO and SISO.
  arXiv  Detail & Related papers  (2024-01-08T16:37:45Z)
• Application-Agnostic Language Modeling for On-Device ASR [6.03523493247947]
  On-device automatic speech recognition systems face several challenges compared to server-based systems. They have to meet stricter constraints in terms of speed, disk size and memory. One of our novel approaches reduces the disk size by half, while maintaining speed and accuracy of the original model.
  arXiv  Detail & Related papers  (2023-05-16T19:31:18Z)
• LACoS-BLOOM: Low-rank Adaptation with Contrastive objective on 8 bits Siamese-BLOOM [2.9327503320877457]
  We present 8-bit Siamese-BLOOM, a multilingual large language model optimized to produce semantically meaningful word embeddings. We fine-tune BLOOM with a scalable adapter (LoRA) and 8-bit Adam for sentence similarity classification. Experiment results show the quality of learned embeddings from LACoS-BLOOM is proportional to the number of model parameters and the amount of unlabeled training data.
  arXiv  Detail & Related papers  (2023-05-10T18:26:42Z)
• LiteTransformerSearch: Training-free On-device Search for Efficient Autoregressive Language Models [34.673688610935876]
  We show that the latency and perplexity pareto-frontier can be found without need for any model training. We evaluate our method, dubbed Lightweight Transformer Search (LTS), on diverse devices. We show that the perplexity of Transformer-XL can be achieved with up to 2x lower latency.
  arXiv  Detail & Related papers  (2022-03-04T02:10:43Z)
• LightSpeech: Lightweight and Fast Text to Speech with Neural Architecture Search [127.56834100382878]
  We propose LightSpeech to automatically design more lightweight and efficient TTS models based on FastSpeech. Experiments show that the model discovered by our method achieves 15x model compression ratio and 6.5x inference speedup on CPU with on par voice quality.
  arXiv  Detail & Related papers  (2021-02-08T07:45:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.