AC-Lite : A Lightweight Image Captioning Model for Low-Resource Assamese Language

• URL: http://arxiv.org/abs/2503.01453v2
• Date: Wed, 30 Apr 2025 08:01:00 GMT
• Title: AC-Lite : A Lightweight Image Captioning Model for Low-Resource Assamese Language
• Abstract summary: This work presents AC-Lite, a computationally efficient model for image captioning in low-resource Assamese language.<n>A combination of ShuffleNetv2x1.5 with GRU based language decoder along with bilinear attention is found to provide the best performance with minimum compute.<n>AC-Lite was observed to achieve an 82.3 CIDEr score on the COCO-AC dataset with 2.45 GFLOPs and 22.87M parameters.
• Score: 3.7440657375085986
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Most existing works in image caption synthesis use computation heavy deep neural networks and generates image descriptions in English language. This often restricts this important assistive tool for widespread use across language and accessibility barriers. This work presents AC-Lite, a computationally efficient model for image captioning in low-resource Assamese language. AC-Lite reduces computational requirements by replacing computation-heavy deep network components with lightweight alternatives. The AC-Lite model is designed through extensive ablation experiments with different image feature extractor networks and language decoders. A combination of ShuffleNetv2x1.5 with GRU based language decoder along with bilinear attention is found to provide the best performance with minimum compute. AC-Lite was observed to achieve an 82.3 CIDEr score on the COCO-AC dataset with 2.45 GFLOPs and 22.87M parameters.

Related papers

• AnySimLite: A Lightweight Few-Shot Similarity Encoder for On-Device Speech-Adjacent Classification [2.5057127311354885]
  We show that AnySimLite can consistently achieve state-of-the-art (SOTA) or SOTA-competitive performance in few-shot settings.<n>Even in the worst case, the performance drop remains below 7% while using $frac1250mathrmth$ of the model size of the SOTA qLLaMA_LoRA-7B baseline.
  arXiv  Detail & Related papers  (2026-06-24T23:25:28Z)
• ELIP: Enhanced Visual-Language Foundation Models for Image Retrieval [83.01358520910533]
  We introduce a new framework that can boost the performance of large-scale pre-trained vision- curation models. The approach, Enhanced Language-Image Pre-training (ELIP), uses the text query, via a simple mapping network, to predict a set of visual prompts. ELIP can easily be applied to the commonly used CLIP, SigLIP and BLIP-2 networks.
  arXiv  Detail & Related papers  (2025-02-21T18:59:57Z)
• Lightweight Operations for Visual Speech Recognition [5.254384872541785]
  We develop lightweight visual speech recognition architectures for resource-constrained devices.<n>We train and evaluate our models on a large-scale public dataset for recognition of words from video sequences.
  arXiv  Detail & Related papers  (2025-02-07T11:08:32Z)
• MuLan: Adapting Multilingual Diffusion Models for Hundreds of Languages with Negligible Cost [39.43930856789352]
  We find that, unlike models tuned on high-quality images with multilingual annotations, leveraging text encoders pre-trained on widely available, noisy Internet image-text pairs significantly enhances data efficiency. We introduce MuLan, Multi-Language adapter, a lightweight language adapter with fewer than 20M parameters, trained alongside a frozen text encoder and image diffusion model.
  arXiv  Detail & Related papers  (2024-12-02T08:38:19Z)
• How to Learn a New Language? An Efficient Solution for Self-Supervised Learning Models Unseen Languages Adaption in Low-Resource Scenario [72.02391485962127]
  Speech Self-Supervised Learning (SSL) models achieve impressive performance on Automatic Speech Recognition (ASR)<n>In low-resource language ASR, they encounter the domain mismatch problem between pre-trained and low-resource languages.<n>We extend a conventional efficient fine-tuning scheme based on the adapter to handle these issues.
  arXiv  Detail & Related papers  (2024-11-27T10:51:00Z)
• LowCLIP: Adapting the CLIP Model Architecture for Low-Resource Languages in Multimodal Image Retrieval Task [0.0]
  This research explores the development of vision-language models for image retrieval in low-resource languages, specifically Azerbaijani. We integrated the CLIP model architecture and employed several techniques to balance computational efficiency with performance. Our study found that models like EfficientNet0 and Tiny Swin Transformer perform best on the datasets they were trained on.
  arXiv  Detail & Related papers  (2024-08-25T18:10:16Z)
• SOLO: A Single Transformer for Scalable Vision-Language Modeling [74.05173379908703]
  We present SOLO, a single transformer for visiOn-Language mOdeling.<n>A unified single Transformer architecture, like SOLO, effectively addresses these scalability concerns in LVLMs.<n>In this paper, we introduce the first open-source training recipe for developing SOLO, an open-source 7B LVLM.
  arXiv  Detail & Related papers  (2024-07-08T22:40:15Z)
• Gated Low-rank Adaptation for personalized Code-Switching Automatic Speech Recognition on the low-spec devices [28.06179341376626]
  We introduce a gated low-rank adaptation(GLoRA) for parameter-efficient fine-tuning with minimal performance degradation. Our experiments, conducted on Korean-English code-switching datasets, demonstrate that fine-tuning speech recognition models for code-switching surpasses the performance of traditional code-switching speech recognition models trained from scratch.
  arXiv  Detail & Related papers  (2024-04-24T01:31:39Z)
• Fine-tuning CLIP Text Encoders with Two-step Paraphrasing [83.3736789315201]
  We introduce a straightforward fine-tuning approach to enhance the representations of CLIP models for paraphrases. Our model, which we call ParaCLIP, exhibits significant improvements over baseline CLIP models across various tasks.
  arXiv  Detail & Related papers  (2024-02-23T06:11:50Z)
• Attention-based UNet enabled Lightweight Image Semantic Communication System over Internet of Things [4.62215026195301]
  We study the problem of the lightweight image semantic communication system that is deployed on Internet of Things (IoT) devices. We propose an attention-based UNet enabled lightweight image semantic communication (LSSC) system, which achieves low computational complexity and small model size.
  arXiv  Detail & Related papers  (2024-01-14T16:46:50Z)
• Cross-Speaker Encoding Network for Multi-Talker Speech Recognition [74.97576062152709]
  Cross-MixSpeaker. Network addresses limitations of SIMO models by aggregating cross-speaker representations. Network is integrated with SOT to leverage both the advantages of SIMO and SISO.
  arXiv  Detail & Related papers  (2024-01-08T16:37:45Z)
• CAPIVARA: Cost-Efficient Approach for Improving Multilingual CLIP Performance on Low-Resource Languages [3.760470440988674]
  CAPIVARA is a cost-efficient framework designed to enhance the performance of multilingual CLIP models in low-resource languages. We use image captioning and machine translation to generate synthetic captions in low-resource languages. We show the potential for significant improvements in other low-resource languages, achieved by fine-tuning the pre-trained multilingual CLIP on a single GPU for 2 hours.
  arXiv  Detail & Related papers  (2023-10-20T17:44:25Z)
• Adapting the adapters for code-switching in multilingual ASR [10.316724084739892]
  Large pre-trained multilingual speech models have shown potential in scaling Automatic Speech Recognition to many low-resource languages. Some of these models employ language adapters in their formulation, which helps to improve monolingual performance. This formulation restricts the usability of these models on code-switched speech, where two languages are mixed together in the same utterance. We propose ways to effectively fine-tune such models on code-switched speech, by assimilating information from both language adapters at each language adaptation point in the network.
  arXiv  Detail & Related papers  (2023-10-11T12:15:24Z)
• Towards Practical and Efficient Image-to-Speech Captioning with Vision-Language Pre-training and Multi-modal Tokens [87.52235889917223]
  We set the output of the proposed Im2Sp as discretized speech units, i.e., the quantized speech features of a self-supervised speech model. With the vision-language pre-training strategy, we set new state-of-the-art Im2Sp performances on two widely used benchmark databases.
  arXiv  Detail & Related papers  (2023-09-15T16:48:34Z)
• Babel-ImageNet: Massively Multilingual Evaluation of Vision-and-Language Representations [53.89380284760555]
  We introduce Babel-ImageNet, a massively multilingual benchmark that offers partial translations of ImageNet labels to 100 languages. We evaluate 11 public multilingual CLIP models on our benchmark, demonstrating a significant gap between English ImageNet performance and that of high-resource languages. We show that the performance of multilingual CLIP can be drastically improved for low-resource languages with parameter-efficient language-specific training.
  arXiv  Detail & Related papers  (2023-06-14T17:53:06Z)
• Model Blending for Text Classification [0.15229257192293197]
  We try reducing the complexity of state of the art LSTM models for natural language tasks such as text classification, by distilling their knowledge to CNN based models, thus reducing the inference time(or latency) during testing.
  arXiv  Detail & Related papers  (2022-08-05T05:07:45Z)
• Multimodal Knowledge Alignment with Reinforcement Learning [103.68816413817372]
  ESPER extends language-only zero-shot models to unseen multimodal tasks, like image and audio captioning. Our key novelty is to use reinforcement learning to align multimodal inputs to language model generations without direct supervision. Experiments demonstrate that ESPER outperforms baselines and prior work on a variety of zero-shot tasks.
  arXiv  Detail & Related papers  (2022-05-25T10:12:17Z)
• Event Based Time-Vectors for auditory features extraction: a neuromorphic approach for low power audio recognition [4.206844212918807]
  We present a neuromorphic architecture, capable of unsupervised auditory feature recognition. We then validate the network on a subset of Google's Speech Commands dataset.
  arXiv  Detail & Related papers  (2021-12-13T21:08:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.