Parameter Efficient Multimodal Instruction Tuning for Romanian Vision Language Models

• URL: http://arxiv.org/abs/2512.14926v1
• Date: Tue, 16 Dec 2025 21:36:28 GMT
• Title: Parameter Efficient Multimodal Instruction Tuning for Romanian Vision Language Models
• Abstract summary: We contribute to reducing the multimodal NLP resource gap for Romanian.<n>We translate the widely known Flickr30k dataset into Romanian and further extend it for visual question answering.<n>We demonstrate the usefulness of our datasets by fine-tuning open-source VLMs on Romanian visual question answering.
• Score: 2.1990085634399428
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Focusing on low-resource languages is an essential step toward democratizing generative AI. In this work, we contribute to reducing the multimodal NLP resource gap for Romanian. We translate the widely known Flickr30k dataset into Romanian and further extend it for visual question answering by leveraging open-source LLMs. We demonstrate the usefulness of our datasets by fine-tuning open-source VLMs on Romanian visual question answering. We select VLMs from three widely used model families: LLaMA 3.2, LLaVA 1.6, and Qwen2. For fine-tuning, we employ the parameter-efficient LoRA method. Our models show improved Romanian capabilities in visual QA, as well as on tasks they were not trained on, such as Romanian image description generation. The seven-billion-parameter Qwen2-VL-RoVQA obtains top scores on both tasks, with improvements of +6.05% and +2.61% in BERTScore F1 over its original version. Finally, the models show substantial reductions in grammatical errors compared to their original forms, indicating improvements not only in language understanding but also in Romanian fluency.

Related papers

• Cross-lingual Relation Extraction with Large Language Models: Zero-Shot, Few-Shot, and Fine-Tuned Evaluation on Romanian [2.929334248463484]
  We translate the SemEval-2010 Task 8 benchmark from English to Romanian using an LLM-based translation pipeline.<n>We evaluate Gemma 4 31B under zero-shot, few-shot, and QLoRA fine-tuned configurations.<n>Our results show that Romanian incurs a 3 to 5 percentage point (pp) drop relative to English in prompt-only settings.
  arXiv  Detail & Related papers  (2026-06-30T14:22:46Z)
• "Înţelegi Româneşte?'' A Recipe for Romanian Vision-Language Models [7.120569645707792]
  We present a systematic study of building a language-specific Vision-Language Models (VLMs) for Romanian.<n>We translate established English VLM training and evaluation corpora into Romanian, applying machine translation to textual annotations and to in-image text.<n>Using this data, we train and ablate a series of VLMs to isolate the contribution of vision backbones of varying scale and pretraining.<n>We also curate HoraVQA, a culturally native evaluation set grounded in Romanian everyday scenes.
  arXiv  Detail & Related papers  (2026-05-29T15:04:20Z)
• Probing Visual Language Priors in VLMs [51.016683265437536]
  We introduce ViLP, a benchmark featuring deliberately out-of-distribution images.<n>Each question in ViLP is coupled with three potential answers and three corresponding images.<n>We propose a self-improving framework in which models generate new VQA data, then apply pixel-level and semantic corruptions to form "good-bad" image pairs for self-training.
  arXiv  Detail & Related papers  (2024-12-31T17:54:29Z)
• Looking Beyond Text: Reducing Language bias in Large Vision-Language Models via Multimodal Dual-Attention and Soft-Image Guidance [67.26434607115392]
  Large vision-language models (LVLMs) have achieved impressive results in various vision-language tasks. LVLMs suffer from hallucinations caused by language bias, leading to diminished focus on images and ineffective visual comprehension. We propose LACING to address the language bias of LVLMs with muLtimodal duAl-attention meChanIsm (MDA) aNd soft-image Guidance (IFG)
  arXiv  Detail & Related papers  (2024-11-21T16:33:30Z)
• Think Carefully and Check Again! Meta-Generation Unlocking LLMs for Low-Resource Cross-Lingual Summarization [108.6908427615402]
  Cross-lingual summarization ( CLS) aims to generate a summary for the source text in a different target language.<n>Currently, instruction-tuned large language models (LLMs) excel at various English tasks.<n>Recent studies have shown that LLMs' performance on CLS tasks remains unsatisfactory even with few-shot settings.
  arXiv  Detail & Related papers  (2024-10-26T00:39:44Z)
• RoQLlama: A Lightweight Romanian Adapted Language Model [2.1892046440619626]
  We tackle the problem of reduced computing resources by using QLoRA for training. We release RoQLlama-7b, a quantized LLM, which shows equal or improved results compared to its full-sized counterpart. We introduce a novel Romanian dataset, namely RoMedQA, which contains single-choice medical questions in Romanian.
  arXiv  Detail & Related papers  (2024-10-05T19:14:11Z)
• Amharic LLaMA and LLaVA: Multimodal LLMs for Low Resource Languages [0.0]
  Large Language Models (LLMs) have shown incredible proficiency at natural language processing tasks. LLMs often struggle to perform well on low-resource languages because there is so little training data available. In this work, we explore training LLaMA-2 to speak Amharic, a language which is spoken by over 50 million people world wide.
  arXiv  Detail & Related papers  (2024-03-11T01:04:36Z)
• YAYI 2: Multilingual Open-Source Large Language Models [53.92832054643197]
  We propose YAYI 2, including both base and chat models, with 30 billion parameters. YAYI 2 is pre-trained from scratch on a multilingual corpus which contains 2.65 trillion tokens filtered by our pre-training data processing pipeline. The base model is aligned with human values through supervised fine-tuning with millions of instructions and reinforcement learning from human feedback.
  arXiv  Detail & Related papers  (2023-12-22T17:34:47Z)
• Rephrase, Augment, Reason: Visual Grounding of Questions for Vision-Language Models [59.05769810380928]
  Rephrase, Augment and Reason (RepARe) is a gradient-free framework that extracts salient details about the image using the underlying vision-language model. We show that RepARe can result in a 3.85% (absolute) increase in zero-shot accuracy on VQAv2, 6.41%, and 7.94% points increase on A-OKVQA, and VizWiz respectively.
  arXiv  Detail & Related papers  (2023-10-09T16:57:57Z)
• PolyLM: An Open Source Polyglot Large Language Model [57.64420154135178]
  We present PolyLM, a multilingual large language model (LLMs) trained on 640 billion (B) tokens, avaliable in two model sizes: 1.7B and 13B. To enhance its multilingual capabilities, we 1) integrate bilingual data into training data; and 2) adopt a curriculum learning strategy that increases the proportion of non-English data from 30% in the first stage to 60% in the final stage during pre-training. Further, we propose a multilingual self-instruct method which automatically generates 132.7K diverse multilingual instructions for model fine-tuning.
  arXiv  Detail & Related papers  (2023-07-12T09:00:37Z)
• Boosting Zero-shot Cross-lingual Retrieval by Training on Artificially Code-Switched Data [26.38449396649045]
  We show that the effectiveness of zero-shot rankers diminishes when queries and documents are present in different languages. Motivated by this, we propose to train ranking models on artificially code-switched data instead.
  arXiv  Detail & Related papers  (2023-05-09T09:32:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.