BabyVLM-V2: Toward Developmentally Grounded Pretraining and Benchmarking of Vision Foundation Models

• URL: http://arxiv.org/abs/2512.10932v1
• Date: Thu, 11 Dec 2025 18:57:05 GMT
• Title: BabyVLM-V2: Toward Developmentally Grounded Pretraining and Benchmarking of Vision Foundation Models
• Abstract summary: We introduce BabyVLM-V2, a developmentally grounded framework for infant-inspired vision-language modeling.<n>The pretraining set maximizes coverage while minimizing curation of a longitudinal, infant-centric audiovisual corpus.<n>DevCV Toolbox adapts all vision-related measures of the recently released NIH Baby Toolbox into a benchmark suite of ten multimodal tasks.
• Score: 69.84938298826121
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Early children's developmental trajectories set up a natural goal for sample-efficient pretraining of vision foundation models. We introduce BabyVLM-V2, a developmentally grounded framework for infant-inspired vision-language modeling that extensively improves upon BabyVLM-V1 through a longitudinal, multifaceted pretraining set, a versatile model, and, most importantly, DevCV Toolbox for cognitive evaluation. The pretraining set maximizes coverage while minimizing curation of a longitudinal, infant-centric audiovisual corpus, yielding video-utterance, image-utterance, and multi-turn conversational data that mirror infant experiences. DevCV Toolbox adapts all vision-related measures of the recently released NIH Baby Toolbox into a benchmark suite of ten multimodal tasks, covering spatial reasoning, memory, and vocabulary understanding aligned with early children's capabilities. Experimental results show that a compact model pretrained from scratch can achieve competitive performance on DevCV Toolbox, outperforming GPT-4o on some tasks. We hope the principled, unified BabyVLM-V2 framework will accelerate research in developmentally plausible pretraining of vision foundation models.

Related papers

• GEM: Generative Supervision Helps Embodied Intelligence [60.28190143133428]
  Embodied Vision-Language Models (VLMs) have demonstrated impressive performance and generalization in robotics.<n>A significant gap remains between the high-level semantic focus of standard text-guided pre-training paradigms and the low-level spatial and physical knowledge critical for execution in embodied environments.<n>We introduce GEM, a Generative-supervised Embodied vision-language Model designed to bridge this divide.
  arXiv  Detail & Related papers  (2026-05-27T14:39:42Z)
• EgoBabyVLM: Benchmarking Cross-Modal Learning from Naturalistic Egocentric Video Data [20.729468513123756]
  Children acquire language grounding with remarkable robustness from limited visuo-linguistic input.<n>Current vision-language models fail to generalize to the sparse, weakly-aligned egocentric streams produced by wearable devices, embodied agents, and infant head-cams.<n>We introduce the EgoBabyVLM Challenge to drive the development of models capable of grounded language learning from the kind of naturalistic data that human infants experience.
  arXiv  Detail & Related papers  (2026-05-18T21:30:54Z)
• InVitroVision: a Multi-Modal AI Model for Automated Description of Embryo Development using Natural Language [0.09381376621526816]
  We investigate whether foundational vision-language models can be fine-tuned to predict natural language descriptions of embryo morphology and development.<n>Using a publicly available embryo time-lapse dataset, we fine-tuned PaliGemma-2, a multi-modal vision-language model, with only 1,000 images and corresponding captions.<n>Our results show that the fine-tuned model, InVitroVision, outperformed a commercial model, ChatGPT 5.2, and base models in overall metrics.
  arXiv  Detail & Related papers  (2026-04-22T20:05:37Z)
• PokeVLA: Empowering Pocket-Sized Vision-Language-Action Model with Comprehensive World Knowledge Guidance [24.102770290097435]
  PokeVLA is a lightweight model for embodied manipulation that infuses vision-language understanding into action learning.<n>Our framework introduces a two-stage training paradigm: first, we pre-train a compact vision-language model (PokeVLM) on a multimodal dataset of 2.4M samples.<n>Experiments demonstrate state-of-the-art performance on the LIBERO-Plus benchmark and in real-world deployment.
  arXiv  Detail & Related papers  (2026-04-22T17:58:19Z)
• A Training-Free Guess What Vision Language Model from Snippets to Open-Vocabulary Object Detection [16.166979262501425]
  Open-Vocabulary Object Detection (OVOD) aims to develop the capability to detect anything.<n>In this paper, a training-free Guess What Vision Language Model is proposed to form a universal understanding paradigm.<n>Our proposed GW-VLM can achieve superior OVOD performance compared to the-state-of-the-art methods without any training step.
  arXiv  Detail & Related papers  (2026-01-17T05:14:42Z)
• VITAL: Vision-Encoder-centered Pre-training for LMMs in Visual Quality Assessment [88.83260031198023]
  We propose a vision-encoder-centered generative pre-training pipeline and develop the VITAL-Series LMMs.<n>We construct over 4.5M vision-language (VL) pairs-the largest VQualA training dataset to date.<n>We employ a multi-task training workflow that simultaneously enhances the model's quantitative scoring precision.
  arXiv  Detail & Related papers  (2025-11-22T07:55:21Z)
• Rethinking Visual Intelligence: Insights from Video Pretraining [75.32388528274224]
  Large language models (LLMs) have demonstrated that large-scale pretraining enables systems to adapt rapidly to new problems.<n>We investigate Video Diffusion Models (VDMs) as a promising direction for bridging the gap.
  arXiv  Detail & Related papers  (2025-10-28T14:12:11Z)
• Kwai Keye-VL Technical Report [80.53170317017147]
  We introduce textbfKwai Keye-VL, a multimodal foundation model for short-video understanding.<n>The development of Keye-VL rests on two core pillars: a massive, high-quality dataset with a strong emphasis on video, and an innovative training recipe.<n>To validate our approach, we conduct extensive evaluations, showing that Keye-VL achieves state-of-the-art results on public video benchmarks and remains highly competitive on general image-based tasks.
  arXiv  Detail & Related papers  (2025-07-02T17:57:28Z)
• BabyVLM: Data-Efficient Pretraining of VLMs Inspired by Infant Learning [33.64851748019174]
  Human infants rapidly develop visual reasoning skills from minimal input.<n>Recent efforts have leveraged infant-inspired datasets like SAYCam.<n>We propose BabyVLM, a novel framework comprising comprehensive in-domain evaluation benchmarks and a synthetic training dataset.
  arXiv  Detail & Related papers  (2025-04-13T04:17:12Z)
• Vision-R1: Evolving Human-Free Alignment in Large Vision-Language Models via Vision-Guided Reinforcement Learning [26.14137626882127]
  Large Vision-Language Models (LVLMs) typically follow a two-stage training paradigm-pretraining and supervised fine-tuning.<n> preference optimization, derived from the language domain, has emerged as an effective post-training reinforcement strategy.<n>We propose Vision-R1, a novel vision-guided R1-like reinforcement learning algorithm for LVLMs that rewards models with definitive vision feedback.
  arXiv  Detail & Related papers  (2025-03-23T10:21:14Z)
• ReVLA: Reverting Visual Domain Limitation of Robotic Foundation Models [55.07988373824348]
  We study the visual generalization capabilities of three existing robotic foundation models.<n>Our study shows that the existing models do not exhibit robustness to visual out-of-domain scenarios.<n>We propose a gradual backbone reversal approach founded on model merging.
  arXiv  Detail & Related papers  (2024-09-23T17:47:59Z)
• Harnessing Vision-Language Pretrained Models with Temporal-Aware Adaptation for Referring Video Object Segmentation [34.37450315995176]
  Current Referring Video Object (RVOS) methods typically use vision and language models pretrained independently as backbones. We propose a temporal-aware prompt-tuning method, which adapts pretrained representations for pixel-level prediction. Our method performs favorably against state-of-the-art algorithms and exhibits strong generalization abilities.
  arXiv  Detail & Related papers  (2024-05-17T08:14:22Z)
• Unsupervised Vision-and-Language Pre-training Without Parallel Images and Captions [92.47566804182338]
  We investigate if a strong V&L representation model can be learned through unsupervised pre-training without image-caption corpora. In particular, we propose to conduct mask-and-predict'' pre-training on text-only and image-only corpora. We find that such a simple approach performance close to a model pre-trained with aligned data, on four English V&L benchmarks.
  arXiv  Detail & Related papers  (2020-10-24T08:17:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.