Image-to-Video Transfer Learning based on Image-Language Foundation Models: A Comprehensive Survey

• URL: http://arxiv.org/abs/2510.10671v1
• Date: Sun, 12 Oct 2025 15:56:02 GMT
• Title: Image-to-Video Transfer Learning based on Image-Language Foundation Models: A Comprehensive Survey
• Authors: Jinxuan Li, Chaolei Tan, Haoxuan Chen, Jianxin Ma, Jian-Fang Hu, Wei-Shi Zheng, Jianhuang Lai,
• Abstract summary: Image-Language Foundation Models (ILFM) have demonstrated remarkable success in image-text understanding/generation tasks.<n>This survey provides the first comprehensive review of this emerging field.
• Score: 86.96983249116614
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Image-Language Foundation Models (ILFM) have demonstrated remarkable success in image-text understanding/generation tasks, providing transferable multimodal representations that generalize across diverse downstream image-based tasks. The advancement of video-text research has spurred growing interest in extending image-based models to the video domain. This paradigm, known as image-to-video transfer learning, succeeds in alleviating the substantial data and computational requirements associated with training video-language foundation models from scratch for video-text learning. This survey provides the first comprehensive review of this emerging field, which begins by summarizing the widely used ILFM and their capabilities. We then systematically classify existing image-to-video transfer learning strategies into two categories: frozen features and modified features, depending on whether the original representations from ILFM are preserved or undergo modifications. Building upon the task-specific nature of image-to-video transfer, this survey methodically elaborates these strategies and details their applications across a spectrum of video-text learning tasks, ranging from fine-grained (e.g., spatio-temporal video grounding) to coarse-grained (e.g., video question answering). We further present a detailed experimental analysis to investigate the efficacy of different image-to-video transfer learning paradigms on a range of downstream video understanding tasks. Finally, we identify prevailing challenges and highlight promising directions for future research. By offering a comprehensive and structured overview, this survey aims to establish a structured roadmap for advancing video-text learning based on existing ILFM, and to inspire future research directions in this rapidly evolving domain.

Related papers

• VLM2Vec-V2: Advancing Multimodal Embedding for Videos, Images, and Visual Documents [105.43882565434444]
  We propose VLM2Vec-V2, a unified framework for learning embeddings across diverse visual forms.<n>First, we introduce MMEB-V2, a comprehensive benchmark that extends MMEB with five new task types.<n>Next, we train VLM2Vec-V2, a general-purpose embedding model that supports text, image, video, and visual document inputs.
  arXiv  Detail & Related papers  (2025-07-07T00:51:57Z)
• VINCIE: Unlocking In-context Image Editing from Video [62.88977098700917]
  In this work, we explore whether an in-context image editing model can be learned directly from videos.<n>To effectively learn from this data, we design a block-causal diffusion transformer trained on three proxy tasks.<n>Our model exhibits strong in-context image editing capabilities and achieves state-of-the-art results on two multi-turn image editing benchmarks.
  arXiv  Detail & Related papers  (2025-06-12T17:46:54Z)
• Vision-Language Modeling Meets Remote Sensing: Models, Datasets and Perspectives [36.297745473653166]
  Vision-language modeling (VLM) aims to bridge the information gap between images and natural language.<n>Under the new paradigm of first pre-training on massive image-text pairs and then fine-tuning on task-specific data, VLM in the remote sensing domain has made significant progress.
  arXiv  Detail & Related papers  (2025-05-20T13:47:40Z)
• Video In-context Learning: Autoregressive Transformers are Zero-Shot Video Imitators [46.40277880351059]
  We explore utilizing visual signals as a new interface for models to interact with the environment.<n>We find that the model emerges a zero-shot capability to infer the semantics from a demonstration video, and imitate the semantics to an unseen scenario.<n>Results show that our models can generate high-quality video clips that accurately align with the semantic guidance provided by the demonstration videos.
  arXiv  Detail & Related papers  (2024-07-10T04:27:06Z)
• InternVideo2: Scaling Foundation Models for Multimodal Video Understanding [51.129913789991924]
  InternVideo2 is a new family of video foundation models (FM) that achieve state-of-the-art results in video recognition, video-speech tasks, and video-centric tasks. Our core design is a progressive training approach that unifies the masked video modeling, cross contrastive learning, and prediction token, scaling up to 6B video size.
  arXiv  Detail & Related papers  (2024-03-22T17:57:42Z)
• RTQ: Rethinking Video-language Understanding Based on Image-text Model [55.278942477715084]
  Video-language understanding presents unique challenges due to the inclusion of highly complex semantic details. We propose a novel framework called RTQ, which addresses these challenges simultaneously. Our model demonstrates outstanding performance even in the absence of video-language pre-training.
  arXiv  Detail & Related papers  (2023-12-01T04:51:01Z)
• Self-supervised video pretraining yields robust and more human-aligned visual representations [14.599429594703539]
  General representations far outperform prior video pretraining methods on image understanding tasks.<n>VITO representations are significantly more robust to natural and synthetic deformations than image-, video-, and adversarially-trained ones.<n>These results suggest that video pretraining could be a simple way of learning unified, robust, and human-aligned representations of the visual world.
  arXiv  Detail & Related papers  (2022-10-12T17:30:12Z)
• Self-Supervised Learning for Videos: A Survey [70.37277191524755]
  Self-supervised learning has shown promise in both image and video domains. In this survey, we provide a review of existing approaches on self-supervised learning focusing on the video domain.
  arXiv  Detail & Related papers  (2022-06-18T00:26:52Z)
• Self-Supervised Video Representation Learning with Motion-Contrastive Perception [13.860736711747284]
  Motion-Contrastive Perception Network (MCPNet) MCPNet consists of two branches, namely, Motion Information Perception (MIP) and Contrastive Instance Perception (CIP) Our method outperforms current state-of-the-art visual-only self-supervised approaches.
  arXiv  Detail & Related papers  (2022-04-10T05:34:46Z)
• Reading-strategy Inspired Visual Representation Learning for Text-to-Video Retrieval [41.420760047617506]
  Cross-modal representation learning projects both videos and sentences into common spaces for semantic similarity. Inspired by the reading strategy of humans, we propose a Reading-strategy Inspired Visual Representation Learning (RIVRL) to represent videos. Our model RIVRL achieves a new state-of-the-art on TGIF and VATEX.
  arXiv  Detail & Related papers  (2022-01-23T03:38:37Z)
• A Survey on Deep Learning Technique for Video Segmentation [147.0767454918527]
  Video segmentation plays a critical role in a broad range of practical applications. Deep learning based approaches have been dedicated to video segmentation and delivered compelling performance.
  arXiv  Detail & Related papers  (2021-07-02T15:51:07Z)
• Highlight Timestamp Detection Model for Comedy Videos via Multimodal Sentiment Analysis [1.6181085766811525]
  We propose a multimodal structure to obtain state-of-the-art performance in this field. We select several benchmarks for multimodal video understanding and apply the most suitable model to find the best performance.
  arXiv  Detail & Related papers  (2021-05-28T08:39:19Z)
• TiVGAN: Text to Image to Video Generation with Step-by-Step Evolutionary Generator [34.7504057664375]
  We propose a novel training framework, Text-to-Image-to-Video Generative Adversarial Network (TiVGAN), which evolves frame-by-frame and finally produces a full-length video. Step-by-step learning process helps stabilize the training and enables the creation of high-resolution video based on conditional text descriptions.
  arXiv  Detail & Related papers  (2020-09-04T06:33:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.