Short-Form Video Recommendations with Multimodal Embeddings: Addressing Cold-Start and Bias Challenges

• URL: http://arxiv.org/abs/2507.19346v1
• Date: Fri, 25 Jul 2025 14:57:04 GMT
• Title: Short-Form Video Recommendations with Multimodal Embeddings: Addressing Cold-Start and Bias Challenges
• Authors: Andrii Dzhoha, Katya Mirylenka, Egor Malykh, Marco-Andrea Buchmann, Francesca Catino,
• Abstract summary: In recent years, social media users have spent significant amounts of time on short-form video platforms.<n>established platforms in other domains, such as e-commerce, have begun introducing short-form video content to engage users.<n>This creates new challenges for recommender systems, especially when launching a new video experience.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In recent years, social media users have spent significant amounts of time on short-form video platforms. As a result, established platforms in other domains, such as e-commerce, have begun introducing short-form video content to engage users and increase their time spent on the platform. The success of these experiences is due not only to the content itself but also to a unique UI innovation: instead of offering users a list of choices to click, platforms actively recommend content for users to watch one at a time. This creates new challenges for recommender systems, especially when launching a new video experience. Beyond the limited interaction data, immersive feed experiences introduce stronger position bias due to the UI and duration bias when optimizing for watch-time, as models tend to favor shorter videos. These issues, together with the feedback loop inherent in recommender systems, make it difficult to build effective solutions. In this paper, we highlight the challenges faced when introducing a new short-form video experience and present our experience showing that, even with sufficient video interaction data, it can be more beneficial to leverage a video retrieval system using a fine-tuned multimodal vision-language model to overcome these challenges. This approach demonstrated greater effectiveness compared to conventional supervised learning methods in online experiments conducted on our e-commerce platform.

Related papers

• ARC-Hunyuan-Video-7B: Structured Video Comprehension of Real-World Shorts [56.75723197779384]
  ARC-Hunyuan-Video is a multimodal model that processes visual, audio, and textual signals end-to-end for structured comprehension.<n>Our model is capable of multi-granularity timestamped video captioning and summarization, open-ended video question answering, temporal video grounding, and video reasoning.
  arXiv  Detail & Related papers  (2025-07-28T15:52:36Z)
• Vid-SME: Membership Inference Attacks against Large Video Understanding Models [56.31088116526825]
  We introduce Vid-SME, the first membership inference method tailored for video data used in video understanding models (VULLMs)<n>By leveraging the SME difference between natural and temporally-reversed video frames, Vid-SME derives robust membership scores to determine whether a given video is part of the model's training set.<n> Experiments on various self-trained and open-sourced VULLMs demonstrate the strong effectiveness of Vid-SME.
  arXiv  Detail & Related papers  (2025-05-29T13:17:25Z)
• Switch-a-View: View Selection Learned from Unlabeled In-the-wild Videos [71.01549400773197]
  We introduce SWITCH-A-VIEW, a model that learns to automatically select the viewpoint to display at each timepoint when creating a how-to video.<n>We pose a pretext task that pseudo-labels segments in the training videos for their primary viewpoint.<n>We then discover the patterns between the visual and spoken content in a how-to video on the one hand and its view-switch moments on the other hand.
  arXiv  Detail & Related papers  (2024-12-24T12:16:43Z)
• Shorter Is Different: Characterizing the Dynamics of Short-Form Video Platforms [10.078299014855622]
  We conduct a large-scale data-driven analysis of Kuaishou, one of the largest short-form video platforms in China. Based on 248 million videos uploaded to the platform across all categories, we identify their notable differences from long-form video platforms. We find that videos are shortened by multiples on Kuaishou, with distinctive categorical distributions over-represented by life-related rather than interest-based videos.
  arXiv  Detail & Related papers  (2024-10-21T14:37:26Z)
• SWaT: Statistical Modeling of Video Watch Time through User Behavior Analysis [15.246875830547056]
  We propose a white-box statistical framework that translates various user behavior assumptions in watching (short) videos into statistical watch time models.<n>We test our models extensively on two public datasets, a large-scale offline industrial dataset, and an online A/B test on a short video platform with hundreds of millions of daily-active users.
  arXiv  Detail & Related papers  (2024-08-14T18:19:35Z)
• ExpertAF: Expert Actionable Feedback from Video [81.46431188306397]
  We introduce a novel method to generate actionable feedback from video of a person doing a physical activity, such as basketball or soccer.<n>Our method takes a video demonstration and its accompanying 3D body pose and generates expert commentary describing what the person is doing well and what they could improve.<n>We show how to leverage Ego-Exo4D's [29] videos of skilled activity and expert commentary together with a strong language model to create a weakly-supervised training dataset for this task.
  arXiv  Detail & Related papers  (2024-08-01T16:13:07Z)
• Multimodal Pretraining and Generation for Recommendation: A Tutorial [54.07497722719509]
  The tutorial comprises three parts: multimodal pretraining, multimodal generation, and industrial applications. It aims to facilitate a swift understanding of multimodal recommendation and foster meaningful discussions on the future development of this evolving landscape.
  arXiv  Detail & Related papers  (2024-05-11T06:15:22Z)
• User Welfare Optimization in Recommender Systems with Competing Content Creators [65.25721571688369]
  In this study, we perform system-side user welfare optimization under a competitive game setting among content creators. We propose an algorithmic solution for the platform, which dynamically computes a sequence of weights for each user based on their satisfaction of the recommended content. These weights are then utilized to design mechanisms that adjust the recommendation policy or the post-recommendation rewards, thereby influencing creators' content production strategies.
  arXiv  Detail & Related papers  (2024-04-28T21:09:52Z)
• ChatVideo: A Tracklet-centric Multimodal and Versatile Video Understanding System [119.51012668709502]
  We present our vision for multimodal and versatile video understanding and propose a prototype system, system. Our system is built upon a tracklet-centric paradigm, which treats tracklets as the basic video unit. All the detected tracklets are stored in a database and interact with the user through a database manager.
  arXiv  Detail & Related papers  (2023-04-27T17:59:58Z)
• Two-Stage Constrained Actor-Critic for Short Video Recommendation [23.12631658373264]
  We formulate the problem of short video recommendation as a Constrained Markov Decision Process (CMDP) We propose a novel two-stage constrained actor-critic method to optimize each auxiliary signal. Our method significantly outperforms other baselines in terms of both watch time and interactions.
  arXiv  Detail & Related papers  (2023-02-03T12:02:54Z)
• Constrained Reinforcement Learning for Short Video Recommendation [18.492477839791274]
  Short videos on social media platforms pose new challenges to optimize recommender systems. We propose a two-stage reinforcement learning approach based on actor-critic framework. Our approach has been fully launched in the production system to optimize user experiences.
  arXiv  Detail & Related papers  (2022-05-26T09:36:20Z)
• Short Video-based Advertisements Evaluation System: Self-Organizing Learning Approach [22.2568038582329]
  We propose a novel end-to-end self-organizing framework for user behavior prediction. Our model is able to learn the optimal topology of neural network architecture, as well as optimal weights, through training data.
  arXiv  Detail & Related papers  (2020-10-23T20:52:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.