Sequential Modality Dropout for Robust Multi-Modal Sequential Recommendation

• URL: http://arxiv.org/abs/2608.10240v1
• Date: Mon, 10 Aug 2026 21:18:25 GMT
• Title: Sequential Modality Dropout for Robust Multi-Modal Sequential Recommendation
• Abstract summary: Multi-modal sequential recommenders assume every item carries every modality, but real product catalogs often miss images or text.<n>We propose Sequential Modality Dropout (SMD)<n>During training, each modality stream (image and text) is independently erased with probability p for an entire user interaction history, so the model learns to predict the next item without relying on any single modality.
• Score: 16.247560684789534
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multi-modal sequential recommenders assume every item carries every modality, but real product catalogs often miss images or text, and a model trained on complete data loses much of its recommendation accuracy when a modality is unavailable at serving time. We propose Sequential Modality Dropout (SMD): during training, each modality stream (image and text) is independently erased with probability p for an entire user interaction history, so the model learns to predict the next item without relying on any single modality. We measure robustness by retention, the fraction of a model's full-modality accuracy (HR@10) that survives when a modality is removed at test time. Across four backbones (MM-SASRec, IISAN, MISSRec, and fMRLRec) on four Amazon domains, SMD raises text retention by 1.0 to 3.2x at essentially no cost to full-modality accuracy; under an extreme 95% per-item missing rate, it retains 61% of HR@10 versus 22% without (a 2.8x improvement). An optional cross-modal reconstruction loss further lifts retention from 90% to 98% on a simple additive backbone under severe text missingness. SMD is a four-line, architecture-agnostic change that makes multi-modal sequential recommenders robust to the missing modalities they actually encounter in deployment.

Related papers

• A Generative AI Integrated Multimodal Framework for Low-Latency Multi-Camera Person Re-Identification [0.18427704077999105]
  We propose a generative AI integrated multimodal ReID framework designed explicitly for robustness under missing cues and low latency deployment.<n>We report person re-identification performance using mAP and Rank-1 accuracy on the Market-1501 and DukeMTMC-reID benchmarks.
  arXiv  Detail & Related papers  (2026-09-13T10:40:38Z)
• Data Scarcity and Model Sparsity: Mixtures-of-Experts Overfit More to Repeated Data [119.14301046837822]
  We show that Mixture-of-Experts (MoEs) degrade more rapidly under data repetition.<n>MoEs can outperform dense models even when data is repeated more than 64 times.<n>We analyze internal mechanisms correlated with MoE overfitting in high repetition regimes.
  arXiv  Detail & Related papers  (2026-09-10T17:57:33Z)
• Omni2LoRA: Coherence-Preserving Parametric Memory for Efficient Omni Language Models [65.49950267695267]
  We introduce Omni2LoRA, a framework for efficient parametric memory compression via coherence-preserving context distillation.<n>By converting multimodal memory into a fixed-budget, reusable parameter state, our method drives answer-time multimodal-token load to zero.
  arXiv  Detail & Related papers  (2026-08-10T07:49:10Z)
• Meta-Modal Agent: Sequential Evidence Routing for Missing-Modality Candidate Reranking [12.476704430465754]
  We introduce the Meta-Modal Agent (MMA), a large language model based candidate-pool reranker that treats missingness as a sequential evidence-routing problem.<n>MMA is trained with balanced missingnesstask reinforcement learning over masked-modality episodes.<n>MMA is evaluated in two variants: MMA-Auto, which uses only automated text, image, and graph tools, and MMA-Interactive, which additionally permits clarification questions grounded in surviving modalities as an upper-bound diagnostic.
  arXiv  Detail & Related papers  (2026-05-24T11:22:03Z)
• Calibrated Multimodal Representation Learning with Missing Modalities [100.55774771852468]
  Multimodal representation learning harmonizes distinct modalities by aligning them into a unified latent space.<n>Recent research generalizes traditional cross-modal alignment to produce enhanced multimodal synergy but requires all modalities to be present for a common instance.<n>We provide theoretical insights into this issue from an anchor shift perspective.<n>We propose CalMRL for multimodal representation learning to calibrate incomplete alignments caused by missing modalities.
  arXiv  Detail & Related papers  (2025-11-15T05:01:43Z)
• ReID5o: Achieving Omni Multi-modal Person Re-identification in a Single Model [59.00754756072231]
  We investigate a new challenging problem called Omni Multi-modal Person Re-identification (OM-ReID)<n>We construct ORBench, the first high-quality multi-modal dataset comprising 1,000 unique identities across five modalities.<n>We also propose ReID5o, a novel multi-modal learning framework for person ReID.
  arXiv  Detail & Related papers  (2025-06-11T04:26:13Z)
• RMMSS: Towards Advanced Robust Multi-Modal Semantic Segmentation with Hybrid Prototype Distillation and Feature Selection [9.418241223504252]
  We present RMMSS, a two-stage framework designed to enhance model robustness under missing-modality conditions.<n>It comprises two key components: the Hybrid Prototype Distillation Module (HPDM) and the Feature Selection Module (FSM)<n>Our experiments on three datasets demonstrate that our method improves missing-modality performance by 2.80%, 3.89%, and 0.89%, respectively.
  arXiv  Detail & Related papers  (2025-05-19T08:46:03Z)
• Benchmarking Multi-modal Semantic Segmentation under Sensor Failures: Missing and Noisy Modality Robustness [61.87055159919641]
  Multi-modal semantic segmentation (MMSS) addresses the limitations of single-modality data by integrating complementary information across modalities.<n>Despite notable progress, a significant gap persists between research and real-world deployment due to variability and uncertainty in multi-modal data quality.<n>We introduce a robustness benchmark that evaluates MMSS models under three scenarios: Entire-Missing Modality (EMM), Random-Missing Modality (RMM), and Noisy Modality (NM)
  arXiv  Detail & Related papers  (2025-03-24T08:46:52Z)
• Robust Modality-incomplete Anomaly Detection: A Modality-instructive Framework with Benchmark [69.02666229531322]
  We introduce a pioneering study that investigates Modality-Incomplete Industrial Anomaly Detection (MIIAD)<n>We find that most existing MIAD methods perform poorly on the MIIAD Bench, leading to significant performance degradation.<n>We propose a novel two-stage Robust modAlity-aware fusing and Detecting framewoRk, abbreviated as RADAR.
  arXiv  Detail & Related papers  (2024-10-02T16:47:55Z)
• Dealing with All-stage Missing Modality: Towards A Universal Model with Robust Reconstruction and Personalization [14.606035444283984]
  Current approaches focus on developing models that handle modality-incomplete inputs during inference. We propose a robust universal model with modality reconstruction and model personalization. Our method has been extensively validated on two brain tumor segmentation benchmarks.
  arXiv  Detail & Related papers  (2024-06-04T06:07:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.