MOON3.0: Reasoning-aware Multimodal Representation Learning for E-commerce Product Understanding

• URL: http://arxiv.org/abs/2604.00513v2
• Date: Thu, 02 Apr 2026 12:17:15 GMT
• Title: MOON3.0: Reasoning-aware Multimodal Representation Learning for E-commerce Product Understanding
• Abstract summary: We argue that leveraging the reasoning capabilities of MLLMs to explicitly model fine-grained product attributes holds significant potential.<n>We propose MOON3.0, the first reasoning-aware MLLM-based model for product representation learning.
• Score: 13.985610562698348
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: With the rapid growth of e-commerce, exploring general representations rather than task-specific ones has attracted increasing attention. Although recent multimodal large language models (MLLMs) have driven significant progress in product understanding, they are typically employed as feature extractors that implicitly encode product information into global embeddings, thereby limiting their ability to capture fine-grained attributes. Therefore, we argue that leveraging the reasoning capabilities of MLLMs to explicitly model fine-grained product attributes holds significant potential. Nevertheless, achieving this goal remains non-trivial due to several key challenges: (i) long-context reasoning tends to dilute the model's attention to salient information in the raw input; (ii) supervised fine-tuning (SFT) primarily encourages rigid imitation, limiting the exploration of effective reasoning strategies; and (iii) fine-grained details are progressively attenuated during forward propagation. To address these issues, we propose MOON3.0, the first reasoning-aware MLLM-based model for product representation learning. Our method (1) employs a multi-head modality fusion module to adaptively integrate raw signals; (2) incorporates a joint contrastive and reinforcement learning framework to autonomously explore more effective reasoning strategies; and (3) introduces a fine-grained residual enhancement module to progressively preserve local details throughout the network. Additionally, we release a large-scale multimodal e-commerce benchmark MBE3.0. Experimentally, our model demonstrates state-of-the-art zero-shot performance across various downstream tasks on both our benchmark and public datasets.

Related papers

• A Comprehensive Information-Decomposition Analysis of Large Vision-Language Models [24.18585407100395]
  Large vision-language models (LVLMs) achieve impressive performance, yet their internal decision-making processes remain opaque.<n>We introduce a novel framework using partial information decomposition (PID) to quantitatively measure the "information spectrum" of LVLMs.<n>Our analysis reveals two key results: (i) two task regimes (synergy-driven vs. knowledge-driven) and (ii) two stable, contrasting family-level strategies (fusion-centric vs. language-centric)
  arXiv  Detail & Related papers  (2026-03-31T12:32:29Z)
• Actial: Activate Spatial Reasoning Ability of Multimodal Large Language Models [75.45940282834327]
  We introduce Viewpoint Learning, a task designed to evaluate and improve the spatial reasoning capabilities of MLLMs.<n>We present the Viewpoint-100K dataset, consisting of 100K object-centric image pairs with diverse viewpoints and corresponding question-answer pairs.<n>Our approach employs a two-stage fine-tuning strategy, resulting in significant improvements across multiple tasks.
  arXiv  Detail & Related papers  (2025-11-03T14:27:00Z)
• Think Then Embed: Generative Context Improves Multimodal Embedding [51.76690812535934]
  We propose a Think-Then-Embed (TTE) framework for Universal Multimodal Embeddings (UME), composed of a reasoner and an embedder.<n>By leveraging a powerful MLLM reasoner, we achieve state-of-the-art performance on the MMEB-V2 benchmark, surpassing proprietary models trained on massive in-house datasets.
  arXiv  Detail & Related papers  (2025-10-06T16:53:56Z)
• MOON: Generative MLLM-based Multimodal Representation Learning for E-commerce Product Understanding [19.89836326556511]
  We argue that generative Multimodal Large Language Models hold significant potential for improving product representation learning.<n>We propose the first generative MLLM-based model named MOON for product representation learning.<n>Our method employs a guided Mixture-of-Experts (MoE) module for targeted modeling of multimodal and aspect-specific product content.
  arXiv  Detail & Related papers  (2025-08-16T09:59:25Z)
• MoCa: Modality-aware Continual Pre-training Makes Better Bidirectional Multimodal Embeddings [75.0617088717528]
  MoCa is a framework for transforming pre-trained VLM backbones into effective bidirectional embedding models.<n>MoCa consistently improves performance across MMEB and ViDoRe-v2 benchmarks, achieving new state-of-the-art results.
  arXiv  Detail & Related papers  (2025-06-29T06:41:00Z)
• Enhanced Multimodal Aspect-Based Sentiment Analysis by LLM-Generated Rationales [7.119479942471737]
  Existing methods rely on pre-trained small language models (SLMs) to collect information related to aspects and sentiments from both image and text.<n>We propose a novel framework, termed LRSA, which combines the decision-making capabilities of SLMs with additional information provided by LLMs for MABSA.
  arXiv  Detail & Related papers  (2025-05-20T15:28:26Z)
• X2-DFD: A framework for eXplainable and eXtendable Deepfake Detection [55.77552681618732]
  X2-DFD is an eXplainable and eXtendable framework based on multimodal large-language models (MLLMs) for deepfake detection.<n>The first stage, Model Feature Assessment, systematically evaluates the detectability of forgery-related features for the MLLM.<n>The second stage, Explainable dataset Construction, consists of two key modules: Strong Feature Strengthening and Weak Feature Supplementing.<n>The third stage, Fine-tuning and Inference, involves fine-tuning the MLLM on the constructed dataset and deploying it for final detection and explanation.
  arXiv  Detail & Related papers  (2024-10-08T15:28:33Z)
• Fine-tuning Multimodal Large Language Models for Product Bundling [53.01642741096356]
  We introduce Bundle-MLLM, a novel framework that fine-tunes large language models (LLMs) through a hybrid item tokenization approach.<n>Specifically, we integrate textual, media, and relational data into a unified tokenization, introducing a soft separation token to distinguish between textual and non-textual tokens.<n>We propose a progressive optimization strategy that fine-tunes LLMs for disentangled objectives: 1) learning bundle patterns and 2) enhancing multimodal semantic understanding specific to product bundling.
  arXiv  Detail & Related papers  (2024-07-16T13:30:14Z)
• Improving Open Information Extraction with Large Language Models: A Study on Demonstration Uncertainty [52.72790059506241]
  Open Information Extraction (OIE) task aims at extracting structured facts from unstructured text. Despite the potential of large language models (LLMs) like ChatGPT as a general task solver, they lag behind state-of-the-art (supervised) methods in OIE tasks.
  arXiv  Detail & Related papers  (2023-09-07T01:35:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.