Talk, Snap, Complain: Validation-Aware Multimodal Expert Framework for Fine-Grained Customer Grievances

• URL: http://arxiv.org/abs/2511.14693v1
• Date: Tue, 18 Nov 2025 17:29:28 GMT
• Title: Talk, Snap, Complain: Validation-Aware Multimodal Expert Framework for Fine-Grained Customer Grievances
• Abstract summary: Existing approaches to complaint analysis largely rely on unimodal, short-form content such as tweets or product reviews.<n>We introduce VALOR, a Validation-Aware Learner with Expert Routing, tailored for this multimodal setting.<n>We evaluate VALOR on a curated multimodal complaint dataset annotated with fine-grained aspect severity and labels.
• Score: 14.30884038757821
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Existing approaches to complaint analysis largely rely on unimodal, short-form content such as tweets or product reviews. This work advances the field by leveraging multimodal, multi-turn customer support dialogues, where users often share both textual complaints and visual evidence (e.g., screenshots, product photos) to enable fine-grained classification of complaint aspects and severity. We introduce VALOR, a Validation-Aware Learner with Expert Routing, tailored for this multimodal setting. It employs a multi-expert reasoning setup using large-scale generative models with Chain-of-Thought (CoT) prompting for nuanced decision-making. To ensure coherence between modalities, a semantic alignment score is computed and integrated into the final classification through a meta-fusion strategy. In alignment with the United Nations Sustainable Development Goals (UN SDGs), the proposed framework supports SDG 9 (Industry, Innovation and Infrastructure) by advancing AI-driven tools for robust, scalable, and context-aware service infrastructure. Further, by enabling structured analysis of complaint narratives and visual context, it contributes to SDG 12 (Responsible Consumption and Production) by promoting more responsive product design and improved accountability in consumer services. We evaluate VALOR on a curated multimodal complaint dataset annotated with fine-grained aspect and severity labels, showing that it consistently outperforms baseline models, especially in complex complaint scenarios where information is distributed across text and images. This study underscores the value of multimodal interaction and expert validation in practical complaint understanding systems. Resources related to data and codes are available here: https://github.com/sarmistha-D/VALOR

Related papers

• DocRetriever: A Plug-and-Play Framework for Multimodal Document Retrieval with Comprehensive Benchmark [48.84943754804533]
  Multimodal documents contain diverse elements, such as tables, figures, and layouts.<n>Current approaches typically combine dense visual embedding models with supervised rerankers to achieve high-precision retrieval.<n>We propose DocRetriever, a plug-and-play framework that enhances visual retrieval via a layout-aware sparse embedding technique.
  arXiv  Detail & Related papers  (2026-05-28T14:50:53Z)
• SKG-VLA: Scene Knowledge Graph Priors for Structured Scene Semantics and Multimodal Reasoning for Decision Making [10.54549578058132]
  We present SKG-VLA for multimodal complaint decision making.<n>The core idea is to model each case as a structured complaint scene and represent its decision-relevant semantics with a emphScene Knowledge Graph (SKG)<n>We build a data synthesis pipeline that generates complaint scene descriptions, rule-consistent graph generalizations, question-answer supervision, and decision recommendations.<n>Experiments show that SKG-VLA consistently improves policy-grounded reasoning, complaint decision accuracy, long-tail generalization, and robustness under incomplete evidence.
  arXiv  Detail & Related papers  (2026-05-10T05:43:58Z)
• Enhancing Multimodal In-Context Learning via Inductive-Deductive Reasoning [65.15766304205657]
  In-context learning (ICL) allows large models to adapt to tasks using a few examples, yet its extension to vision-language models (VLMs) remains fragile.<n>Our analysis reveals that the fundamental limitation lies in an inductive gap, models often produce correct answers from flawed reasoning.<n>We introduce a framework that restructures multimodal ICL as a principled inductive-deductive process.
  arXiv  Detail & Related papers  (2026-05-04T09:18:19Z)
• MURE: Hierarchical Multi-Resolution Encoding via Vision-Language Models for Visual Document Retrieval [111.99106496142474]
  Visual Document Retrieval (VDR) requires representations that capture both fine-grained visual details and global document structure.<n>Existing VDR models struggle to balance effectiveness and efficiency when processing high-resolution documents.<n>We propose a new X-VisEmb paradigm that progresses from multi-resolution sampling and encoding, through cross-granularity feature fusion, to adaptive representation distillation.
  arXiv  Detail & Related papers  (2026-03-07T15:17:22Z)
• MARA: A Multimodal Adaptive Retrieval-Augmented Framework for Document Question Answering [51.19392014547221]
  Retrieval-based multimodal document QA aims to identify and integrate relevant information from visually rich documents with complex multimodal structures.<n>Current approaches rely on query-agnostic document representations that overlook salient content.<n>We propose the Multimodal Adaptive Retrieval-Augmented (MARA) framework, which introduces query-adaptive generation.
  arXiv  Detail & Related papers  (2026-02-01T12:27:40Z)
• A Multi-Agent System for Generating Actionable Business Advice [6.322241356158281]
  Customer reviews contain rich signals about product weaknesses and unmet user needs.<n>Existing analytic methods rarely move beyond descriptive tasks.<n>We present a multi-agent, LLM-based framework for prescriptive decision support.
  arXiv  Detail & Related papers  (2026-01-17T12:07:55Z)
• Multimodal Detection of Fake Reviews using BERT and ResNet-50 [0.0]
  A robust multimodal fake review detection framework is proposed, integrating textual features encoded with BERT and visual features extracted using ResNet-50.<n> Experimental results indicate that the multimodal model outperforms unimodal baselines, achieving an F1-score of 0.934 on the test set.<n>This study demonstrates the critical role of multimodal learning in safeguarding digital trust and offers a scalable solution for content moderation.
  arXiv  Detail & Related papers  (2025-10-24T01:24:53Z)
• Scaling Beyond Context: A Survey of Multimodal Retrieval-Augmented Generation for Document Understanding [61.36285696607487]
  Document understanding is critical for applications from financial analysis to scientific discovery.<n>Current approaches, whether OCR-based pipelines feeding Large Language Models (LLMs) or native Multimodal LLMs (MLLMs) face key limitations.<n>Retrieval-Augmented Generation (RAG) helps ground models in external data, but documents' multimodal nature, combining text, tables, charts, and layout, demands a more advanced paradigm: Multimodal RAG.
  arXiv  Detail & Related papers  (2025-10-17T02:33:16Z)
• RAG-IGBench: Innovative Evaluation for RAG-based Interleaved Generation in Open-domain Question Answering [50.42577862494645]
  We present RAG-IGBench, a benchmark designed to evaluate the task of Interleaved Generation based on Retrieval-Augmented Generation (RAG-IG) in open-domain question answering.<n>RAG-IG integrates multimodal large language models (MLLMs) with retrieval mechanisms, enabling the models to access external image-text information for generating coherent multimodal content.
  arXiv  Detail & Related papers  (2025-10-11T03:06:39Z)
• OpinioRAG: Towards Generating User-Centric Opinion Highlights from Large-scale Online Reviews [12.338320566839483]
  We study the problem of opinion highlights generation from large volumes of user reviews.<n>Existing methods either fail to scale or produce generic, one-size-fits-all summaries that overlook personalized needs.<n>We introduce OpinioRAG, a scalable, training-free framework that combines RAG-based evidence retrieval with LLMs to efficiently produce tailored summaries.
  arXiv  Detail & Related papers  (2025-08-30T00:00:34Z)
• CAL-RAG: Retrieval-Augmented Multi-Agent Generation for Content-Aware Layout Design [6.830055289299306]
  CAL-RAG is a retrieval-augmented, agentic framework for content-aware layout generation.<n>We implement our framework using LangGraph and evaluate it on a benchmark rich in semantic variability.<n>Results demonstrate that combining retrieval augmentation with agentic multi-step reasoning yields a scalable, interpretable, and high-fidelity solution.
  arXiv  Detail & Related papers  (2025-06-27T06:09:56Z)
• DashCLIP: Leveraging multimodal models for generating semantic embeddings for DoorDash [0.12314765641075437]
  We introduce a joint training framework for product and user queries by aligning uni-modal and multi-modal encoders through contrastive learning on image-text data.<n>Our novel approach trains a query encoder with an LLM-curated relevance dataset, eliminating the reliance on engagement history.<n>For personalized ads recommendation, a significant uplift in the click-through rate and conversion rate after the deployment confirms the impact on key business metrics.
  arXiv  Detail & Related papers  (2025-03-18T20:38:31Z)
• VisDoM: Multi-Document QA with Visually Rich Elements Using Multimodal Retrieval-Augmented Generation [100.06122876025063]
  This paper introduces VisDoMBench, the first comprehensive benchmark designed to evaluate QA systems in multi-document settings.<n>We propose VisDoMRAG, a novel multimodal Retrieval Augmented Generation (RAG) approach that simultaneously utilizes visual and textual RAG.
  arXiv  Detail & Related papers  (2024-12-14T06:24:55Z)
• BabelBench: An Omni Benchmark for Code-Driven Analysis of Multimodal and Multistructured Data [61.936320820180875]
  Large language models (LLMs) have become increasingly pivotal across various domains. BabelBench is an innovative benchmark framework that evaluates the proficiency of LLMs in managing multimodal multistructured data with code execution. Our experimental findings on BabelBench indicate that even cutting-edge models like ChatGPT 4 exhibit substantial room for improvement.
  arXiv  Detail & Related papers  (2024-10-01T15:11:24Z)
• Learning Multi-Aspect Item Palette: A Semantic Tokenization Framework for Generative Recommendation [55.99632509895994]
  We introduce LAMIA, a novel approach for multi-aspect semantic tokenization.<n>Unlike RQ-VAE, which uses a single embedding, LAMIA learns an item palette''--a collection of independent and semantically parallel embeddings.<n>Our results demonstrate significant improvements in recommendation accuracy over existing methods.
  arXiv  Detail & Related papers  (2024-09-11T13:49:48Z)
• Exploiting Modality-Specific Features For Multi-Modal Manipulation Detection And Grounding [54.49214267905562]
  We construct a transformer-based framework for multi-modal manipulation detection and grounding tasks. Our framework simultaneously explores modality-specific features while preserving the capability for multi-modal alignment. We propose an implicit manipulation query (IMQ) that adaptively aggregates global contextual cues within each modality.
  arXiv  Detail & Related papers  (2023-09-22T06:55:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.