SKG-VLA: Scene Knowledge Graph Priors for Structured Scene Semantics and Multimodal Reasoning for Decision Making

• URL: http://arxiv.org/abs/2605.09343v1
• Date: Sun, 10 May 2026 05:43:58 GMT
• Title: SKG-VLA: Scene Knowledge Graph Priors for Structured Scene Semantics and Multimodal Reasoning for Decision Making
• Abstract summary: We present SKG-VLA for multimodal complaint decision making.<n>The core idea is to model each case as a structured complaint scene and represent its decision-relevant semantics with a emphScene Knowledge Graph (SKG)<n>We build a data synthesis pipeline that generates complaint scene descriptions, rule-consistent graph generalizations, question-answer supervision, and decision recommendations.<n>Experiments show that SKG-VLA consistently improves policy-grounded reasoning, complaint decision accuracy, long-tail generalization, and robustness under incomplete evidence.
• Score: 10.54549578058132
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Decision making in large-scale complaint handling systems increasingly relies on heterogeneous evidence, including complaint narratives, screenshots, order metadata, historical interactions, and platform policies. Existing complaint understanding systems mainly perform shallow classification or template matching over isolated modalities, while underutilizing explicit scene structure, rule knowledge, and cross-evidence dependencies. To address this limitation, we present SKG-VLA for multimodal complaint decision making. The core idea is to model each case as a structured complaint scene and represent its decision-relevant semantics with a \emph{Scene Knowledge Graph} (SKG), which organizes complaint entities, evidence items, policy clauses, temporal events, transactional states, and action-relevant relations into a unified graph. Based on SKG, we build a data synthesis pipeline that generates complaint scene descriptions, rule-consistent graph generalizations, question-answer supervision, and decision recommendations. We further construct a large-scale complaint scene dataset with both text-only and multimodal in-domain benchmarks. Finally, we adopt a three-stage training strategy -- domain-adaptive pre-training, task-oriented instruction fine-tuning, and end-to-end multimodal alignment -- to inject structured scene priors into a multimodal decision model. Experiments show that SKG-VLA consistently improves policy-grounded reasoning, complaint decision accuracy, long-tail generalization, and robustness under incomplete evidence.

Related papers

• CLAIM: Leading Open-domain Active Clarification of Large Language Models with Uncertainty Measurement [50.91045324738942]
  CLAIM is an uncertainty-driven framework for active clarification learning in open-domain settings.<n>We propose an entropy-driven synthetic data generation pipeline that integrates uncertainty estimation with semantic clustering and reasoning-based judgments.<n> Experimental results demonstrate that CLAIM can learn stable and generalizable clarification strategies without relying on manually labeled data.
  arXiv  Detail & Related papers  (2026-08-12T04:27:45Z)
• Graph-GRPO: Dependency-Aware Credit Assignment for Generative E-commerce Search Relevance [7.682863634260091]
  We present Graph-GRPO, a graph-structured extension of GRPO for relevance reasoning.<n>It propagates outcome-level rewards over the graph to derive step-level credit signals.<n>We also introduce a main-loss-driven controller that adaptively adjusts edge-wise credit-propagation coefficients.
  arXiv  Detail & Related papers  (2026-05-29T08:36:08Z)
• Citation-Closure Retrieval and Per-Rule Attribution for Real-World Regulatory Compliance Question Answering [54.97384993676565]
  We formalize Regulatory Compliance QA with RegOps-Bench, a novel benchmark featuring an Operational Knowledge Graph derived from complex national R&D regulations.<n>RefWalk traverses cross-document citations, fuses multi-view candidates via max-based aggregation, and enforces per-rule attribution to explicitly map claims to sources.
  arXiv  Detail & Related papers  (2026-05-28T10:38:38Z)
• FoodMonitor: Benchmarking MLLMs for Explainable Compliance Analysis [29.80090524705393]
  FoodMonitor is a benchmark for explainable compliance analysis in commercial kitchen surveillance.<n>The best-performing model achieves only 0.360 $C_textscore$, with spatial localization and fine-grained rule understanding emerging as the primary bottlenecks.
  arXiv  Detail & Related papers  (2026-05-23T10:19:41Z)
• Enhancing Multimodal In-Context Learning via Inductive-Deductive Reasoning [65.15766304205657]
  In-context learning (ICL) allows large models to adapt to tasks using a few examples, yet its extension to vision-language models (VLMs) remains fragile.<n>Our analysis reveals that the fundamental limitation lies in an inductive gap, models often produce correct answers from flawed reasoning.<n>We introduce a framework that restructures multimodal ICL as a principled inductive-deductive process.
  arXiv  Detail & Related papers  (2026-05-04T09:18:19Z)
• PassiveQA: A Three-Action Framework for Epistemically Calibrated Question Answering via Supervised Finetuning [0.0]
  We study decision-aware query resolution under incomplete information, where a model must determine whether to Answer, Ask for clarification, or Abstain.<n>We propose PassiveQA, a three-action framework that aligns model behaviour with information sufficiency through supervised finetuning.<n>Our approach integrates structured information-state representations, knowledge graph-grounded context, and a finetuned planner that explicitly models missing variables and decision reasoning.
  arXiv  Detail & Related papers  (2026-04-06T09:54:16Z)
• SocraticKG: Knowledge Graph Construction via QA-Driven Fact Extraction [4.867319754310031]
  We propose an automated KG construction method that introduces question-answer pairs as a structured intermediate representation.<n>SocraticKG captures contextual dependencies and implicit relational links typically lost in direct KG extraction pipelines.
  arXiv  Detail & Related papers  (2026-01-15T02:26:51Z)
• KBQA-R1: Reinforcing Large Language Models for Knowledge Base Question Answering [64.62317305868264]
  We present textbfKBQA-R1, a framework that shifts the paradigm from text imitation to interaction optimization via Reinforcement Learning.<n>Treating KBQA as a multi-turn decision process, our model learns to navigate the knowledge base using a list of actions.<n>Experiments on WebQSP, GrailQA, and GraphQuestions demonstrate that KBQA-R1 achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-12-10T17:45:42Z)
• Talk, Snap, Complain: Validation-Aware Multimodal Expert Framework for Fine-Grained Customer Grievances [14.30884038757821]
  Existing approaches to complaint analysis largely rely on unimodal, short-form content such as tweets or product reviews.<n>We introduce VALOR, a Validation-Aware Learner with Expert Routing, tailored for this multimodal setting.<n>We evaluate VALOR on a curated multimodal complaint dataset annotated with fine-grained aspect severity and labels.
  arXiv  Detail & Related papers  (2025-11-18T17:29:28Z)
• Structuring Reasoning for Complex Rules Beyond Flat Representations [37.11501169845084]
  We propose a novel framework inspired by expert human reasoning processes.<n>The Dynamic Adjudication template ( DAT) structures the inference mechanism into three methodical stages.<n> DAT consistently outperforms conventional Chain-of-Thought (CoT) approaches in complex rule-based tasks.
  arXiv  Detail & Related papers  (2025-10-01T04:10:13Z)
• From Chaos to Clarity: Claim Normalization to Empower Fact-Checking [57.024192702939736]
  Claim Normalization (aka ClaimNorm) aims to decompose complex and noisy social media posts into more straightforward and understandable forms. We propose CACN, a pioneering approach that leverages chain-of-thought and claim check-worthiness estimation. Our experiments demonstrate that CACN outperforms several baselines across various evaluation measures.
  arXiv  Detail & Related papers  (2023-10-22T16:07:06Z)
• Self-regulating Prompts: Foundational Model Adaptation without Forgetting [112.66832145320434]
  We introduce a self-regularization framework for prompting called PromptSRC. PromptSRC guides the prompts to optimize for both task-specific and task-agnostic general representations.
  arXiv  Detail & Related papers  (2023-07-13T17:59:35Z)
• Contextual Reasoning for Scene Generation (Technical Report) [16.54738722019977]
  We show how to apply our theoretical work to real autonomous-vehicle scene data. Goal of this work is to apply MR-CKR to the problem of generating challenging scenes for autonomous vehicle learning. We describe a framework for semantically guided data generation, based on a combination of MR-CKR and Algebraic Measures.
  arXiv  Detail & Related papers  (2023-05-03T16:39:55Z)
• Linear Partial Monitoring for Sequential Decision-Making: Algorithms, Regret Bounds and Applications [70.67112733968654]
  Partial monitoring is an expressive framework for sequential decision-making. We present a simple and unified analysis of partial monitoring, and further extend the model to the contextual and kernelized setting.
  arXiv  Detail & Related papers  (2023-02-07T18:58:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.