Neuro-Symbolic Manipulation Understanding with Enriched Semantic Event Chains

• URL: http://arxiv.org/abs/2604.21053v1
• Date: Wed, 22 Apr 2026 19:53:48 GMT
• Title: Neuro-Symbolic Manipulation Understanding with Enriched Semantic Event Chains
• Abstract summary: We propose eSEC-LAM, a neuro-symbolic framework that transforms eSECs into an explicit event-level symbolic state for manipulation understanding.<n>We evaluate the proposed framework on EPIC-KITCHENS-100, EPIC-KITCHENS VISOR, and Assembly101 across action recognition, next-primitive prediction, robustness to perception noise, and explanation consistency.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Robotic systems operating in human environments must reason about how object interactions evolve over time, which actions are currently being performed, and what manipulation step is likely to follow. Classical enriched Semantic Event Chains (eSECs) provide an interpretable relational description of manipulation, but remain primarily descriptive and do not directly support uncertainty-aware decision making. In this paper, we propose eSEC-LAM, a neuro-symbolic framework that transforms eSECs into an explicit event-level symbolic state for manipulation understanding. The proposed formulation augments classical eSECs with confidence-aware predicates, functional object roles, affordance priors, primitive-level abstraction, and saliency-guided explanation cues. These enriched symbolic states are derived from a foundation-model-based perception front-end through deterministic predicate extraction, while current-action inference and next-primitive prediction are performed using lightweight symbolic reasoning over primitive pre- and post-conditions. We evaluate the proposed framework on EPIC-KITCHENS-100, EPIC-KITCHENS VISOR, and Assembly101 across action recognition, next-primitive prediction, robustness to perception noise, and explanation consistency. Experimental results show that eSEC-LAM achieves competitive action recognition, substantially improves next-primitive prediction, remains more robust under degraded perceptual conditions than both classical symbolic and end-to-end video baselines, and provides temporally consistent explanation traces grounded in explicit relational evidence. These findings demonstrate that enriched Semantic Event Chains can serve not only as interpretable descriptors of manipulation, but also as effective internal states for neuro-symbolic action reasoning.

Related papers

• SJEPA: Learning Elegant Latent Dynamics with Hybrid Symbolic-Neural Predictors [0.0]
  We introduce SJEPA, a reconstruction-free JEPA framework that learns predictive representations whose induced dynamics admit compact symbolic descriptions.<n>We formalise this principle through induced-dynamics complexity, analyse predictive-coordinate non-identifiability, and show that unconstrained operator compression creates a direct shortcut to representation collapse.<n>The results expose a controllable trade-off among predictive fidelity, representation quality, symbolic parsimony, and symbolic-neural allocation.
  arXiv  Detail & Related papers  (2026-08-04T13:08:07Z)
• PACE: A Neuro-Symbolic Framework for Plausible and Actionable Counterfactual Explanations [0.0]
  PACE is a modular neuro-symbolic framework for generating feasibility-aware counterfactual explanations.<n>By explicitly modeling feasible interventions, the framework produces explanations consistent with domain knowledge.<n>Results highlight the trade-off between counterfactual validity and plausibility.
  arXiv  Detail & Related papers  (2026-07-01T16:55:06Z)
• Provably Secure Agent Guardrail [89.79561918065122]
  Existing defense architectures rely on empirical semantic guardrails and probabilistic large model adjudicators.<n>This paper proposes a new security paradigm for agents based on the fundamental limitations of logical reasoning.
  arXiv  Detail & Related papers  (2026-05-28T02:12:41Z)
• \textsc{NaVIDA}: Vision-Language Navigation with Inverse Dynamics Augmentation [50.027425808733994]
  textscNaVIDA is a unified VLN framework that couples policy learning with action-grounded visual dynamics and adaptive execution.<n>textscNaVIDA augments training with chunk-based inverse-dynamics supervision to learn causal relationship between visual changes and corresponding actions.<n>Experiments show that textscNaVIDA achieves superior navigation performance compared to state-of-the-art methods with fewer parameters.
  arXiv  Detail & Related papers  (2026-01-26T06:16:17Z)
• Stable Language Guidance for Vision-Language-Action Models [62.80963701282789]
  Residual Semantic Steering is a probabilistic framework that disentangles physical affordance from semantic execution.<n> RSS achieves state-of-the-art robustness, maintaining performance even under adversarial linguistic perturbations.
  arXiv  Detail & Related papers  (2026-01-07T16:16:10Z)
• EASE: Embodied Active Event Perception via Self-Supervised Energy Minimization [6.249768559720122]
  Active event perception is essential for embodied intelligence in tasks such as human-AI collaboration, assistive robotics, and autonomous navigation.<n>We propose EASE, a self-supervised framework that unifies representation learning and embodied control through free energy.
  arXiv  Detail & Related papers  (2025-06-20T23:45:51Z)
• ECATS: Explainable-by-design concept-based anomaly detection for time series [0.5956301166481089]
  We propose ECATS, a concept-based neuro-symbolic architecture where concepts are represented as Signal Temporal Logic (STL) formulae. We show that our model is able to achieve great classification performance while ensuring local interpretability.
  arXiv  Detail & Related papers  (2024-05-17T08:12:53Z)
• Motion-Scenario Decoupling for Rat-Aware Video Position Prediction: Strategy and Benchmark [49.58762201363483]
  We introduce RatPose, a bio-robot motion prediction dataset constructed by considering the influence factors of individuals and environments. We propose a Dual-stream Motion-Scenario Decoupling framework that effectively separates scenario-oriented and motion-oriented features. We demonstrate significant performance improvements of the proposed textitDMSD framework on different difficulty-level tasks.
  arXiv  Detail & Related papers  (2023-05-17T14:14:31Z)
• Latent Event-Predictive Encodings through Counterfactual Regularization [0.9449650062296823]
  We introduce a SUrprise-GAted Recurrent neural network (SUGAR) using a novel form of counterfactual regularization. We test the model on a hierarchical sequence prediction task, where sequences are generated by alternating hidden graph structures.
  arXiv  Detail & Related papers  (2021-05-12T18:30:09Z)
• Abstract Spatial-Temporal Reasoning via Probabilistic Abduction and Execution [97.50813120600026]
  Spatial-temporal reasoning is a challenging task in Artificial Intelligence (AI) Recent works have focused on an abstract reasoning task of this kind -- Raven's Progressive Matrices ( RPM) We propose a neuro-symbolic Probabilistic Abduction and Execution learner (PrAE) learner.
  arXiv  Detail & Related papers  (2021-03-26T02:42:18Z)
• Analogous Process Structure Induction for Sub-event Sequence Prediction [111.10887596684276]
  We propose an Analogous Process Structure Induction APSI framework to predict the whole sub-event sequence of previously unseen processes. As our experiments and analysis indicate, APSI supports the generation of meaningful sub-event sequences for unseen processes and can help predict missing events.
  arXiv  Detail & Related papers  (2020-10-16T17:35:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.