Engagement Process: Rethinking the Temporal Interface of Action and Observation

• URL: http://arxiv.org/abs/2605.11484v1
• Date: Tue, 12 May 2026 04:02:03 GMT
• Title: Engagement Process: Rethinking the Temporal Interface of Action and Observation
• Abstract summary: Engagement Process (EP) represents actions and observations as decoupled event streams along time, rather than updates paired at fixed decision steps.<n>EP captures single-agent timing issues such as deliberation, latency, delayed feedback, and persistent actions.<n>EP exposes temporal behaviors hidden by step-based interfaces and enables policies to adapt under explicit time costs.
• Score: 26.709060042723348
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Task completion in digital and physical environments increasingly involves complex temporal interaction, where actions and observations unfold over different time scales rather than align with fixed observation--action steps. To model such interactions, we propose \emph{Engagement Process} (EP), an interaction formalism that inherits the decision-theoretic structure of POMDPs while making time explicit in the action--observation interface. EP represents actions and observations as decoupled event streams along time, rather than updates paired at fixed decision steps. This interface captures single-agent timing issues such as deliberation latency, delayed feedback, and persistent actions, while supporting richer agent-side organization, multi-rate coordination, and compositional interaction among subsystems. Across toy, LLM-agent, and learning experiments, EP exposes temporal behaviors hidden by step-based interfaces and enables policies to adapt under explicit time costs.

Related papers

• Learning to Watch: Active Video Anomaly Understanding via Interleaved Policy Optimization [56.150184676745205]
  Video anomaly understanding (VAU) relies on sparse, context-dependent cues.<n>We propose $Anomtext-$, a closed-loop framework that reconceptualizes video understanding as an active sequential decision-making process.<n>Inspired by human video-reviewing behavior, this framework unifies internal cognitive reasoning and strategic evidence acquisition into an interleaved policy.
  arXiv  Detail & Related papers  (2026-07-01T08:41:49Z)
• Actionable Real-Time Modeling of Surgical Team Dynamics via Time-Expanded Interaction Graphs [9.813151093947097]
  We propose a real-time actionable approach for modeling surgical team dynamics using time-expanded interaction graphs.<n>This work advances surgical AI toward real-time, team-aware, and actionable decision support in the operating room.
  arXiv  Detail & Related papers  (2026-05-05T18:05:24Z)
• IMPACT-Scribe: Interactive Temporal Action Segmentation with Boundary Scribbles and Query Planning [75.72434806468884]
  IMPACT-Scribe is a correction-driven framework for dense labeling.<n>It combines uncertainty-aware boundary supervision, local proposal modeling, cost-aware query planning, structured propagation, and correction-driven adaptation.<n> Experiments and a human study show that this closed-loop design improves labeling quality per effort, enhances boundary accuracy, and fosters better humanmachine interaction over time.
  arXiv  Detail & Related papers  (2026-05-03T01:45:58Z)
• Enhancing Temporal Awareness in LLMs for Temporal Point Processes [53.596733432865626]
  Temporal point processes (TPPs) are crucial for analyzing events over time.<n> TPP-TAL is a novel plug-and-play framework designed to enhance temporal reasoning within large language models.<n> TPP-TAL delivers substantial improvements in temporal likelihood estimation and event prediction accuracy.
  arXiv  Detail & Related papers  (2025-12-29T03:01:24Z)
• Action-Dynamics Modeling and Cross-Temporal Interaction for Online Action Understanding [23.87664450145037]
  Action understanding, encompassing action detection and anticipation, plays a crucial role in numerous practical applications.<n>We propose a novel framework called the State-Specific Model (SSM), designed to unify and enhance both action detection and anticipation tasks.
  arXiv  Detail & Related papers  (2025-10-12T16:10:40Z)
• I-PHYRE: Interactive Physical Reasoning [25.02124862476204]
  I-PHYRE is a framework that challenges agents to simultaneously exhibit intuitive physical reasoning, multi-step planning, and in-situ intervention. We formulate four game splits to scrutinize agents' learning and generalization of essential principles of interactive physical reasoning.
  arXiv  Detail & Related papers  (2023-12-04T19:01:19Z)
• DIDER: Discovering Interpretable Dynamically Evolving Relations [14.69985920418015]
  This paper introduces DIDER, Discovering Interpretable Dynamically Evolving Relations, a generic end-to-end interaction modeling framework with intrinsic interpretability. We evaluate DIDER on both synthetic and real-world datasets.
  arXiv  Detail & Related papers  (2022-08-22T20:55:56Z)
• Interaction Transformer for Human Reaction Generation [61.22481606720487]
  We propose a novel interaction Transformer (InterFormer) consisting of a Transformer network with both temporal and spatial attentions. Our method is general and can be used to generate more complex and long-term interactions.
  arXiv  Detail & Related papers  (2022-07-04T19:30:41Z)
• Divide and Rule: Recurrent Partitioned Network for Dynamic Processes [25.855428321990328]
  Many dynamic processes are involved with interacting variables, from physical systems to sociological analysis. Our goal is to represent a system with a part-whole hierarchy and discover the implied dependencies among intra-system variables. The proposed architecture consists of (i) a perceptive module that extracts a hierarchical and temporally consistent representation of the observation at multiple levels, (ii) a deductive module for determining the relational connection between neurons at each level, and (iii) a statistical module that can predict the future by conditioning on the temporal distributional estimation.
  arXiv  Detail & Related papers  (2021-06-01T06:45:56Z)
• Modeling Multi-Label Action Dependencies for Temporal Action Localization [53.53490517832068]
  Real-world videos contain many complex actions with inherent relationships between action classes. We propose an attention-based architecture that models these action relationships for the task of temporal action localization in unoccurrence videos. We show improved performance over state-of-the-art methods on multi-label action localization benchmarks.
  arXiv  Detail & Related papers  (2021-03-04T13:37:28Z)
• A Spatial-Temporal Attentive Network with Spatial Continuity for Trajectory Prediction [74.00750936752418]
  We propose a novel model named spatial-temporal attentive network with spatial continuity (STAN-SC) First, spatial-temporal attention mechanism is presented to explore the most useful and important information. Second, we conduct a joint feature sequence based on the sequence and instant state information to make the generative trajectories keep spatial continuity.
  arXiv  Detail & Related papers  (2020-03-13T04:35:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.