Is there progress in activity progress prediction?

• URL: http://arxiv.org/abs/2308.05533v1
• Date: Thu, 10 Aug 2023 12:23:47 GMT
• Title: Is there progress in activity progress prediction?
• Abstract summary: We find that current progress prediction methods seem not to extract useful visual information for the progress prediction task. We design a precisely controlled dataset for activity progress prediction and on this synthetic dataset we show that the considered methods can make use of the visual information. We conclude that the progress prediction task is ill-posed on the currently used real-world datasets.
• Score: 19.841311026235875
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Activity progress prediction aims to estimate what percentage of an activity has been completed. Currently this is done with machine learning approaches, trained and evaluated on complicated and realistic video datasets. The videos in these datasets vary drastically in length and appearance. And some of the activities have unanticipated developments, making activity progression difficult to estimate. In this work, we examine the results obtained by existing progress prediction methods on these datasets. We find that current progress prediction methods seem not to extract useful visual information for the progress prediction task. Therefore, these methods fail to exceed simple frame-counting baselines. We design a precisely controlled dataset for activity progress prediction and on this synthetic dataset we show that the considered methods can make use of the visual information, when this directly relates to the progress prediction. We conclude that the progress prediction task is ill-posed on the currently used real-world datasets. Moreover, to fairly measure activity progression we advise to consider a, simple but effective, frame-counting baseline.

Related papers

• Progress Reward Modeling for Robotic Learning: A Comprehensive Survey [18.0494714278744]
  A terminal success signal only tells the robot whether the task is completed.<n>Recent studies have increasingly explored progress rewards that provide feedback during task execution.<n>Existing methods use different observations, goal specifications, output signals, supervision sources, and evaluation protocols.<n>This makes it difficult to compare them and understand what their results actually validate.
  arXiv  Detail & Related papers  (2026-07-22T19:29:52Z)
• ProgressVLA: Progress-Guided Diffusion Policy for Vision-Language Robotic Manipulation [57.07494675832939]
  Most existing vision-language-action (VLA) models for robotic manipulation lack progress awareness.<n>In this work, we investigate the estimation and integration of task progress, proposing a novel model named textbf vla<n>Experiments on the CALVIN and LIBERO benchmarks, alongside real-world robot deployment, consistently demonstrate substantial improvements in success rates.
  arXiv  Detail & Related papers  (2026-03-29T12:38:11Z)
• Future Success Prediction in Open-Vocabulary Object Manipulation Tasks Based on End-Effector Trajectories [1.0742675209112622]
  This study addresses a task designed to predict the future success or failure of openvocabulary object manipulation. In this task, the model is required to make predictions based on natural language instructions, egocentric view images before manipulation, and the given end-effector trajectories. We propose a novel approach that enables the prediction of success or failure by aligning the given trajectories and images with natural language instructions.
  arXiv  Detail & Related papers  (2024-12-26T08:11:41Z)
• Human Action Anticipation: A Survey [86.415721659234]
  The literature on behavior prediction spans various tasks, including action anticipation, activity forecasting, intent prediction, goal prediction, and so on. Our survey aims to tie together this fragmented literature, covering recent technical innovations as well as the development of new large-scale datasets for model training and evaluation.
  arXiv  Detail & Related papers  (2024-10-17T21:37:40Z)
• A Supervised Contrastive Learning Pretrain-Finetune Approach for Time Series [15.218841180577135]
  We introduce a novel pretraining procedure that leverages supervised contrastive learning to distinguish features within each pretraining dataset. We then propose a fine-tuning procedure designed to enhance the accurate prediction of the target data by aligning it more closely with the learned dynamics of the pretraining datasets.
  arXiv  Detail & Related papers  (2023-11-21T02:06:52Z)
• Temporal DINO: A Self-supervised Video Strategy to Enhance Action Prediction [15.696593695918844]
  This paper introduces a novel self-supervised video strategy for enhancing action prediction inspired by DINO (self-distillation with no labels) The experimental results showcase significant improvements in prediction performance across 3D-ResNet, Transformer, and LSTM architectures. These findings highlight the potential of our approach in diverse video-based tasks such as activity recognition, motion planning, and scene understanding.
  arXiv  Detail & Related papers  (2023-08-08T21:18:23Z)
• ALP: Action-Aware Embodied Learning for Perception [60.64801970249279]
  We introduce Action-Aware Embodied Learning for Perception (ALP) ALP incorporates action information into representation learning through a combination of optimizing a reinforcement learning policy and an inverse dynamics prediction objective. We show that ALP outperforms existing baselines in several downstream perception tasks.
  arXiv  Detail & Related papers  (2023-06-16T21:51:04Z)
• Reinforcement Learning from Passive Data via Latent Intentions [86.4969514480008]
  We show that passive data can still be used to learn features that accelerate downstream RL. Our approach learns from passive data by modeling intentions. Our experiments demonstrate the ability to learn from many forms of passive data, including cross-embodiment video data and YouTube videos.
  arXiv  Detail & Related papers  (2023-04-10T17:59:05Z)
• Uncertainty estimation for Cross-dataset performance in Trajectory prediction [1.9399281609371257]
  We study the performance of a state-of-the-art trajectory prediction method across four different datasets. We highlight which datasets work best on others, and study how uncertainty estimation allows for a better transferable performance.
  arXiv  Detail & Related papers  (2022-05-15T15:28:02Z)
• Injecting Knowledge in Data-driven Vehicle Trajectory Predictors [82.91398970736391]
  Vehicle trajectory prediction tasks have been commonly tackled from two perspectives: knowledge-driven or data-driven. In this paper, we propose to learn a "Realistic Residual Block" (RRB) which effectively connects these two perspectives. Our proposed method outputs realistic predictions by confining the residual range and taking into account its uncertainty.
  arXiv  Detail & Related papers  (2021-03-08T16:03:09Z)
• Action Sequence Predictions of Vehicles in Urban Environments using Map and Social Context [152.0714518512966]
  This work studies the problem of predicting the sequence of future actions for surround vehicles in real-world driving scenarios. The first contribution is an automatic method to convert the trajectories recorded in real-world driving scenarios to action sequences with the help of HD maps. The second contribution lies in applying the method to the well-known traffic agent tracking and prediction dataset Argoverse, resulting in 228,000 action sequences. The third contribution is to propose a novel action sequence prediction method by integrating past positions and velocities of the traffic agents, map information and social context into a single end-to-end trainable neural network
  arXiv  Detail & Related papers  (2020-04-29T14:59:58Z)
• TTPP: Temporal Transformer with Progressive Prediction for Efficient Action Anticipation [46.28067541184604]
  Video action anticipation aims to predict future action categories from observed frames. Current state-of-the-art approaches mainly resort to recurrent neural networks to encode history information into hidden states. This paper proposes a simple yet efficient Temporal Transformer with Progressive Prediction framework.
  arXiv  Detail & Related papers  (2020-03-07T07:59:42Z)
• Spatiotemporal Relationship Reasoning for Pedestrian Intent Prediction [57.56466850377598]
  Reasoning over visual data is a desirable capability for robotics and vision-based applications. In this paper, we present a framework on graph to uncover relationships in different objects in the scene for reasoning about pedestrian intent. Pedestrian intent, defined as the future action of crossing or not-crossing the street, is a very crucial piece of information for autonomous vehicles.
  arXiv  Detail & Related papers  (2020-02-20T18:50:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.