Instantaneous Physiological Estimation using Video Transformers

• URL: http://arxiv.org/abs/2202.12368v1
• Date: Thu, 24 Feb 2022 21:25:09 GMT
• Title: Instantaneous Physiological Estimation using Video Transformers
• Abstract summary: Video-based physiological signal estimation has been limited primarily to predicting episodic scores in windowed intervals. We propose a video Transformer for estimating instantaneous heart rate and respiration rate from face videos.
• Score: 6.772249211312724
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Video-based physiological signal estimation has been limited primarily to predicting episodic scores in windowed intervals. While these intermittent values are useful, they provide an incomplete picture of patients' physiological status and may lead to late detection of critical conditions. We propose a video Transformer for estimating instantaneous heart rate and respiration rate from face videos. Physiological signals are typically confounded by alignment errors in space and time. To overcome this, we formulated the loss in the frequency domain. We evaluated the method on the large scale Vision-for-Vitals (V4V) benchmark. It outperformed both shallow and deep learning based methods for instantaneous respiration rate estimation. In the case of heart-rate estimation, it achieved an instantaneous-MAE of 13.0 beats-per-minute.

Related papers

• PhysVR: Vision-Language Model Guided Interference-aware Temporal Feature Refinement for Remote Physiological Measurement [30.385089857503207]
  We present PhysVR, a vision-language model guided interference-aware temporal feature refinement framework for r estimation.<n>In parallel, a frozen vision-language model processes sampled facial frames under an interference-oriented prompt, and an evidence head extracts visual interference evidence from the VLM output.<n>Experiments on five public benchmarks demonstrate that PhysVR consistently outperforms representative methods under both intra-dataset and cross-dataset evaluation protocols.
  arXiv  Detail & Related papers  (2026-08-30T08:56:46Z)
• How Good are Foundation Models in Longitudinal MRI Disease Progression Reasoning? [49.14050864978079]
  We introduce the Time-Aware Multi-View MRI Benchmark, an evaluation framework unifying multi-view anatomical input, temporal reasoning, and structured localization guidance.<n>The benchmark comprises 3,920 expert-verified question-answer pairs derived from 890 patients across over 3,200 longitudinal MRI timepoints.
  arXiv  Detail & Related papers  (2026-08-13T14:35:20Z)
• Proprio: Latent Self-Scoring and Inference-Time Refinement for Physically Plausible Video Generation [78.41956766048308]
  Proprio is a training-free framework that enables a frozen generator to assess and improve the physical plausibility of its own outputs.<n>Inspired by proprioception, the biological sense of one's own movement, Proprio treats the model's flow residual as a self-scoring signal.<n>Proprio consistently improves physical plausibility across text-to-video and image-to-video benchmarks.
  arXiv  Detail & Related papers  (2026-05-27T09:44:18Z)
• Towards Athlete Fatigue Assessment from Association Football Videos [53.20057980542362]
  This paper builds on state-of-theart Game State Reconstruction methods to extract player trajectories in pitch coordinates.<n>We then construct acceleration--speed profiles from these signals and analyze their behavior as fatigue-related performance indicators.<n>Our results indicate that monocular broadcast videos can recover kinematic patterns that are compatible with A-S-temporal while also revealing sensitivity to trajectory noise, calibration errors, and temporal discontinuities inherent to broadcast footage.
  arXiv  Detail & Related papers  (2026-04-07T09:36:28Z)
• Wav2Arrest 2.0: Long-Horizon Cardiac Arrest Prediction with Time-to-Event Modeling, Identity-Invariance, and Pseudo-Lab Alignment [5.706374608871095]
  High-frequency physiological waveform modality offers deep, real-time insights into patient status.<n>Recently, physiological foundation models have been shown to predict critical events, including Cardiac Arrest.<n>We offer three improvements to improve PPG-only CA systems by using minimal auxiliary information.
  arXiv  Detail & Related papers  (2025-09-25T23:46:39Z)
• Recovering Pulse Waves from Video Using Deep Unrolling and Deep Equilibrium Models [45.94962431110573]
  Camera-based monitoring of vital signs, also known as imaging photoplethysmography (i), has seen applications in driver-monitoring, affective computing, and more.<n>We introduce methods that combine signal processing and deep learning methods in an inverse problem.
  arXiv  Detail & Related papers  (2025-03-21T16:11:21Z)
• Leveraging Cardiovascular Simulations for In-Vivo Prediction of Cardiac Biomarkers [43.17768785084301]
  We train an amortized neural posterior estimator on a newly built large dataset of cardiac simulations.<n>We incorporate elements modeling effects to better align simulated data with real-world measurements.<n>The proposed framework can further integrate in-vivo data sources to refine its predictive capabilities on real-world data.
  arXiv  Detail & Related papers  (2024-12-23T13:05:17Z)
• Chaos in Motion: Unveiling Robustness in Remote Heart Rate Measurement through Brain-Inspired Skin Tracking [7.688280190165613]
  Existing remote heart rate measurement methods have three serious problems. We apply chaos theory to computer vision tasks for the first time, thus a brain-inspired framework. Our method achieves Robust Skin Tracking for Heart Rate measurement, called HR-RST.
  arXiv  Detail & Related papers  (2024-04-11T12:26:10Z)
• VSViG: Real-time Video-based Seizure Detection via Skeleton-based Spatiotemporal ViG [8.100646331930953]
  An accurate and efficient epileptic seizure onset detection can significantly benefit patients. Traditional diagnostic methods, primarily relying on electroencephalograms (EEGs), often result in cumbersome and non-portable solutions. We propose a novel Video-based Seizure detection model via a skeleton-basedtemporal Vision Graph neural network.
  arXiv  Detail & Related papers  (2023-11-24T15:07:29Z)
• Semantic-aware Temporal Channel-wise Attention for Cardiac Function Assessment [69.02116920364311]
  Existing video-based methods do not pay much attention to the left ventricular region, nor the left ventricular changes caused by motion. We propose a semi-supervised auxiliary learning paradigm with a left ventricular segmentation task, which contributes to the representation learning for the left ventricular region. Our approach achieves state-of-the-art performance on the Stanford dataset with an improvement of 0.22 MAE, 0.26 RMSE, and 1.9% $R2$.
  arXiv  Detail & Related papers  (2023-10-09T05:57:01Z)
• CenTime: Event-Conditional Modelling of Censoring in Survival Analysis [49.44664144472712]
  We introduce CenTime, a novel approach to survival analysis that directly estimates the time to event. Our method features an innovative event-conditional censoring mechanism that performs robustly even when uncensored data is scarce. Our results indicate that CenTime offers state-of-the-art performance in predicting time-to-death while maintaining comparable ranking performance.
  arXiv  Detail & Related papers  (2023-09-07T17:07:33Z)
• Time-distance vision transformers in lung cancer diagnosis from longitudinal computed tomography [4.924544172166966]
  longitudinal medical imaging presents the unique challenge of sparse, irregular time intervals in data acquisition. We propose two interpretations of a time-distance vision transformer (ViT) by using vector embeddings of continuous time and a temporal emphasis model to scale self-attention weights. This work represents the first self-attention-based framework for classifying longitudinal medical images.
  arXiv  Detail & Related papers  (2022-09-04T19:08:44Z)
• A Deep Learning Approach to Predicting Collateral Flow in Stroke Patients Using Radiomic Features from Perfusion Images [58.17507437526425]
  Collateral circulation results from specialized anastomotic channels which provide oxygenated blood to regions with compromised blood flow. The actual grading is mostly done through manual inspection of the acquired images. We present a deep learning approach to predicting collateral flow grading in stroke patients based on radiomic features extracted from MR perfusion data.
  arXiv  Detail & Related papers  (2021-10-24T18:58:40Z)
• Learning Higher-Order Dynamics in Video-Based Cardiac Measurement [5.571184025017747]
  In the cardiac pulse, the second derivative can be used as an indicator of blood pressure and arterial disease. Recent developments in camera-based vital sign measurement have shown that cardiac measurements can be recovered with impressive accuracy from videos. We provide evidence that higher-order dynamics are better estimated by neural models when explicitly optimized for in the loss function.
  arXiv  Detail & Related papers  (2021-10-07T16:29:55Z)
• Perceptual Learned Video Compression with Recurrent Conditional GAN [158.0726042755]
  We propose a Perceptual Learned Video Compression (PLVC) approach with recurrent conditional generative adversarial network. PLVC learns to compress video towards good perceptual quality at low bit-rate. The user study further validates the outstanding perceptual performance of PLVC in comparison with the latest learned video compression approaches.
  arXiv  Detail & Related papers  (2021-09-07T13:36:57Z)
• Real-time Webcam Heart-Rate and Variability Estimation with Clean Ground Truth for Evaluation [9.883261192383612]
  Remote photo-plethysmography (r) uses a camera to estimate a person's heart rate (HR) HRV is a measure of the fine fluctuations in the intervals between heart beats. We introduce a refined and efficient real-time r pipeline with novel filtering and motion suppression.
  arXiv  Detail & Related papers  (2020-12-31T18:57:05Z)
• Intrinsic Temporal Regularization for High-resolution Human Video Synthesis [59.54483950973432]
  temporal consistency is crucial for extending image processing pipelines to the video domain. We propose an effective intrinsic temporal regularization scheme, where an intrinsic confidence map is estimated via the frame generator to regulate motion estimation. We apply our intrinsic temporal regulation to single-image generator, leading to a powerful " INTERnet" capable of generating $512times512$ resolution human action videos.
  arXiv  Detail & Related papers  (2020-12-11T05:29:45Z)
• Fast Motion Understanding with Spatiotemporal Neural Networks and Dynamic Vision Sensors [99.94079901071163]
  This paper presents a Dynamic Vision Sensor (DVS) based system for reasoning about high speed motion. We consider the case of a robot at rest reacting to a small, fast approaching object at speeds higher than 15m/s. We highlight the results of our system to a toy dart moving at 23.4m/s with a 24.73deg error in $theta$, 18.4mm average discretized radius prediction error, and 25.03% median time to collision prediction error.
  arXiv  Detail & Related papers  (2020-11-18T17:55:07Z)
• Identification of deep breath while moving forward based on multiple body regions and graph signal analysis [45.62293065676075]
  This paper presents an unobtrusive solution that can automatically identify deep breath when a person is walking past the global depth camera. In validation experiments, the proposed approach outperforms the comparative methods with the accuracy, precision, recall and F1 of 75.5%, 76.2%, 75.0% and 75.2%, respectively.
  arXiv  Detail & Related papers  (2020-10-20T08:26:50Z)
• Video-based Remote Physiological Measurement via Cross-verified Feature Disentangling [121.50704279659253]
  We propose a cross-verified feature disentangling strategy to disentangle the physiological features with non-physiological representations. We then use the distilled physiological features for robust multi-task physiological measurements. The disentangled features are finally used for the joint prediction of multiple physiological signals like average HR values and r signals.
  arXiv  Detail & Related papers  (2020-07-16T09:39:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.