Context-Aware Streaming Perception in Dynamic Environments
- URL: http://arxiv.org/abs/2208.07479v1
- Date: Tue, 16 Aug 2022 00:33:04 GMT
- Title: Context-Aware Streaming Perception in Dynamic Environments
- Authors: Gur-Eyal Sela, Ionel Gog, Justin Wong, Kumar Krishna Agrawal, Xiangxi
Mo, Sukrit Kalra, Peter Schafhalter, Eric Leong, Xin Wang, Bharathan Balaji,
Joseph Gonzalez, Ion Stoica
- Abstract summary: Real-time vision applications like autonomous driving operate in streaming settings, where ground truth changes between inference start and finish.
We propose to maximize streaming accuracy for every environment context.
Our method improves tracking performance (S-MOTA) by 7.4% over the conventional static approach.
- Score: 25.029862642968457
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Efficient vision works maximize accuracy under a latency budget. These works
evaluate accuracy offline, one image at a time. However, real-time vision
applications like autonomous driving operate in streaming settings, where
ground truth changes between inference start and finish. This results in a
significant accuracy drop. Therefore, a recent work proposed to maximize
accuracy in streaming settings on average. In this paper, we propose to
maximize streaming accuracy for every environment context. We posit that
scenario difficulty influences the initial (offline) accuracy difference, while
obstacle displacement in the scene affects the subsequent accuracy degradation.
Our method, Octopus, uses these scenario properties to select configurations
that maximize streaming accuracy at test time. Our method improves tracking
performance (S-MOTA) by 7.4% over the conventional static approach. Further,
performance improvement using our method comes in addition to, and not instead
of, advances in offline accuracy.
Related papers
- Understanding Optimization in Deep Learning with Central Flows [53.66160508990508]
We show that an RMS's implicit behavior can be explicitly captured by a "central flow:" a differential equation.
We show that these flows can empirically predict long-term optimization trajectories of generic neural networks.
arXiv Detail & Related papers (2024-10-31T17:58:13Z) - Real-time Stereo-based 3D Object Detection for Streaming Perception [12.52037626475608]
We introduce StreamDSGN, the first real-time stereo-based 3D object detection framework designed for streaming perception.
StreamDSGN directly predicts the 3D properties of objects in the next moment by leveraging historical information.
Compared with the strong baseline, StreamDSGN significantly improves the streaming average precision by up to 4.33%.
arXiv Detail & Related papers (2024-10-16T09:23:02Z) - DaDe: Delay-adaptive Detector for Streaming Perception [0.0]
In real-time environment, surrounding environment changes when processing is over.
Streaming perception is proposed to assess the latency and accuracy of real-time video perception.
We develop a model that can reflect processing delays in real time and produce the most reasonable results.
arXiv Detail & Related papers (2022-12-22T09:25:46Z) - StreamYOLO: Real-time Object Detection for Streaming Perception [84.2559631820007]
We endow the models with the capacity of predicting the future, significantly improving the results for streaming perception.
We consider multiple velocities driving scene and propose Velocity-awared streaming AP (VsAP) to jointly evaluate the accuracy.
Our simple method achieves the state-of-the-art performance on Argoverse-HD dataset and improves the sAP and VsAP by 4.7% and 8.2% respectively.
arXiv Detail & Related papers (2022-07-21T12:03:02Z) - Real-time Object Detection for Streaming Perception [84.2559631820007]
Streaming perception is proposed to jointly evaluate the latency and accuracy into a single metric for video online perception.
We build a simple and effective framework for streaming perception.
Our method achieves competitive performance on Argoverse-HD dataset and improves the AP by 4.9% compared to the strong baseline.
arXiv Detail & Related papers (2022-03-23T11:33:27Z) - AuxAdapt: Stable and Efficient Test-Time Adaptation for Temporally
Consistent Video Semantic Segmentation [81.87943324048756]
In video segmentation, generating temporally consistent results across frames is as important as achieving frame-wise accuracy.
Existing methods rely on optical flow regularization or fine-tuning with test data to attain temporal consistency.
This paper presents an efficient, intuitive, and unsupervised online adaptation method, AuxAdapt, for improving the temporal consistency of most neural network models.
arXiv Detail & Related papers (2021-10-24T07:07:41Z) - Towards Streaming Perception [70.68520310095155]
We present an approach that coherently integrates latency and accuracy into a single metric for real-time online perception.
The key insight behind this metric is to jointly evaluate the output of the entire perception stack at every time instant.
We focus on the illustrative tasks of object detection and instance segmentation in urban video streams, and contribute a novel dataset with high-quality and temporally-dense annotations.
arXiv Detail & Related papers (2020-05-21T01:51:35Z) - Tracking Performance of Online Stochastic Learners [57.14673504239551]
Online algorithms are popular in large-scale learning settings due to their ability to compute updates on the fly, without the need to store and process data in large batches.
When a constant step-size is used, these algorithms also have the ability to adapt to drifts in problem parameters, such as data or model properties, and track the optimal solution with reasonable accuracy.
We establish a link between steady-state performance derived under stationarity assumptions and the tracking performance of online learners under random walk models.
arXiv Detail & Related papers (2020-04-04T14:16:27Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.