NOVIS: A Case for End-to-End Near-Online Video Instance Segmentation

• URL: http://arxiv.org/abs/2308.15266v2
• Date: Mon, 18 Sep 2023 14:46:11 GMT
• Title: NOVIS: A Case for End-to-End Near-Online Video Instance Segmentation
• Authors: Tim Meinhardt and Matt Feiszli and Yuchen Fan and Laura Leal-Taixe and Rakesh Ranjan
• Abstract summary: Video Instance (VIS) community operated under the common belief that offline methods are generally superior to a frame by frame online processing. We present a detailed analysis on different processing paradigms and a new end-to-end Video Instance method. Our NOVIS represents the first near-online VIS approach which avoids any handcrafted trackings.
• Score: 22.200700685751826
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Until recently, the Video Instance Segmentation (VIS) community operated under the common belief that offline methods are generally superior to a frame by frame online processing. However, the recent success of online methods questions this belief, in particular, for challenging and long video sequences. We understand this work as a rebuttal of those recent observations and an appeal to the community to focus on dedicated near-online VIS approaches. To support our argument, we present a detailed analysis on different processing paradigms and the new end-to-end trainable NOVIS (Near-Online Video Instance Segmentation) method. Our transformer-based model directly predicts spatio-temporal mask volumes for clips of frames and performs instance tracking between clips via overlap embeddings. NOVIS represents the first near-online VIS approach which avoids any handcrafted tracking heuristics. We outperform all existing VIS methods by large margins and provide new state-of-the-art results on both YouTube-VIS (2019/2021) and the OVIS benchmarks.

Related papers

Err

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.