An Open Software Suite for Event-Based Video
- URL: http://arxiv.org/abs/2401.17151v1
- Date: Tue, 30 Jan 2024 16:32:37 GMT
- Title: An Open Software Suite for Event-Based Video
- Authors: Andrew C. Freeman
- Abstract summary: Event-based video is a new paradigm that forgoes image frames altogether.
Until now, researchers have lacked a cohesive software framework for exploring the representation, compression, and applications of event-based video.
I present the AD$Delta$ER software suite to fill this gap.
- Score: 0.8158530638728501
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: While traditional video representations are organized around discrete image
frames, event-based video is a new paradigm that forgoes image frames
altogether. Rather, pixel samples are temporally asynchronous and independent
of one another. Until now, researchers have lacked a cohesive software
framework for exploring the representation, compression, and applications of
event-based video. I present the AD$\Delta$ER software suite to fill this gap.
This framework includes utilities for transcoding framed and multimodal
event-based video sources to a common representation, rate control mechanisms,
lossy compression, application support, and an interactive GUI for transcoding
and playback. In this paper, I describe these various software components and
their usage.
Related papers
- Rethinking Video with a Universal Event-Based Representation [0.0]
I introduce Address, Decimation, DeltaER, a novel intermediate video representation and system framework.
I demonstrate that ADDeltaER achieves state-of-the-art application speed and compression performance for scenes with high temporal redundancy.
I discuss the implications for event-based video on large-scale video surveillance and resource-constrained sensing.
arXiv Detail & Related papers (2024-08-12T16:00:17Z) - VidToMe: Video Token Merging for Zero-Shot Video Editing [100.79999871424931]
We propose a novel approach to enhance temporal consistency in generated videos by merging self-attention tokens across frames.
Our method improves temporal coherence and reduces memory consumption in self-attention computations.
arXiv Detail & Related papers (2023-12-17T09:05:56Z) - Accelerated Event-Based Feature Detection and Compression for
Surveillance Video Systems [1.5390526524075634]
We propose a novel system which conveys temporal redundancy within a sparse decompressed representation.
We leverage a video representation framework called ADDER to transcode framed videos to sparse, asynchronous intensity samples.
Our work paves the way for upcoming neuromorphic sensors and is amenable to future applications with spiking neural networks.
arXiv Detail & Related papers (2023-12-13T15:30:29Z) - Reuse and Diffuse: Iterative Denoising for Text-to-Video Generation [92.55296042611886]
We propose a framework called "Reuse and Diffuse" dubbed $textitVidRD$ to produce more frames following the frames already generated by an LDM.
We also propose a set of strategies for composing video-text data that involve diverse content from multiple existing datasets.
arXiv Detail & Related papers (2023-09-07T08:12:58Z) - Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translation [93.18163456287164]
This paper proposes a novel text-guided video-to-video translation framework to adapt image models to videos.
Our framework achieves global style and local texture temporal consistency at a low cost.
arXiv Detail & Related papers (2023-06-13T17:52:23Z) - VideoComposer: Compositional Video Synthesis with Motion Controllability [52.4714732331632]
VideoComposer allows users to flexibly compose a video with textual conditions, spatial conditions, and more importantly temporal conditions.
We introduce the motion vector from compressed videos as an explicit control signal to provide guidance regarding temporal dynamics.
In addition, we develop a Spatio-Temporal Condition encoder (STC-encoder) that serves as a unified interface to effectively incorporate the spatial and temporal relations of sequential inputs.
arXiv Detail & Related papers (2023-06-03T06:29:02Z) - An Asynchronous Intensity Representation for Framed and Event Video
Sources [2.9097303137825046]
We introduce an intensity representation for both framed and non-framed data sources.
We show that our representation can increase intensity precision and greatly reduce the number of samples per pixel.
We argue that our method provides the computational efficiency and temporal granularity necessary to build real-time intensity-based applications for event cameras.
arXiv Detail & Related papers (2023-01-20T19:46:23Z) - FFNeRV: Flow-Guided Frame-Wise Neural Representations for Videos [5.958701846880935]
We propose FFNeRV, a novel method for incorporating flow information into frame-wise representations to exploit the temporal redundancy across the frames in videos.
With model compression techniques, FFNeRV outperforms widely-used standard video codecs (H.264 and HEVC) and performs on par with state-of-the-art video compression algorithms.
arXiv Detail & Related papers (2022-12-23T12:51:42Z) - Task Agnostic Restoration of Natural Video Dynamics [10.078712109708592]
In many video restoration/translation tasks, image processing operations are na"ively extended to the video domain by processing each frame independently.
We propose a general framework for this task that learns to infer and utilize consistent motion dynamics from inconsistent videos to mitigate the temporal flicker.
The proposed framework produces SOTA results on two benchmark datasets, DAVIS and videvo.net, processed by numerous image processing applications.
arXiv Detail & Related papers (2022-06-08T09:00:31Z) - Video Imprint [107.1365846180187]
A new unified video analytics framework (ER3) is proposed for complex event retrieval, recognition and recounting.
The proposed video imprint representation exploits temporal correlations among image features across video frames.
The video imprint is fed into a reasoning network and a feature aggregation module, for event recognition/recounting and event retrieval tasks, respectively.
arXiv Detail & Related papers (2021-06-07T00:32:47Z) - Conditional Entropy Coding for Efficient Video Compression [82.35389813794372]
We propose a very simple and efficient video compression framework that only focuses on modeling the conditional entropy between frames.
We first show that a simple architecture modeling the entropy between the image latent codes is as competitive as other neural video compression works and video codecs.
We then propose a novel internal learning extension on top of this architecture that brings an additional 10% savings without trading off decoding speed.
arXiv Detail & Related papers (2020-08-20T20:01:59Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.