An Angular-Temporal Interaction Network for Light Field Object Tracking in Low-Light Scenes

• URL: http://arxiv.org/abs/2507.21460v1
• Date: Tue, 29 Jul 2025 03:01:10 GMT
• Title: An Angular-Temporal Interaction Network for Light Field Object Tracking in Low-Light Scenes
• Authors: Mianzhao Wang, Fan Shi, Xu Cheng, Feifei Zhang, Shengyong Chen,
• Abstract summary: We propose a novel light field epipolar-plane structure image (ESI) representation that explicitly defines the geometric structure within the light field.<n>We also propose an angular-temporal interaction network (ATINet) for light field object tracking that learns angular-aware representations from the geometric structural cues and angular-temporal interaction cues of light fields.
• Score: 30.806699796022258
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: High-quality 4D light field representation with efficient angular feature modeling is crucial for scene perception, as it can provide discriminative spatial-angular cues to identify moving targets. However, recent developments still struggle to deliver reliable angular modeling in the temporal domain, particularly in complex low-light scenes. In this paper, we propose a novel light field epipolar-plane structure image (ESI) representation that explicitly defines the geometric structure within the light field. By capitalizing on the abrupt changes in the angles of light rays within the epipolar plane, this representation can enhance visual expression in low-light scenes and reduce redundancy in high-dimensional light fields. We further propose an angular-temporal interaction network (ATINet) for light field object tracking that learns angular-aware representations from the geometric structural cues and angular-temporal interaction cues of light fields. Furthermore, ATINet can also be optimized in a self-supervised manner to enhance the geometric feature interaction across the temporal domain. Finally, we introduce a large-scale light field low-light dataset for object tracking. Extensive experimentation demonstrates that ATINet achieves state-of-the-art performance in single object tracking. Furthermore, we extend the proposed method to multiple object tracking, which also shows the effectiveness of high-quality light field angular-temporal modeling.

Related papers

• PBIR-NIE: Glossy Object Capture under Non-Distant Lighting [30.325872237020395]
  Glossy objects present a significant challenge for 3D reconstruction from multi-view input images under natural lighting. We introduce PBIR-NIE, an inverse rendering framework designed to holistically capture the geometry, material attributes, and surrounding illumination of such objects.
  arXiv  Detail & Related papers  (2024-08-13T13:26:24Z)
• Gear-NeRF: Free-Viewpoint Rendering and Tracking with Motion-aware Spatio-Temporal Sampling [70.34875558830241]
  We present a way for learning a-temporal (4D) embedding, based on semantic semantic gears to allow for stratified modeling of dynamic regions of rendering the scene. At the same time, almost for free, our tracking approach enables free-viewpoint of interest - a functionality not yet achieved by existing NeRF-based methods.
  arXiv  Detail & Related papers  (2024-06-06T03:37:39Z)
• Neural Directional Encoding for Efficient and Accurate View-Dependent Appearance Modeling [47.86734601629109]
  NDE transfers the concept of feature-grid-based spatial encoding to the angular domain. Experiments on both synthetic and real datasets show that a NeRF model with NDE outperforms the state of the art on view synthesis of specular objects.
  arXiv  Detail & Related papers  (2024-05-23T17:56:34Z)
• NeLF-Pro: Neural Light Field Probes for Multi-Scale Novel View Synthesis [27.362216326282145]
  NeLF-Pro is a novel representation to model and reconstruct light fields in diverse natural scenes. Our central idea is to bake the scene's light field into spatially varying learnable representations.
  arXiv  Detail & Related papers  (2023-12-20T17:18:44Z)
• Unsupervised Discovery and Composition of Object Light Fields [57.198174741004095]
  We propose to represent objects in an object-centric, compositional scene representation as light fields. We propose a novel light field compositor module that enables reconstructing the global light field from a set of object-centric light fields.
  arXiv  Detail & Related papers  (2022-05-08T17:50:35Z)
• Multitask AET with Orthogonal Tangent Regularity for Dark Object Detection [84.52197307286681]
  We propose a novel multitask auto encoding transformation (MAET) model to enhance object detection in a dark environment. In a self-supervision manner, the MAET learns the intrinsic visual structure by encoding and decoding the realistic illumination-degrading transformation. We have achieved the state-of-the-art performance using synthetic and real-world datasets.
  arXiv  Detail & Related papers  (2022-05-06T16:27:14Z)
• Light Field Reconstruction Using Convolutional Network on EPI and Extended Applications [78.63280020581662]
  A novel convolutional neural network (CNN)-based framework is developed for light field reconstruction from a sparse set of views. We demonstrate the high performance and robustness of the proposed framework compared with state-of-the-art algorithms.
  arXiv  Detail & Related papers  (2021-03-24T08:16:32Z)
• Spatial-Angular Attention Network for Light Field Reconstruction [64.27343801968226]
  We propose a spatial-angular attention network to perceive correspondences in the light field non-locally. Motivated by the non-local attention mechanism, a spatial-angular attention module is introduced to compute the responses from all the positions in the epipolar plane for each pixel in the light field. We then propose a multi-scale reconstruction structure to efficiently implement the non-local attention in the low spatial scale.
  arXiv  Detail & Related papers  (2020-07-05T06:55:29Z)
• Learning Light Field Angular Super-Resolution via a Geometry-Aware Network [101.59693839475783]
  We propose an end-to-end learning-based approach aiming at angularly super-resolving a sparsely-sampled light field with a large baseline. Our method improves the PSNR of the second best method up to 2 dB in average, while saves the execution time 48$times$.
  arXiv  Detail & Related papers  (2020-02-26T02:36:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.