Lattice Boltzmann Model for Learning Real-World Pixel Dynamicity

• URL: http://arxiv.org/abs/2509.16527v2
• Date: Sat, 01 Nov 2025 01:04:57 GMT
• Title: Lattice Boltzmann Model for Learning Real-World Pixel Dynamicity
• Abstract summary: This work proposes the Lattice Boltzmann Model (LBM) to learn real-world pixel dynamicity for visual tracking.<n>LBM decomposes visual representations into dynamic pixel lattices and solves pixel motion states through collision-streaming processes.<n>Compared with existing methods, LBM demonstrates practical applicability in an online and real-time manner.
• Score: 21.760291945557828
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This work proposes the Lattice Boltzmann Model (LBM) to learn real-world pixel dynamicity for visual tracking. LBM decomposes visual representations into dynamic pixel lattices and solves pixel motion states through collision-streaming processes. Specifically, the high-dimensional distribution of the target pixels is acquired through a multilayer predict-update network to estimate the pixel positions and visibility. The predict stage formulates lattice collisions among the spatial neighborhood of target pixels and develops lattice streaming within the temporal visual context. The update stage rectifies the pixel distributions with online visual representations. Compared with existing methods, LBM demonstrates practical applicability in an online and real-time manner, which can efficiently adapt to real-world visual tracking tasks. Comprehensive evaluations of real-world point tracking benchmarks such as TAP-Vid and RoboTAP validate LBM's efficiency. A general evaluation of large-scale open-world object tracking benchmarks such as TAO, BFT, and OVT-B further demonstrates LBM's real-world practicality.

Related papers

• Point Tracking Improves World Action Models [29.784606797453662]
  pixel-level prediction entangles dynamics with nuisance factors such as lighting and texture, making learned representations vulnerable to task-irrelevant visual variation.<n>We propose JOPAT, a JOint Pixel-And-Track World-Action Model that predicts latent visual observations, 2D point tracks with visibility, and actions in a single denoising diffusion transformer.
  arXiv  Detail & Related papers  (2026-05-22T17:08:37Z)
• Featurising Pixels from Dynamic 3D Scenes with Linear In-Context Learners [52.436846702622915]
  We present a framework that learns pixel-accurate feature descriptors from videos.<n>The core of our training framework is linear in-context learning.<n>We demonstrate compelling empirical benefits of the learned representation across a diverse suite of vision tasks.
  arXiv  Detail & Related papers  (2026-04-29T09:51:56Z)
• Track4World: Feedforward World-centric Dense 3D Tracking of All Pixels [67.36972154532761]
  Estimating the 3D trajectory of every pixel from a monocular video is crucial and promising for a comprehensive understanding of the 3D dynamics of videos.<n>Recent monocular 3D tracking works demonstrate impressive performance, but are limited to either tracking sparse points on the first frame or a slow optimization-based framework for dense tracking.<n>We propose a feedforward model, called Track4World, enabling an efficient holistic 3D tracking of every pixel in the world-centric coordinate system.
  arXiv  Detail & Related papers  (2026-03-03T03:45:43Z)
• Pixel-Wise Multimodal Contrastive Learning for Remote Sensing Images [3.346542738111468]
  We propose a novel approach that leverages pixel-wise two-dimensional (2D) representations to encode visual property variations from satellite images.<n>We also introduce PIxel-wise Multimodal Contrastive (PIMC), a new multimodal self-supervision approach.<n>Our experimental results show that the use of 2D representations significantly enhances feature extraction from SITS.
  arXiv  Detail & Related papers  (2026-01-07T17:41:11Z)
• Learning Terrain Aware Bipedal Locomotion via Reduced Dimensional Perceptual Representations [6.407946291544721]
  This work introduces a hierarchical strategy for terrain-aware bipedal locomotion.<n>It integrates reduced-dimensional representations to enhance reinforcement learning (RL)-based high-level (HL) policies for real-time gait generation.
  arXiv  Detail & Related papers  (2025-12-15T05:38:08Z)
• Optimizing Grasping in Legged Robots: A Deep Learning Approach to Loco-Manipulation [0.6533458718563319]
  This paper presents a framework designed to enhance the grasping capabilities of quadrupeds equipped with arms.<n>We developed a pipeline within the Genesis simulation environment to generate a synthetic dataset of grasp attempts on common objects.<n>This dataset was used to train a custom CNN with a U-Net-like architecture that processes multi-modal input from an onboard RGB and depth cameras.<n>We validated the complete framework on a four-legged robot.
  arXiv  Detail & Related papers  (2025-08-24T17:47:56Z)
• Bayesian Active Learning for Semantic Segmentation [9.617769135242973]
  We introduce a Bayesian active learning framework based on sparse pixel-level annotation. BalEnt captures the information between the models' predicted marginalized probability distribution and the pixel labels. We train our proposed active learning framework for Cityscapes, Camvid, ADE20K and VOC2012 benchmark datasets.
  arXiv  Detail & Related papers  (2024-08-03T07:26:10Z)
• Deep Domain Adaptation: A Sim2Real Neural Approach for Improving Eye-Tracking Systems [80.62854148838359]
  Eye image segmentation is a critical step in eye tracking that has great influence over the final gaze estimate. We use dimensionality-reduction techniques to measure the overlap between the target eye images and synthetic training data. Our methods result in robust, improved performance when tackling the discrepancy between simulation and real-world data samples.
  arXiv  Detail & Related papers  (2024-03-23T22:32:06Z)
• LEAP-VO: Long-term Effective Any Point Tracking for Visual Odometry [52.131996528655094]
  We present the Long-term Effective Any Point Tracking (LEAP) module. LEAP innovatively combines visual, inter-track, and temporal cues with mindfully selected anchors for dynamic track estimation. Based on these traits, we develop LEAP-VO, a robust visual odometry system adept at handling occlusions and dynamic scenes.
  arXiv  Detail & Related papers  (2024-01-03T18:57:27Z)
• Pixel-Inconsistency Modeling for Image Manipulation Localization [59.968362815126326]
  Digital image forensics plays a crucial role in image authentication and manipulation localization. This paper presents a generalized and robust manipulation localization model through the analysis of pixel inconsistency artifacts. Experiments show that our method successfully extracts inherent pixel-inconsistency forgery fingerprints.
  arXiv  Detail & Related papers  (2023-09-30T02:54:51Z)
• MPI-Flow: Learning Realistic Optical Flow with Multiplane Images [18.310665144874775]
  We investigate generating realistic optical flow datasets from real-world images. To generate highly realistic new images, we construct a layered depth representation, known as multiplane images (MPI), from single-view images. To ensure the realism of motion, we present an independent object motion module that can separate the camera and dynamic object motion in MPI.
  arXiv  Detail & Related papers  (2023-09-13T04:31:00Z)
• Generalizing Event-Based Motion Deblurring in Real-World Scenarios [62.995994797897424]
  Event-based motion deblurring has shown promising results by exploiting low-latency events. We propose a scale-aware network that allows flexible input spatial scales and enables learning from different temporal scales of motion blur. A two-stage self-supervised learning scheme is then developed to fit real-world data distribution.
  arXiv  Detail & Related papers  (2023-08-11T04:27:29Z)
• HVC-Net: Unifying Homography, Visibility, and Confidence Learning for Planar Object Tracking [5.236567998857959]
  We present a unified convolutional neural network (CNN) model that jointly considers homography, visibility, and confidence. Our approach outperforms the state-of-the-art methods on public POT and TMT datasets.
  arXiv  Detail & Related papers  (2022-09-19T11:11:56Z)
• Towards Scale Consistent Monocular Visual Odometry by Learning from the Virtual World [83.36195426897768]
  We propose VRVO, a novel framework for retrieving the absolute scale from virtual data. We first train a scale-aware disparity network using both monocular real images and stereo virtual data. The resulting scale-consistent disparities are then integrated with a direct VO system.
  arXiv  Detail & Related papers  (2022-03-11T01:51:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.