Primate vision reveals a missing principle for robust dynamic AI

• URL: http://arxiv.org/abs/2608.23790v1
• Date: Mon, 24 Aug 2026 19:38:54 GMT
• Title: Primate vision reveals a missing principle for robust dynamic AI
• Abstract summary: We compare human perception and neural activity in macaque inferior temporal cortex with representations from image- and video-based neural networks.<n>We find progressive integration of motion into object representations as a principle of robust dynamic vision.
• Score: 9.82700229401299
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: How does an intelligent visual system combine what objects look like with how they move while remaining robust as appearance changes? We addressed this question by comparing human perception and neural activity in macaque inferior temporal cortex with representations from image- and video-based neural networks spanning recognition, segmentation, optic-flow processing and predictive world modeling. Temporal integration improved object representations, but most video recognition models generalized poorly when appearance was disrupted while motion structure was preserved. Humans and macaque IT remained robust. Notably, predictive world models combined strong cross-appearance generalization with the closest correspondence to IT, outperforming other video-modeling approaches in neural fidelity. Yet no model reproduced the cortical transformation from early appearance-dominated responses toward later appearance-invariant motion coding. These results identify progressive integration of motion into object representations as a principle of robust dynamic vision and implicate predictive learning as a promising route toward realizing this computation in artificial systems.

Related papers

• Self-Supervised Learning of Structured Dynamics from Videos [76.23834498074805]
  We study whether structured motion representations can be recovered from frozen features of a pretrained image vision transformer.<n>We propose the Structured Dynamics Model (SDM), which explicitly separates the dominant source of temporal change from residual dynamics.<n>We evaluate SDM on ProbeMotion, a new evaluation suite spanning synthetic and real videos with camera motion, object motion, and combined dynamics.
  arXiv  Detail & Related papers  (2026-07-23T17:55:07Z)
• Native Video-Action Pretraining for Generalizable Robot Control [77.59831077077176]
  LingBot-VA 2.0 is a video-action foundation model built from the ground up for embodiment.<n>To meet the demands of high-frequency inference, our model employs a sparse MoE backbone.<n>Real-world deployment validates LingBot-VA 2.0 as a robust foundation model.
  arXiv  Detail & Related papers  (2026-07-09T16:15:43Z)
• Tempered Self-Similarity Alignment for Physically Plausible Video Generation [83.40337664171939]
  Self-similarity loss transforms into probabilistic correspondence and trains the video generative model to align its correspondence with those of the visual foundation model on dynamically changing regions.<n>Our method demonstrates substantial improvements in physical plausibility across diverse interaction scenarios, validating the effectiveness of transferring relational knowledge for video generation.
  arXiv  Detail & Related papers  (2026-05-24T09:28:05Z)
• GenMatter: Perceiving Physical Objects with Generative Matter Models [50.121713214509604]
  Humans robustly segment moving entities that constitute independently moveable chunks of matter.<n>Existing computer vision systems lack a unified approach that works across diverse settings.<n>We propose a generative model that hierarchically groups low-level motion cues and high-level appearance features into particles.
  arXiv  Detail & Related papers  (2026-04-24T02:18:18Z)
• Do vision models perceive illusory motion in static images like humans? [2.1349209400003932]
  Deep neural networks (DNNs) achieve strong performance in optical flow estimation, but remain less robust than humans.<n>Visual motion illusions provide a powerful probe into these mechanisms, revealing how human and machine vision align or diverge.<n>We evaluate several representative optical flow models on Rotating Snakes and show that most fail to generate flow fields consistent with human perception.
  arXiv  Detail & Related papers  (2026-04-10T19:33:33Z)
• Puppeteer: Rig and Animate Your 3D Models [105.11046762553121]
  Puppeteer is a comprehensive framework that addresses both automatic rigging and animation for diverse 3D objects.<n>Our system first predicts plausible skeletal structures via an auto-regressive transformer.<n>It then infers skinning weights via an attention-based architecture.
  arXiv  Detail & Related papers  (2025-08-14T17:59:31Z)
• Zero-Shot Visual Generalization in Robot Manipulation [0.13280779791485384]
  Current approaches often sidestep the problem by relying on invariant representations such as point clouds and depth.<n>Disentangled representation learning has recently shown promise in enabling vision-based reinforcement learning policies to be robust to visual distribution shifts.<n>We demonstrate zero-shot adaptability to visual perturbations in both simulation and on real hardware.
  arXiv  Detail & Related papers  (2025-05-16T22:01:46Z)
• InterDyn: Controllable Interactive Dynamics with Video Diffusion Models [50.38647583839384]
  We propose InterDyn, a framework that generates videos of interactive dynamics given an initial frame and a control signal encoding the motion of a driving object or actor.<n>Our key insight is that large video generation models can act as both neurals and implicit physics simulators'', having learned interactive dynamics from large-scale video data.
  arXiv  Detail & Related papers  (2024-12-16T13:57:02Z)
• Modelling Human Visual Motion Processing with Trainable Motion Energy Sensing and a Self-attention Network [1.9458156037869137]
  We propose an image-computable model of human motion perception by bridging the gap between biological and computer vision models. This model architecture aims to capture the computations in V1-MT, the core structure for motion perception in the biological visual system. In silico neurophysiology reveals that our model's unit responses are similar to mammalian neural recordings regarding motion pooling and speed tuning.
  arXiv  Detail & Related papers  (2023-05-16T04:16:07Z)
• Reconstruction-guided attention improves the robustness and shape processing of neural networks [5.156484100374057]
  We build an iterative encoder-decoder network that generates an object reconstruction and uses it as top-down attentional feedback. Our model shows strong generalization performance against various image perturbations. Our study shows that modeling reconstruction-based feedback endows AI systems with a powerful attention mechanism.
  arXiv  Detail & Related papers  (2022-09-27T18:32:22Z)
• Human Eyes Inspired Recurrent Neural Networks are More Robust Against Adversarial Noises [7.689542442882423]
  We designed a dual-stream vision model inspired by the human brain. This model features retina-like input layers and includes two streams: one determining the next point of focus (the fixation), while the other interprets the visuals surrounding the fixation. We evaluated this model against various benchmarks in terms of object recognition, gaze behavior and adversarial robustness.
  arXiv  Detail & Related papers  (2022-06-15T03:44:42Z)
• Learning Motion-Dependent Appearance for High-Fidelity Rendering of Dynamic Humans from a Single Camera [49.357174195542854]
  A key challenge of learning the dynamics of the appearance lies in the requirement of a prohibitively large amount of observations. We show that our method can generate a temporally coherent video of dynamic humans for unseen body poses and novel views given a single view video.
  arXiv  Detail & Related papers  (2022-03-24T00:22:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.