FOVI: A biologically-inspired foveated interface for deep vision models

• URL: http://arxiv.org/abs/2602.03766v1
• Date: Tue, 03 Feb 2026 17:26:54 GMT
• Title: FOVI: A biologically-inspired foveated interface for deep vision models
• Authors: Nicholas M. Blauch, George A. Alvarez, Talia Konkle,
• Abstract summary: We propose a foveated vision interface based on the human retina and primary visual cortex.<n>Receptive fields are defined as k-nearest-neighborhoods (kNNs) on the sensor manifold.<n>We demonstrate two use cases: (1) an end-to-end kNN-convolutional architecture, and (2) a foveated adaptation of the foundational DINOv3 ViT model.
• Score: 5.6075902312642745
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Human vision is foveated, with variable resolution peaking at the center of a large field of view; this reflects an efficient trade-off for active sensing, allowing eye-movements to bring different parts of the world into focus with other parts of the world in context. In contrast, most computer vision systems encode the visual world at a uniform resolution, raising challenges for processing full-field high-resolution images efficiently. We propose a foveated vision interface (FOVI) based on the human retina and primary visual cortex, that reformats a variable-resolution retina-like sensor array into a uniformly dense, V1-like sensor manifold. Receptive fields are defined as k-nearest-neighborhoods (kNNs) on the sensor manifold, enabling kNN-convolution via a novel kernel mapping technique. We demonstrate two use cases: (1) an end-to-end kNN-convolutional architecture, and (2) a foveated adaptation of the foundational DINOv3 ViT model, leveraging low-rank adaptation (LoRA). These models provide competitive performance at a fraction of the computational cost of non-foveated baselines, opening pathways for efficient and scalable active sensing for high-resolution egocentric vision. Code and pre-trained models are available at https://github.com/nblauch/fovi and https://huggingface.co/fovi-pytorch.

Related papers

• Emulating Human-like Adaptive Vision for Efficient and Flexible Machine Visual Perception [93.20637973889434]
  We introduce AdaptiveNN, a general framework aiming to drive a paradigm shift from 'passive' to 'active' vision models.<n> AdaptiveNN formulates visual perception as a coarse-to-fine sequential decision-making process.<n>We assess AdaptiveNN on 17 benchmarks spanning 9 tasks, including large-scale visual recognition, fine-grained discrimination, visual search, and processing images from real driving and medical scenarios.
  arXiv  Detail & Related papers  (2025-09-18T18:25:43Z)
• DINO-VO: A Feature-based Visual Odometry Leveraging a Visual Foundation Model [2.163881720692685]
  Learning-based monocular visual odometry (VO) poses robustness, generalization, and efficiency challenges in robotics.<n>Recent advances in visual foundation models, such as DINOv2, have improved robustness and generalization in various vision tasks.<n>We present DINO-VO, a feature-based VO system leveraging DINOv2 visual foundation model for its sparse feature matching.
  arXiv  Detail & Related papers  (2025-07-17T14:09:34Z)
• A Sensorimotor Vision Transformer [0.0]
  Sensorimotor Transformer (SMT) is a vision model inspired by human saccadic eye movements.<n>SMT identifies and selects the most salient patches based on intrinsic two-dimensional (i2D) features.
  arXiv  Detail & Related papers  (2025-04-03T12:37:44Z)
• LaVin-DiT: Large Vision Diffusion Transformer [99.98106406059333]
  LaVin-DiT is a scalable and unified foundation model designed to tackle over 20 computer vision tasks in a generative framework.<n>We introduce key innovations to optimize generative performance for vision tasks.<n>The model is scaled from 0.1B to 3.4B parameters, demonstrating substantial scalability and state-of-the-art performance across diverse vision tasks.
  arXiv  Detail & Related papers  (2024-11-18T12:05:27Z)
• Learning 1D Causal Visual Representation with De-focus Attention Networks [108.72931590504406]
  This paper explores the feasibility of representing images using 1D causal modeling. We propose De-focus Attention Networks, which employ learnable bandpass filters to create varied attention patterns.
  arXiv  Detail & Related papers  (2024-06-06T17:59:56Z)
• FViT: A Focal Vision Transformer with Gabor Filter [6.237269022600682]
  We discuss the potential advantages of combining vision transformers with Gabor filters.<n>A learnable Gabor filter (LGF) using convolution is proposed.<n>A Bionic Focal Vision (BFV) block is designed based on the LGF.<n>A unified and efficient family of pyramid backbone networks called Focal Vision Transformers (FViTs) is developed.
  arXiv  Detail & Related papers  (2024-02-17T15:03:25Z)
• VMamba: Visual State Space Model [98.0517369083152]
  We adapt Mamba, a state-space language model, into VMamba, a vision backbone with linear time complexity.<n>At the core of VMamba is a stack of Visual State-Space (VSS) blocks with the 2D Selective Scan (SS2D) module.
  arXiv  Detail & Related papers  (2024-01-18T17:55:39Z)
• Laplacian-Former: Overcoming the Limitations of Vision Transformers in Local Texture Detection [3.784298636620067]
  Vision Transformer (ViT) models have demonstrated a breakthrough in a wide range of computer vision tasks. These models struggle to capture high-frequency components of images, which can limit their ability to detect local textures and edge information. We propose a new technique, Laplacian-Former, that enhances the self-attention map by adaptively re-calibrating the frequency information in a Laplacian pyramid.
  arXiv  Detail & Related papers  (2023-08-31T19:56:14Z)
• Vision Transformers: From Semantic Segmentation to Dense Prediction [139.15562023284187]
  We explore the global context learning potentials of vision transformers (ViTs) for dense visual prediction. Our motivation is that through learning global context at full receptive field layer by layer, ViTs may capture stronger long-range dependency information. We formulate a family of Hierarchical Local-Global (HLG) Transformers, characterized by local attention within windows and global-attention across windows in a pyramidal architecture.
  arXiv  Detail & Related papers  (2022-07-19T15:49:35Z)
• Visual Attention Network [90.0753726786985]
  We propose a novel large kernel attention (LKA) module to enable self-adaptive and long-range correlations in self-attention. We also introduce a novel neural network based on LKA, namely Visual Attention Network (VAN) VAN outperforms the state-of-the-art vision transformers and convolutional neural networks with a large margin in extensive experiments.
  arXiv  Detail & Related papers  (2022-02-20T06:35:18Z)
• Global Filter Networks for Image Classification [90.81352483076323]
  We present a conceptually simple yet computationally efficient architecture that learns long-term spatial dependencies in the frequency domain with log-linear complexity. Our results demonstrate that GFNet can be a very competitive alternative to transformer-style models and CNNs in efficiency, generalization ability and robustness.
  arXiv  Detail & Related papers  (2021-07-01T17:58:16Z)
• MetaAvatar: Learning Animatable Clothed Human Models from Few Depth Images [60.56518548286836]
  To generate realistic cloth deformations from novel input poses, watertight meshes or dense full-body scans are usually needed as inputs. We propose an approach that can quickly generate realistic clothed human avatars, represented as controllable neural SDFs, given only monocular depth images.
  arXiv  Detail & Related papers  (2021-06-22T17:30:12Z)
• Emergent Properties of Foveated Perceptual Systems [3.3504365823045044]
  This work is inspired by the foveated human visual system, which has higher acuity at the center of gaze and texture-like encoding in the periphery. We introduce models consisting of a first-stage textitfixed image transform followed by a second-stage textitlearnable convolutional neural network. We find that foveation with peripheral texture-based computations yields an efficient, distinct, and robust representational format of scene information.
  arXiv  Detail & Related papers  (2020-06-14T19:34:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.