Seeing like a Cephalopod: Colour Vision with a Monochrome Event Camera

• URL: http://arxiv.org/abs/2504.10984v1
• Date: Tue, 15 Apr 2025 08:47:11 GMT
• Title: Seeing like a Cephalopod: Colour Vision with a Monochrome Event Camera
• Authors: Sami Arja, Nimrod Kruger, Alexandre Marcireau, Nicholas Owen Ralph, Saeed Afshar, Gregory Cohen,
• Abstract summary: Cephalopods exhibit unique colour discrimination capabilities despite having one type of photoreceptor.<n>We take inspiration from this biological mechanism to design a spectral imaging system that combines a ball lens with an event-based camera.
• Score: 37.881496223977706
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Cephalopods exhibit unique colour discrimination capabilities despite having one type of photoreceptor, relying instead on chromatic aberration induced by their ocular optics and pupil shapes to perceive spectral information. We took inspiration from this biological mechanism to design a spectral imaging system that combines a ball lens with an event-based camera. Our approach relies on a motorised system that shifts the focal position, mirroring the adaptive lens motion in cephalopods. This approach has enabled us to achieve wavelength-dependent focusing across the visible light and near-infrared spectrum, making the event a spectral sensor. We characterise chromatic aberration effects, using both event-based and conventional frame-based sensors, validating the effectiveness of bio-inspired spectral discrimination both in simulation and in a real setup as well as assessing the spectral discrimination performance. Our proposed approach provides a robust spectral sensing capability without conventional colour filters or computational demosaicing. This approach opens new pathways toward new spectral sensing systems inspired by nature's evolutionary solutions. Code and analysis are available at: https://samiarja.github.io/neuromorphic_octopus_eye/

Related papers

• CARL: Camera-Agnostic Representation Learning for Spectral Image Analysis [75.25966323298003]
  Spectral imaging offers promising applications across diverse domains, including medicine and urban scene understanding. variability in channel dimensionality and captured wavelengths among spectral cameras impede the development of AI-driven methodologies. We introduce $textbfCARL$, a model for $textbfC$amera-$textbfA$gnostic $textbfR$esupervised $textbfL$ across RGB, multispectral, and hyperspectral imaging modalities.
  arXiv  Detail & Related papers  (2025-04-27T13:06:40Z)
• Spectrum from Defocus: Fast Spectral Imaging with Chromatic Focal Stack [3.51545442219172]
  Hyperspectral cameras face harsh trade-offs between spatial, spectral, and temporal resolution in an inherently low-photon regime.<n>We present Spectrum from Defocus, a chromatic focal sweep method that recovers state-of-the-art hyperspectral images with a small system of off-the-shelf optics and 1 second of compute.
  arXiv  Detail & Related papers  (2025-03-26T03:17:43Z)
• Snapshot multi-spectral imaging through defocusing and a Fourier imager network [13.068631760956265]
  We introduce a snapshot multi-spectral imaging approach employing a standard monochrome image sensor with no additional spectral filters or customized components.<n>This deep learning-powered framework achieves high-quality multi-spectral image reconstruction using snapshot image acquisition with a monochrome image sensor.
  arXiv  Detail & Related papers  (2025-01-24T07:04:27Z)
• Paleoinspired Vision: From Exploring Colour Vision Evolution to Inspiring Camera Design [44.353407126161514]
  We present a model of visual transduction in the retina, introducing a novel opsin layer.<n>We quantify evolutionary pressures by measuring machine vision recognition accuracy on colour images shaped by specific opsins.<n>We develop an evolutionary conservation optimisation algorithm to reconstruct the spectral sensitivity of opsins.
  arXiv  Detail & Related papers  (2024-12-27T04:07:52Z)
• Spatial super-resolution in nanosensing with blinking emitters [79.16635054977068]
  We propose a method of spatial resolution enhancement in metrology with blinking fluorescent nanosensors. We believe that blinking fluorescent sensing agents being complemented with the developed image analysis technique could be utilized routinely in the life science sector.
  arXiv  Detail & Related papers  (2024-02-27T10:38:05Z)
• Multispectral Stereo-Image Fusion for 3D Hyperspectral Scene Reconstruction [4.2056926734482065]
  We present a novel approach combining two calibrated multispectral real-time capable snapshot cameras, covering different spectral ranges, into a stereo-system. The combined use of different multispectral snapshot cameras enables both 3D reconstruction and spectral analysis.
  arXiv  Detail & Related papers  (2023-12-15T13:20:35Z)
• ColorSense: A Study on Color Vision in Machine Visual Recognition [57.916512479603064]
  We collect 110,000 non-trivial human annotations of foreground and background color labels from visual recognition benchmarks.<n>We validate the use of our datasets by demonstrating that the level of color discrimination has a dominating effect on the performance of machine perception models.<n>Our findings suggest that object recognition tasks such as classification and localization are susceptible to color vision bias.
  arXiv  Detail & Related papers  (2022-12-16T18:51:41Z)
• Inverse-Designed Meta-Optics with Spectral-Spatial Engineered Response to Mimic Color Perception [0.1773006951689229]
  inverse-design approach is utilized to create a set of single-layer meta-optics that simultaneously focus light and shape the spectra of focused light without using any filters. Results show that both spatial and spectral properties of the meta-optics are optimized, resulting in spectra that mimic the color matching functions of the CIE 1931 XYZ color space.
  arXiv  Detail & Related papers  (2022-04-28T14:12:54Z)
• Single-shot Hyperspectral-Depth Imaging with Learned Diffractive Optics [72.9038524082252]
  We propose a compact single-shot monocular hyperspectral-depth (HS-D) imaging method. Our method uses a diffractive optical element (DOE), the point spread function of which changes with respect to both depth and spectrum. To facilitate learning the DOE, we present a first HS-D dataset by building a benchtop HS-D imager.
  arXiv  Detail & Related papers  (2020-09-01T14:19:35Z)
• Cross-Spectral Periocular Recognition with Conditional Adversarial Networks [59.17685450892182]
  We propose Conditional Generative Adversarial Networks, trained to con-vert periocular images between visible and near-infrared spectra. We obtain a cross-spectral periocular performance of EER=1%, and GAR>99% @ FAR=1%, which is comparable to the state-of-the-art with the PolyU database.
  arXiv  Detail & Related papers  (2020-08-26T15:02:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.