New York Smells: A Large Multimodal Dataset for Olfaction

• URL: http://arxiv.org/abs/2511.20544v1
• Date: Tue, 25 Nov 2025 17:44:50 GMT
• Title: New York Smells: A Large Multimodal Dataset for Olfaction
• Abstract summary: We present New York Smells, a large dataset of paired image and olfactory signals captured in the wild''<n>Our dataset contains 7,000 smell-image pairs from 3,500 distinct objects across indoor and outdoor environments.<n>Our benchmark has three tasks: cross-modal smell-to-image retrieval, recognizing scenes, objects, and materials from smell alone, and fine-grained discrimination between grass species.
• Score: 93.162210916881
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: While olfaction is central to how animals perceive the world, this rich chemical sensory modality remains largely inaccessible to machines. One key bottleneck is the lack of diverse, multimodal olfactory training data collected in natural settings. We present New York Smells, a large dataset of paired image and olfactory signals captured ``in the wild.'' Our dataset contains 7,000 smell-image pairs from 3,500 distinct objects across indoor and outdoor environments, with approximately 70$\times$ more objects than existing olfactory datasets. Our benchmark has three tasks: cross-modal smell-to-image retrieval, recognizing scenes, objects, and materials from smell alone, and fine-grained discrimination between grass species. Through experiments on our dataset, we find that visual data enables cross-modal olfactory representation learning, and that our learned olfactory representations outperform widely-used hand-crafted features.

Related papers

• What Images Cannot Say: Language-Guided Olfactory Representation Learning [17.56125098945589]
  We introduce SCENT, a framework that uses language guidance as a semantic bridge between vision and olfaction.<n>We train a smell encoder that maps electronic-nose signals into a shared embedding space aligned with both visual and textual representations.<n>Our results reveal the importance of contextual semantic information for grounding olfactory perception in multimodal learning.
  arXiv  Detail & Related papers  (2026-07-07T15:31:55Z)
• See & Sniff: Learning Visuo-Olfactory Representations [42.26043073031702]
  We introduce SmellNet-V, a scalable visuo-olfactory dataset built on the insight that odor identity is largely invariant to visual transformations within a semantic category.<n>This allows us to synthetically pair smell-only samples with semantically aligned in-the-wild web images, converting a unimodal olfactory dataset into a cross-modal benchmark.<n>We propose See & Sniff, a self-supervised framework that learns joint visuo-olfactory representations via dense local alignment.
  arXiv  Detail & Related papers  (2026-06-25T17:26:47Z)
• SMELLNET: A Large-scale Dataset for Real-world Smell Recognition [26.27822839167258]
  SmellNet is the first large-scale database that digitizes a diverse range of smells in the natural world.<n>ScentFormer is a Transformer-based architecture combining temporal differencing and sliding-window augmentation for smell data.<n>ScentFormer and SmellNet lay the groundwork for real-world olfactory applications across healthcare, food and beverage, environmental monitoring, manufacturing, and entertainment.
  arXiv  Detail & Related papers  (2025-05-30T21:15:25Z)
• X-Capture: An Open-Source Portable Device for Multi-Sensory Learning [11.632896115888261]
  We introduce X-Capture, an open-source, portable, and cost-effective device for real-world multi-sensory data collection.<n>With a build cost under $1,000, X-Capture democratizes the creation of multi-sensory datasets.<n>X-Capture lays the groundwork for advancing human-like sensory representations in AI.
  arXiv  Detail & Related papers  (2025-04-03T06:44:25Z)
• Estimating the distribution of numerosity and non-numerical visual magnitudes in natural scenes using computer vision [0.08192907805418582]
  We show that in natural visual scenes the frequency of appearance of different numerosities follows a power law distribution. We show that the correlational structure for numerosity and continuous magnitudes is stable across datasets and scene types.
  arXiv  Detail & Related papers  (2024-09-17T09:49:29Z)
• Data Science In Olfaction [1.4499463058550683]
  We conceptualize smell from a Data Science and AI perspective, that relates the properties of odorants to how they are sensed and analyzed in the olfactory system from the nose to the brain. Drawing distinctions to color vision, we argue that smell presents unique measurement challenges, including the complexity of stimuli, the high dimensionality of the sensory apparatus, as well as what constitutes ground truth. We present results using machine learning-based classification of neural responses to odors as they are recorded in the mouse olfactory bulb with calcium imaging.
  arXiv  Detail & Related papers  (2024-04-08T13:25:02Z)
• DNA-Rendering: A Diverse Neural Actor Repository for High-Fidelity Human-centric Rendering [126.00165445599764]
  We present DNA-Rendering, a large-scale, high-fidelity repository of human performance data for neural actor rendering. Our dataset contains over 1500 human subjects, 5000 motion sequences, and 67.5M frames' data volume. We construct a professional multi-view system to capture data, which contains 60 synchronous cameras with max 4096 x 3000 resolution, 15 fps speed, and stern camera calibration steps.
  arXiv  Detail & Related papers  (2023-07-19T17:58:03Z)
• Habitat Synthetic Scenes Dataset (HSSD-200): An Analysis of 3D Scene Scale and Realism Tradeoffs for ObjectGoal Navigation [70.82403156865057]
  We investigate the impact of synthetic 3D scene dataset scale and realism on the task of training embodied agents to find and navigate to objects. Our experiments show that agents trained on our smaller-scale dataset can match or outperform agents trained on much larger datasets.
  arXiv  Detail & Related papers  (2023-06-20T05:07:23Z)
• The ObjectFolder Benchmark: Multisensory Learning with Neural and Real Objects [51.22194706674366]
  We introduce the Object Benchmark, a benchmark suite of 10 tasks for multisensory object-centric learning. We also introduce the Object Real dataset, including the multisensory measurements for 100 real-world household objects.
  arXiv  Detail & Related papers  (2023-06-01T17:51:22Z)
• StyleGAN-Human: A Data-Centric Odyssey of Human Generation [96.7080874757475]
  This work takes a data-centric perspective and investigates multiple critical aspects in "data engineering" We collect and annotate a large-scale human image dataset with over 230K samples capturing diverse poses and textures. We rigorously investigate three essential factors in data engineering for StyleGAN-based human generation, namely data size, data distribution, and data alignment.
  arXiv  Detail & Related papers  (2022-04-25T17:55:08Z)
• Learning from THEODORE: A Synthetic Omnidirectional Top-View Indoor Dataset for Deep Transfer Learning [4.297070083645049]
  We introduce THEODORE: a novel, large-scale indoor dataset containing 100,000 high-resolution diversified fisheye images with 14 classes. We create 3D virtual environments of living rooms, different human characters and interior textures. We show that our dataset is well suited for fine-tuning CNNs for object detection.
  arXiv  Detail & Related papers  (2020-11-11T11:46:33Z)
• What Can You Learn from Your Muscles? Learning Visual Representation from Human Interactions [50.435861435121915]
  We use human interaction and attention cues to investigate whether we can learn better representations compared to visual-only representations. Our experiments show that our "muscly-supervised" representation outperforms a visual-only state-of-the-art method MoCo.
  arXiv  Detail & Related papers  (2020-10-16T17:46:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.