See & Sniff: Learning Visuo-Olfactory Representations

• URL: http://arxiv.org/abs/2606.27307v1
• Date: Thu, 25 Jun 2026 17:26:47 GMT
• Title: See & Sniff: Learning Visuo-Olfactory Representations
• Abstract summary: We introduce SmellNet-V, a scalable visuo-olfactory dataset built on the insight that odor identity is largely invariant to visual transformations within a semantic category.<n>This allows us to synthetically pair smell-only samples with semantically aligned in-the-wild web images, converting a unimodal olfactory dataset into a cross-modal benchmark.<n>We propose See & Sniff, a self-supervised framework that learns joint visuo-olfactory representations via dense local alignment.
• Score: 42.26043073031702
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: While modern multimodal models integrate vision with language, audio, or touch, olfaction remains largely unexplored due to the lack of paired visuo-olfactory data. We introduce SmellNet-V, a scalable visuo-olfactory dataset built on the insight that odor identity is largely invariant to visual transformations within a semantic category. This allows us to synthetically pair smell-only samples with semantically aligned in-the-wild web images, converting a unimodal olfactory dataset into a cross-modal benchmark without costly co-collection. Building on this dataset, we propose See & Sniff, a self-supervised framework that learns joint visuo-olfactory representations via dense local alignment and naturally produces smell saliency maps for spatial grounding of odor sources. We further introduce pixel-level smell localization task and a benchmark for evaluation. Our method surpasses smell-only baselines by 7% in smell classification from smell alone and generalizes to cross-modal retrieval and smell localization, establishing visuo-olfactory learning as a new direction in multimodal perception.

Related papers

• What Images Cannot Say: Language-Guided Olfactory Representation Learning [17.56125098945589]
  We introduce SCENT, a framework that uses language guidance as a semantic bridge between vision and olfaction.<n>We train a smell encoder that maps electronic-nose signals into a shared embedding space aligned with both visual and textual representations.<n>Our results reveal the importance of contextual semantic information for grounding olfactory perception in multimodal learning.
  arXiv  Detail & Related papers  (2026-07-07T15:31:55Z)
• AromaGen: Interactive Generation of Rich Olfactory Experiences with Multimodal Language Models [29.111586871914167]
  AromaGen is an AI-powered wearable interface capable of real-time, general-purpose aroma generation from free-form text or visual inputs.<n> AromaGen matches human-composed mixtures in zero-shot generation and significantly surpasses them after iterative refinement.
  arXiv  Detail & Related papers  (2026-04-02T05:47:07Z)
• VIGIL: Tackling Hallucination Detection in Image Recontextualization [0.0]
  We introduce VIGIL, the first benchmark dataset and framework providing a fine-grained categorization of hallucinations.<n>Our work addresses a significant gap in multimodal evaluation by decomposing hallucinations into five categories.<n>Our architecture processes recontextualized images through a series of specialized steps targeting object-level fidelity, background consistency, and omission detection.
  arXiv  Detail & Related papers  (2026-02-16T10:47:10Z)
• New York Smells: A Large Multimodal Dataset for Olfaction [93.162210916881]
  We present New York Smells, a large dataset of paired image and olfactory signals captured in the wild''<n>Our dataset contains 7,000 smell-image pairs from 3,500 distinct objects across indoor and outdoor environments.<n>Our benchmark has three tasks: cross-modal smell-to-image retrieval, recognizing scenes, objects, and materials from smell alone, and fine-grained discrimination between grass species.
  arXiv  Detail & Related papers  (2025-11-25T17:44:50Z)
• A Novel Taxonomy and Classification Scheme for Code Smell Interactions [2.6597689982591044]
  This study presents a novel taxonomy and a proposed classification scheme for the possible code smell interactions.<n>Experiments have been carried out using several popular machine learning (ML) models.<n>Results primarily show the presence of code smell interactions namely Inter-smell Detection within domain.
  arXiv  Detail & Related papers  (2025-04-25T16:24:11Z)
• DiffVein: A Unified Diffusion Network for Finger Vein Segmentation and Authentication [50.017055360261665]
  We introduce DiffVein, a unified diffusion model-based framework which simultaneously addresses vein segmentation and authentication tasks. For better feature interaction between these two branches, we introduce two specialized modules. In this way, our framework allows for a dynamic interplay between diffusion and segmentation embeddings.
  arXiv  Detail & Related papers  (2024-02-03T06:49:42Z)
• Dual-View Data Hallucination with Semantic Relation Guidance for Few-Shot Image Recognition [49.26065739704278]
  We propose a framework that exploits semantic relations to guide dual-view data hallucination for few-shot image recognition. An instance-view data hallucination module hallucinates each sample of a novel class to generate new data. A prototype-view data hallucination module exploits semantic-aware measure to estimate the prototype of a novel class.
  arXiv  Detail & Related papers  (2024-01-13T12:32:29Z)
• A Diffusion Weighted Graph Framework for New Intent Discovery [25.364554033681515]
  New Intent Discovery (NID) aims to recognize both new and known intents from unlabeled data. Previous methods generate noisy supervisory signals which cannot strike a balance between quantity and quality. We propose a novel Diffusion Weighted Graph Framework (DWGF) to capture both semantic similarities and structure relationships inherent in data.
  arXiv  Detail & Related papers  (2023-10-24T13:43:01Z)
• Domain Adaptive Synapse Detection with Weak Point Annotations [63.97144211520869]
  We present AdaSyn, a framework for domain adaptive synapse detection with weak point annotations. In the WASPSYN challenge at I SBI 2023, our method ranks the 1st place.
  arXiv  Detail & Related papers  (2023-08-31T05:05:53Z)
• Learning Signal-Agnostic Manifolds of Neural Fields [50.066449953522685]
  We leverage neural fields to capture the underlying structure in image, shape, audio and cross-modal audiovisual domains. We show that by walking across the underlying manifold of GEM, we may generate new samples in our signal domains.
  arXiv  Detail & Related papers  (2021-11-11T18:57:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.