Can Transformers Smell Like Humans?

• URL: http://arxiv.org/abs/2411.03038v1
• Date: Tue, 05 Nov 2024 12:19:39 GMT
• Title: Can Transformers Smell Like Humans?
• Authors: Farzaneh Taleb, Miguel Vasco, Antônio H. Ribeiro, Mårten Björkman, Danica Kragic,
• Abstract summary: We show that representations encoded from transformers pre-trained on general chemical structures are highly aligned with human olfactory perception. We also evaluate the extent to which this alignment is associated with physicochemical features of odorants known to be relevant for olfactory decoding.
• Score: 17.14976015153551
• License:
• Abstract: The human brain encodes stimuli from the environment into representations that form a sensory perception of the world. Despite recent advances in understanding visual and auditory perception, olfactory perception remains an under-explored topic in the machine learning community due to the lack of large-scale datasets annotated with labels of human olfactory perception. In this work, we ask the question of whether pre-trained transformer models of chemical structures encode representations that are aligned with human olfactory perception, i.e., can transformers smell like humans? We demonstrate that representations encoded from transformers pre-trained on general chemical structures are highly aligned with human olfactory perception. We use multiple datasets and different types of perceptual representations to show that the representations encoded by transformer models are able to predict: (i) labels associated with odorants provided by experts; (ii) continuous ratings provided by human participants with respect to pre-defined descriptors; and (iii) similarity ratings between odorants provided by human participants. Finally, we evaluate the extent to which this alignment is associated with physicochemical features of odorants known to be relevant for olfactory decoding.

Related papers

• Are Transformers Able to Reason by Connecting Separated Knowledge in Training Data? [55.90575874130038]
  Humans exhibit remarkable compositional reasoning by integrating knowledge from various sources. We introduce a synthetic learning task to validate the potential of Transformers in replicating this skill. We find that few-shot Chain-of-Thought prompting enables Transformers to perform compositional reasoning on FTCT.
  arXiv  Detail & Related papers  (2025-01-27T08:34:38Z)
• DeepNose: An Equivariant Convolutional Neural Network Predictive Of Human Olfactory Percepts [0.9187505256430946]
  We train a convolutional neural network (CNN) to predict human percepts from semantic datasets. Our network offers high-fidelity perceptual predictions for different olfactory datasets. We propose that the DeepNose network can use 3D molecular shapes to generate high-quality predictions for human olfactory percepts.
  arXiv  Detail & Related papers  (2024-12-11T19:35:24Z)
• Sniff AI: Is My 'Spicy' Your 'Spicy'? Exploring LLM's Perceptual Alignment with Human Smell Experiences [12.203995379495916]
  This work focuses on olfaction, human smell experiences. We conducted a user study with 40 participants to investigate how well AI can interpret human descriptions of scents. Results indicated limited perceptual alignment, with biases towards certain scents, like lemon and peppermint, and continued failing to identify others, like rosemary.
  arXiv  Detail & Related papers  (2024-11-11T12:56:52Z)
• Closely Interactive Human Reconstruction with Proxemics and Physics-Guided Adaption [64.07607726562841]
  Existing multi-person human reconstruction approaches mainly focus on recovering accurate poses or avoiding penetration. In this work, we tackle the task of reconstructing closely interactive humans from a monocular video. We propose to leverage knowledge from proxemic behavior and physics to compensate the lack of visual information.
  arXiv  Detail & Related papers  (2024-04-17T11:55:45Z)
• Data Science In Olfaction [1.4499463058550683]
  We conceptualize smell from a Data Science and AI perspective, that relates the properties of odorants to how they are sensed and analyzed in the olfactory system from the nose to the brain. Drawing distinctions to color vision, we argue that smell presents unique measurement challenges, including the complexity of stimuli, the high dimensionality of the sensory apparatus, as well as what constitutes ground truth. We present results using machine learning-based classification of neural responses to odors as they are recorded in the mouse olfactory bulb with calcium imaging.
  arXiv  Detail & Related papers  (2024-04-08T13:25:02Z)
• ReGenNet: Towards Human Action-Reaction Synthesis [87.57721371471536]
  We analyze the asymmetric, dynamic, synchronous, and detailed nature of human-human interactions. We propose the first multi-setting human action-reaction benchmark to generate human reactions conditioned on given human actions.
  arXiv  Detail & Related papers  (2024-03-18T15:33:06Z)
• Human-Machine Cooperative Multimodal Learning Method for Cross-subject Olfactory Preference Recognition [11.566318118981453]
  Olfactory electroencephalogram (EEG) contains odor and individual features associated with human olfactory preference. An E-nose and olfactory EEG multimodal learning method is proposed for cross-subject olfactory preference recognition.
  arXiv  Detail & Related papers  (2023-11-24T11:59:11Z)
• TransHuman: A Transformer-based Human Representation for Generalizable Neural Human Rendering [52.59454369653773]
  We focus on the task of generalizable neural rendering human which trains conditional Radiance Fields (NeRF) from multi-view videos of different characters. Previous methods have primarily used a SparseConvNet (SPC)-based human representation to process the painted SMPL. We present a brand-new framework named TransHuman, which learns the painted SMPL under the canonical space and captures the global relationships between human parts.
  arXiv  Detail & Related papers  (2023-07-23T10:59:51Z)
• What Makes for Good Tokenizers in Vision Transformer? [62.44987486771936]
  transformers are capable of extracting their pairwise relationships using self-attention. What makes for a good tokenizer has not been well understood in computer vision. Modulation across Tokens (MoTo) incorporates inter-token modeling capability through normalization. Regularization objective TokenProp is embraced in the standard training regime.
  arXiv  Detail & Related papers  (2022-12-21T15:51:43Z)
• Data-driven emotional body language generation for social robotics [58.88028813371423]
  In social robotics, endowing humanoid robots with the ability to generate bodily expressions of affect can improve human-robot interaction and collaboration. We implement a deep learning data-driven framework that learns from a few hand-designed robotic bodily expressions. The evaluation study found that the anthropomorphism and animacy of the generated expressions are not perceived differently from the hand-designed ones.
  arXiv  Detail & Related papers  (2022-05-02T09:21:39Z)
• Does the Brain Infer Invariance Transformations from Graph Symmetries? [0.0]
  The invariance of natural objects under perceptual changes is possibly encoded in the brain by symmetries in the graph of synaptic connections. The graph can be established via unsupervised learning in a biologically plausible process across different perceptual modalities.
  arXiv  Detail & Related papers  (2021-11-11T12:35:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.