Sniff AI: Is My 'Spicy' Your 'Spicy'? Exploring LLM's Perceptual Alignment with Human Smell Experiences

• URL: http://arxiv.org/abs/2411.06950v1
• Date: Mon, 11 Nov 2024 12:56:52 GMT
• Title: Sniff AI: Is My 'Spicy' Your 'Spicy'? Exploring LLM's Perceptual Alignment with Human Smell Experiences
• Authors: Shu Zhong, Zetao Zhou, Christopher Dawes, Giada Brianz, Marianna Obrist,
• Abstract summary: This work focuses on olfaction, human smell experiences. We conducted a user study with 40 participants to investigate how well AI can interpret human descriptions of scents. Results indicated limited perceptual alignment, with biases towards certain scents, like lemon and peppermint, and continued failing to identify others, like rosemary.
• Score: 12.203995379495916
• License:
• Abstract: Aligning AI with human intent is important, yet perceptual alignment-how AI interprets what we see, hear, or smell-remains underexplored. This work focuses on olfaction, human smell experiences. We conducted a user study with 40 participants to investigate how well AI can interpret human descriptions of scents. Participants performed "sniff and describe" interactive tasks, with our designed AI system attempting to guess what scent the participants were experiencing based on their descriptions. These tasks evaluated the Large Language Model's (LLMs) contextual understanding and representation of scent relationships within its internal states - high-dimensional embedding space. Both quantitative and qualitative methods were used to evaluate the AI system's performance. Results indicated limited perceptual alignment, with biases towards certain scents, like lemon and peppermint, and continued failing to identify others, like rosemary. We discuss these findings in light of human-AI alignment advancements, highlighting the limitations and opportunities for enhancing HCI systems with multisensory experience integration.

Related papers

• Memento No More: Coaching AI Agents to Master Multiple Tasks via Hints Internalization [56.674356045200696]
  We propose a novel method to train AI agents to incorporate knowledge and skills for multiple tasks without the need for cumbersome note systems or prior high-quality demonstration data. Our approach employs an iterative process where the agent collects new experiences, receives corrective feedback from humans in the form of hints, and integrates this feedback into its weights. We demonstrate the efficacy of our approach by implementing it in a Llama-3-based agent which, after only a few rounds of feedback, outperforms advanced models GPT-4o and DeepSeek-V3 in a taskset.
  arXiv  Detail & Related papers  (2025-02-03T17:45:46Z)
• Data Science In Olfaction [1.4499463058550683]
  We conceptualize smell from a Data Science and AI perspective, that relates the properties of odorants to how they are sensed and analyzed in the olfactory system from the nose to the brain. Drawing distinctions to color vision, we argue that smell presents unique measurement challenges, including the complexity of stimuli, the high dimensionality of the sensory apparatus, as well as what constitutes ground truth. We present results using machine learning-based classification of neural responses to odors as they are recorded in the mouse olfactory bulb with calcium imaging.
  arXiv  Detail & Related papers  (2024-04-08T13:25:02Z)
• LookALike: Human Mimicry based collaborative decision making [2.9419583394098425]
  General Intelligence falls short when communicating role specific nuances to other systems. We propose and evaluate a novel method that leads to knowledge distillation among LLM agents. We also evaluate how our system performs better in simulated real world tasks compared to state of the art.
  arXiv  Detail & Related papers  (2024-03-16T06:25:53Z)
• Generative agents in the streets: Exploring the use of Large Language Models (LLMs) in collecting urban perceptions [0.0]
  This study explores the current advancements in Generative agents powered by large language models (LLMs) The experiment employs Generative agents to interact with the urban environments using street view images to plan their journey toward specific goals. Since LLMs do not possess embodiment, nor have access to the visual realm, and lack a sense of motion or direction, we designed movement and visual modules that help agents gain an overall understanding of surroundings.
  arXiv  Detail & Related papers  (2023-12-20T15:45:54Z)
• EgoTaskQA: Understanding Human Tasks in Egocentric Videos [89.9573084127155]
  EgoTaskQA benchmark provides home for crucial dimensions of task understanding through question-answering on real-world egocentric videos. We meticulously design questions that target the understanding of (1) action dependencies and effects, (2) intents and goals, and (3) agents' beliefs about others. We evaluate state-of-the-art video reasoning models on our benchmark and show their significant gaps between humans in understanding complex goal-oriented egocentric videos.
  arXiv  Detail & Related papers  (2022-10-08T05:49:05Z)
• Imagination-Augmented Natural Language Understanding [71.51687221130925]
  We introduce an Imagination-Augmented Cross-modal (iACE) to solve natural language understanding tasks. iACE enables visual imagination with external knowledge transferred from the powerful generative and pre-trained vision-and-language models. Experiments on GLUE and SWAG show that iACE achieves consistent improvement over visually-supervised pre-trained models.
  arXiv  Detail & Related papers  (2022-04-18T19:39:36Z)
• From Psychological Curiosity to Artificial Curiosity: Curiosity-Driven Learning in Artificial Intelligence Tasks [56.20123080771364]
  Psychological curiosity plays a significant role in human intelligence to enhance learning through exploration and information acquisition. In the Artificial Intelligence (AI) community, artificial curiosity provides a natural intrinsic motivation for efficient learning. CDL has become increasingly popular, where agents are self-motivated to learn novel knowledge.
  arXiv  Detail & Related papers  (2022-01-20T17:07:03Z)
• Building Human-like Communicative Intelligence: A Grounded Perspective [1.0152838128195465]
  After making astounding progress in language learning, AI systems seem to approach the ceiling that does not reflect important aspects of human communicative capacities. This paper suggests that the dominant cognitively-inspired AI directions, based on nativist and symbolic paradigms, lack necessary substantiation and concreteness to guide progress in modern AI. I propose a list of concrete, implementable components for building "grounded" linguistic intelligence.
  arXiv  Detail & Related papers  (2022-01-02T01:43:24Z)
• Exploring the Sensory Spaces of English Perceptual Verbs in Natural Language Data [0.40611352512781856]
  We focus on the most frequent perception verbs of English analyzed from an and Agentive vs. Experiential distinction. In this study we report on a data-driven approach based on distributional-semantic word embeddings and clustering models.
  arXiv  Detail & Related papers  (2021-10-19T03:58:44Z)
• The MineRL BASALT Competition on Learning from Human Feedback [58.17897225617566]
  The MineRL BASALT competition aims to spur forward research on this important class of techniques. We design a suite of four tasks in Minecraft for which we expect it will be hard to write down hardcoded reward functions. We provide a dataset of human demonstrations on each of the four tasks, as well as an imitation learning baseline.
  arXiv  Detail & Related papers  (2021-07-05T12:18:17Z)
• Bongard-LOGO: A New Benchmark for Human-Level Concept Learning and Reasoning [78.13740873213223]
  Bongard problems (BPs) were introduced as an inspirational challenge for visual cognition in intelligent systems. We propose a new benchmark Bongard-LOGO for human-level concept learning and reasoning.
  arXiv  Detail & Related papers  (2020-10-02T03:19:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.