Categorizing the Visual Environment and Analyzing the Visual Attention of Dogs

• URL: http://arxiv.org/abs/2311.11988v1
• Date: Mon, 20 Nov 2023 18:21:18 GMT
• Title: Categorizing the Visual Environment and Analyzing the Visual Attention of Dogs
• Authors: Shreyas Sundara Raman, Madeline H. Pelgrim, Daphna Buchsbaum and Thomas Serre
• Abstract summary: We collect and study a dataset with over 11,698 gazes to categorize the objects available to be gazed at by 11 dogs. A small portion of the collected data is used to fine tune a MaskRCNN for the novel image domain. There are few individual differences between the 11 dogs and we observe greater visual fixations on buses, plants, pavement, and construction equipment.
• Score: 11.511035466613109
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Dogs have a unique evolutionary relationship with humans and serve many important roles e.g. search and rescue, blind assistance, emotional support. However, few datasets exist to categorize visual features and objects available to dogs, as well as how dogs direct their visual attention within their environment. We collect and study a dataset with over 11,698 gazes to categorize the objects available to be gazed at by 11 dogs in everyday outdoor environments i.e. a walk around a college campus and urban area. We explore the availability of these object categories and the visual attention of dogs over these categories using a head mounted eye tracking apparatus. A small portion (approx. 600 images or < 20% of total dataset) of the collected data is used to fine tune a MaskRCNN for the novel image domain to segment objects present in the scene, enabling further statistical analysis on the visual gaze tendencies of dogs. The MaskRCNN, with eye tracking apparatus, serves as an end to end model for automatically classifying the visual fixations of dogs. The fine tuned MaskRCNN performs far better than chance. There are few individual differences between the 11 dogs and we observe greater visual fixations on buses, plants, pavement, and construction equipment. This work takes a step towards understanding visual behavior of dogs and their interaction with the physical world.

Related papers

• DogFLW: Dog Facial Landmarks in the Wild Dataset [5.51724397504775]
  We develop an analogous dataset containing 3,274 annotated images of dogs. Our dataset is based on a scheme of 46 facial anatomy-based landmarks.
  arXiv  Detail & Related papers  (2024-05-19T09:59:36Z)
• Computer Vision for Primate Behavior Analysis in the Wild [61.08941894580172]
  Video-based behavioral monitoring has great potential for transforming how we study animal cognition and behavior. There is still a fairly large gap between the exciting prospects and what can actually be achieved in practice today.
  arXiv  Detail & Related papers  (2024-01-29T18:59:56Z)
• Visual attention information can be traced on cortical response but not on the retina: evidence from electrophysiological mouse data using natural images as stimuli [0.0]
  In primary visual cortex (V1), a subset of around $10%$ of the neurons responds differently to salient versus non-salient visual regions. It appears that the retina remains naive concerning visual attention; cortical response gets to interpret visual attention information.
  arXiv  Detail & Related papers  (2023-08-01T13:09:48Z)
• BI AVAN: Brain inspired Adversarial Visual Attention Network [67.05560966998559]
  We propose a brain-inspired adversarial visual attention network (BI-AVAN) to characterize human visual attention directly from functional brain activity. Our model imitates the biased competition process between attention-related/neglected objects to identify and locate the visual objects in a movie frame the human brain focuses on in an unsupervised manner.
  arXiv  Detail & Related papers  (2022-10-27T22:20:36Z)
• How Would The Viewer Feel? Estimating Wellbeing From Video Scenarios [73.24092762346095]
  We introduce two large-scale datasets with over 60,000 videos annotated for emotional response and subjective wellbeing. The Video Cognitive Empathy dataset contains annotations for distributions of fine-grained emotional responses, allowing models to gain a detailed understanding of affective states. The Video to Valence dataset contains annotations of relative pleasantness between videos, which enables predicting a continuous spectrum of wellbeing.
  arXiv  Detail & Related papers  (2022-10-18T17:58:25Z)
• Deep Learning Models for Automated Classification of Dog Emotional States from Facial Expressions [1.32383730641561]
  We apply recent deep learning techniques to classify (positive) anticipation and (negative) frustration of dogs. To the best of our knowledge, this work is the first to address the task of automatic classification of canine emotions.
  arXiv  Detail & Related papers  (2022-06-11T21:37:38Z)
• What Can You Learn from Your Muscles? Learning Visual Representation from Human Interactions [50.435861435121915]
  We use human interaction and attention cues to investigate whether we can learn better representations compared to visual-only representations. Our experiments show that our "muscly-supervised" representation outperforms a visual-only state-of-the-art method MoCo.
  arXiv  Detail & Related papers  (2020-10-16T17:46:53Z)
• Visual Chirality [51.685596116645776]
  We investigate how statistics of visual data are changed by reflection. Our work has implications for data augmentation, self-supervised learning, and image forensics.
  arXiv  Detail & Related papers  (2020-06-16T20:48:23Z)
• VisualEchoes: Spatial Image Representation Learning through Echolocation [97.23789910400387]
  Several animal species (e.g., bats, dolphins, and whales) and even visually impaired humans have the remarkable ability to perform echolocation. We propose a novel interaction-based representation learning framework that learns useful visual features via echolocation. Our work opens a new path for representation learning for embodied agents, where supervision comes from interacting with the physical world.
  arXiv  Detail & Related papers  (2020-05-04T16:16:58Z)
• Identifying Individual Dogs in Social Media Images [1.14219428942199]
  The work described here is part of joint project done with Pet2Net, a social network focused on pets and their owners. In order to detect and recognize individual dogs we combine transfer learning and object detection approaches on Inception v3 and SSD Inception v2 architectures. We show that it can achieve 94.59% accuracy in identifying individual dogs.
  arXiv  Detail & Related papers  (2020-03-14T21:11:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.