Iconic Gesture Semantics

• URL: http://arxiv.org/abs/2404.18708v1
• Date: Mon, 29 Apr 2024 13:58:03 GMT
• Title: Iconic Gesture Semantics
• Authors: Andy Lücking, Alexander Henlein, Alexander Mehler,
• Abstract summary: Informational evaluation is spelled out as extended exemplification (extemplification) in terms of perceptual classification of a gesture's visual iconic model. We argue that the perceptual classification of instances of visual communication requires a notion of meaning different from Frege/Montague frameworks. An iconic gesture semantics is introduced which covers the full range from gesture representations over model-theoretic evaluation to inferential interpretation in dynamic semantic frameworks.
• Score: 87.00251241246136
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: The "meaning" of an iconic gesture is conditioned on its informational evaluation. Only informational evaluation lifts a gesture to a quasi-linguistic level that can interact with verbal content. Interaction is either vacuous or regimented by usual lexicon-driven inferences. Informational evaluation is spelled out as extended exemplification (extemplification) in terms of perceptual classification of a gesture's visual iconic model. The iconic model is derived from Frege/Montague-like truth-functional evaluation of a gesture's form within spatially extended domains. We further argue that the perceptual classification of instances of visual communication requires a notion of meaning different from Frege/Montague frameworks. Therefore, a heuristic for gesture interpretation is provided that can guide the working semanticist. In sum, an iconic gesture semantics is introduced which covers the full range from kinematic gesture representations over model-theoretic evaluation to inferential interpretation in dynamic semantic frameworks.

Related papers

• Integrating Representational Gestures into Automatically Generated Embodied Explanations and its Effects on Understanding and Interaction Quality [0.0]
  This study investigates how different types of gestures influence perceived interaction quality and listener understanding. Our model combines beat gestures generated by a learned speech-driven module with manually captured iconic gestures. Findings indicate that neither the use of iconic gestures alone nor their combination with beat gestures outperforms the baseline or beat-only conditions in terms of understanding.
  arXiv  Detail & Related papers  (2024-06-18T12:23:00Z)
• Semantic Gesticulator: Semantics-Aware Co-Speech Gesture Synthesis [25.822870767380685]
  We present Semantic Gesticulator, a framework designed to synthesize realistic gestures with strong semantic correspondence. Our system demonstrates robustness in generating gestures that are rhythmically coherent and semantically explicit. Our system outperforms state-of-the-art systems in terms of semantic appropriateness by a clear margin.
  arXiv  Detail & Related papers  (2024-05-16T05:09:01Z)
• Neural Semantic Parsing with Extremely Rich Symbolic Meaning Representations [7.774674200374255]
  We introduce a novel compositional symbolic representation for concepts based on their position in the taxonomical hierarchy. This representation provides richer semantic information and enhances interpretability. Our experimental findings demonstrate that the taxonomical model, trained on much richer and complex meaning representations, is slightly subordinate in performance to the traditional model using the standard metrics for evaluation, but outperforms it when dealing with out-of-vocabulary concepts.
  arXiv  Detail & Related papers  (2024-04-19T08:06:01Z)
• Semantics-aware Motion Retargeting with Vision-Language Models [19.53696208117539]
  We present a novel Semantics-aware Motion reTargeting (SMT) method with the advantage of vision-language models to extract and maintain meaningful motion semantics. We utilize a differentiable module to render 3D motions and the high-level motion semantics are incorporated into the motion process by feeding the vision-language model and aligning the extracted semantic embeddings. To ensure the preservation of fine-grained motion details and high-level semantics, we adopt two-stage pipeline consisting of skeleton-aware pre-training and fine-tuning with semantics and geometry constraints.
  arXiv  Detail & Related papers  (2023-12-04T15:23:49Z)
• ALADIN-NST: Self-supervised disentangled representation learning of artistic style through Neural Style Transfer [60.6863849241972]
  We learn a representation of visual artistic style more strongly disentangled from the semantic content depicted in an image. We show that strongly addressing the disentanglement of style and content leads to large gains in style-specific metrics.
  arXiv  Detail & Related papers  (2023-04-12T10:33:18Z)
• Variational Cross-Graph Reasoning and Adaptive Structured Semantics Learning for Compositional Temporal Grounding [143.5927158318524]
  Temporal grounding is the task of locating a specific segment from an untrimmed video according to a query sentence. We introduce a new Compositional Temporal Grounding task and construct two new dataset splits. We argue that the inherent structured semantics inside the videos and language is the crucial factor to achieve compositional generalization.
  arXiv  Detail & Related papers  (2023-01-22T08:02:23Z)
• Visual Superordinate Abstraction for Robust Concept Learning [80.15940996821541]
  Concept learning constructs visual representations that are connected to linguistic semantics. We ascribe the bottleneck to a failure of exploring the intrinsic semantic hierarchy of visual concepts. We propose a visual superordinate abstraction framework for explicitly modeling semantic-aware visual subspaces.
  arXiv  Detail & Related papers  (2022-05-28T14:27:38Z)
• Cross-modal Representation Learning for Zero-shot Action Recognition [67.57406812235767]
  We present a cross-modal Transformer-based framework, which jointly encodes video data and text labels for zero-shot action recognition (ZSAR) Our model employs a conceptually new pipeline by which visual representations are learned in conjunction with visual-semantic associations in an end-to-end manner. Experiment results show our model considerably improves upon the state of the arts in ZSAR, reaching encouraging top-1 accuracy on UCF101, HMDB51, and ActivityNet benchmark datasets.
  arXiv  Detail & Related papers  (2022-05-03T17:39:27Z)
• Semantic Disentangling Generalized Zero-Shot Learning [50.259058462272435]
  Generalized Zero-Shot Learning (GZSL) aims to recognize images from both seen and unseen categories. In this paper, we propose a novel feature disentangling approach based on an encoder-decoder architecture. The proposed model aims to distill quality semantic-consistent representations that capture intrinsic features of seen images.
  arXiv  Detail & Related papers  (2021-01-20T05:46:21Z)
• A Neural Network Model of Lexical Competition during Infant Spoken Word Recognition [0.0]
  Visual world studies show that upon hearing a word in a target-absent visual context, toddlers and adults briefly direct their gaze towards phonologically related items. We present a neural network model that processes dynamic unfolding phonological representations and maps them to static internal semantic and visual representations.
  arXiv  Detail & Related papers  (2020-06-01T15:04:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.