Exp-Graph: How Connections Learn Facial Attributes in Graph-based Expression Recognition

• URL: http://arxiv.org/abs/2507.14608v1
• Date: Sat, 19 Jul 2025 13:10:21 GMT
• Title: Exp-Graph: How Connections Learn Facial Attributes in Graph-based Expression Recognition
• Authors: Nandani Sharma, Dinesh Singh,
• Abstract summary: We propose Exp-Graph, a novel framework to represent the structural relationships among facial attributes.<n>For facial attributes graph representation, facial landmarks are used as the graph's vertices.<n>The edges are determined based on the proximity of the facial landmark and the similarity of the local appearance of the facial attributes encoded using the vision transformer.
• Score: 0.6042269506496204
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Facial expression recognition is crucial for human-computer interaction applications such as face animation, video surveillance, affective computing, medical analysis, etc. Since the structure of facial attributes varies with facial expressions, incorporating structural information into facial attributes is essential for facial expression recognition. In this paper, we propose Exp-Graph, a novel framework designed to represent the structural relationships among facial attributes using graph-based modeling for facial expression recognition. For facial attributes graph representation, facial landmarks are used as the graph's vertices. At the same time, the edges are determined based on the proximity of the facial landmark and the similarity of the local appearance of the facial attributes encoded using the vision transformer. Additionally, graph convolutional networks are utilized to capture and integrate these structural dependencies into the encoding of facial attributes, thereby enhancing the accuracy of expression recognition. Thus, Exp-Graph learns from the facial attribute graphs highly expressive semantic representations. On the other hand, the vision transformer and graph convolutional blocks help the framework exploit the local and global dependencies among the facial attributes that are essential for the recognition of facial expressions. We conducted comprehensive evaluations of the proposed Exp-Graph model on three benchmark datasets: Oulu-CASIA, eNTERFACE05, and AFEW. The model achieved recognition accuracies of 98.09\%, 79.01\%, and 56.39\%, respectively. These results indicate that Exp-Graph maintains strong generalization capabilities across both controlled laboratory settings and real-world, unconstrained environments, underscoring its effectiveness for practical facial expression recognition applications.

Related papers

• The Importance of Facial Features in Vision-based Sign Language Recognition: Eyes, Mouth or Full Face? [5.92177182003275]
  We show that the mouth is the most important non-manual facial feature, significantly improving accuracy.<n>Our findings highlight the necessity of incorporating facial features in automatic sign language recognition.
  arXiv  Detail & Related papers  (2025-07-28T14:36:46Z)
• Facial Landmark Visualization and Emotion Recognition Through Neural Networks [0.0]
  Emotion recognition from facial images is a crucial task in human-computer interaction.<n>Previous studies have shown that facial images can be used to train deep learning models.<n>We propose facial landmark box plots, a visualization technique designed to identify outliers in facial datasets.
  arXiv  Detail & Related papers  (2025-06-20T17:45:34Z)
• Face-LLaVA: Facial Expression and Attribute Understanding through Instruction Tuning [5.178801281905521]
  We propose Face-LLaVA, a large language model for face-centered, in-context learning, including facial expression and attribute recognition.<n>We first developed FaceInstruct-1M, a face-centered database for instruction tuning MLLMs for face processing.<n>We then developed a novel face-specific visual encoder powered by Face-Region Guided Cross-Attention.
  arXiv  Detail & Related papers  (2025-04-09T18:26:07Z)
• CIAO! A Contrastive Adaptation Mechanism for Non-Universal Facial Expression Recognition [80.07590100872548]
  We propose Contrastive Inhibitory Adaptati On (CIAO), a mechanism that adapts the last layer of facial encoders to depict specific affective characteristics on different datasets. CIAO presents an improvement in facial expression recognition performance over six different datasets with very unique affective representations.
  arXiv  Detail & Related papers  (2022-08-10T15:46:05Z)
• Emotion Separation and Recognition from a Facial Expression by Generating the Poker Face with Vision Transformers [57.1091606948826]
  We propose a novel FER model, named Poker Face Vision Transformer or PF-ViT, to address these challenges. PF-ViT aims to separate and recognize the disturbance-agnostic emotion from a static facial image via generating its corresponding poker face. PF-ViT utilizes vanilla Vision Transformers, and its components are pre-trained as Masked Autoencoders on a large facial expression dataset.
  arXiv  Detail & Related papers  (2022-07-22T13:39:06Z)
• Face-GCN: A Graph Convolutional Network for 3D Dynamic Face Identification/Recognition [21.116748155592752]
  We propose a novel framework for dynamic 3D face identification/recognition based on facial keypoints. Each dynamic sequence of facial expressions is represented as a-temporal graph, which is constructed using 3D facial landmarks. We evaluate our approach on a challenging dynamic 3D facial expression dataset.
  arXiv  Detail & Related papers  (2021-04-19T09:05:39Z)
• Adaptive Graph Representation Learning and Reasoning for Face Parsing [55.086151726427104]
  Face parsing infers a pixel-wise label to each facial component. Component-wise relationship is a critical clue in discriminating ambiguous pixels in facial area. We propose adaptive graph representation learning and reasoning over facial components.
  arXiv  Detail & Related papers  (2021-01-18T12:17:40Z)
• Human Expression Recognition using Facial Shape Based Fourier Descriptors Fusion [15.063379178217717]
  This paper aims to produce a new facial expression recognition method based on the changes in the facial muscles. The geometric features are used to specify the facial regions i.e., mouth, eyes, and nose. A multi-class support vector machine is applied for classification of seven human expression.
  arXiv  Detail & Related papers  (2020-12-28T05:01:44Z)
• Facial Expressions as a Vulnerability in Face Recognition [73.85525896663371]
  This work explores facial expression bias as a security vulnerability of face recognition systems. We present a comprehensive analysis of how facial expression bias impacts the performance of face recognition technologies.
  arXiv  Detail & Related papers  (2020-11-17T18:12:41Z)
• Edge-aware Graph Representation Learning and Reasoning for Face Parsing [61.5045850197694]
  Face parsing infers a pixel-wise label to each facial component, which has drawn much attention recently. Previous methods have shown their efficiency in face parsing, which however overlook the correlation among different face regions. We propose to model and reason the region-wise relations by learning graph representations.
  arXiv  Detail & Related papers  (2020-07-22T07:46:34Z)
• Structured Landmark Detection via Topology-Adapting Deep Graph Learning [75.20602712947016]
  We present a new topology-adapting deep graph learning approach for accurate anatomical facial and medical landmark detection. The proposed method constructs graph signals leveraging both local image features and global shape features. Experiments are conducted on three public facial image datasets (WFLW, 300W, and COFW-68) as well as three real-world X-ray medical datasets (Cephalometric (public), Hand and Pelvis)
  arXiv  Detail & Related papers  (2020-04-17T11:55:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.