CILF-CIAE: CLIP-driven Image-Language Fusion for Correcting Inverse Age Estimation

• URL: http://arxiv.org/abs/2312.01758v3
• Date: Sat, 31 Aug 2024 12:56:28 GMT
• Title: CILF-CIAE: CLIP-driven Image-Language Fusion for Correcting Inverse Age Estimation
• Authors: Yuntao Shou, Wei Ai, Tao Meng, Nan Yin, Keqin Li,
• Abstract summary: The age estimation task aims to predict the age of an individual by analyzing facial features in an image. Existing CLIP-based age estimation methods require high memory usage and lack an error feedback mechanism. We propose a novel CLIP-driven Image-Language Fusion for Correcting Inverse Age Estimation (CILF-CIAE)
• Score: 14.639340916340801
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The age estimation task aims to predict the age of an individual by analyzing facial features in an image. The development of age estimation can improve the efficiency and accuracy of various applications (e.g., age verification and secure access control, etc.). In recent years, contrastive language-image pre-training (CLIP) has been widely used in various multimodal tasks and has made some progress in the field of age estimation. However, existing CLIP-based age estimation methods require high memory usage (quadratic complexity) when globally modeling images, and lack an error feedback mechanism to prompt the model about the quality of age prediction results. To tackle the above issues, we propose a novel CLIP-driven Image-Language Fusion for Correcting Inverse Age Estimation (CILF-CIAE). Specifically, we first introduce the CLIP model to extract image features and text semantic information respectively, and map them into a highly semantically aligned high-dimensional feature space. Next, we designed a new Transformer architecture (i.e., FourierFormer) to achieve channel evolution and spatial interaction of images, and to fuse image and text semantic information. Compared with the quadratic complexity of the attention mechanism, the proposed Fourierformer is of linear log complexity. To further narrow the semantic gap between image and text features, we utilize an efficient contrastive multimodal learning module that supervises the multimodal fusion process of FourierFormer through contrastive loss for image-text matching, thereby improving the interaction effect between different modalities. Finally, we introduce reversible age estimation, which uses end-to-end error feedback to reduce the error rate of age predictions. Through extensive experiments on multiple data sets, CILF-CIAE has achieved better age prediction results.

Related papers

• A Multi-view Mask Contrastive Learning Graph Convolutional Neural Network for Age Estimation [13.197551708300345]
  This paper proposes a Multi-view Mask Contrastive Learning Graph Convolutional Neural Network (MMCL-GCN) for age estimation. The overall structure of the MMCL-GCN network contains a feature extraction stage and an age estimation stage. Extensive experiments show that MMCL-GCN can effectively reduce the error of age estimation on benchmark datasets.
  arXiv  Detail & Related papers  (2024-07-23T07:17:46Z)
• Continual Panoptic Perception: Towards Multi-modal Incremental Interpretation of Remote Sensing Images [16.0258685984844]
  Continual learning (CL) breaks off the one-way training manner and enables a model to adapt to new data, semantics and tasks continuously. We propose a unified continual learning model that leverages multi-task joint learning covering pixel-level classification, instance-level segmentation and image-level perception.
  arXiv  Detail & Related papers  (2024-07-19T12:22:32Z)
• Masked Contrastive Graph Representation Learning for Age Estimation [44.96502862249276]
  This paper utilizes the property of graph representation learning in dealing with image redundancy information. We propose a novel Masked Contrastive Graph Representation Learning (MCGRL) method for age estimation. Experimental results on real-world face image datasets demonstrate the superiority of our proposed method over other state-of-the-art age estimation approaches.
  arXiv  Detail & Related papers  (2023-06-16T15:53:21Z)
• Pluralistic Aging Diffusion Autoencoder [63.50599304294062]
  Face aging is an ill-posed problem because multiple plausible aging patterns may correspond to a given input. This paper proposes a novel CLIP-driven Pluralistic Aging Diffusion Autoencoder to enhance the diversity of aging patterns.
  arXiv  Detail & Related papers  (2023-03-20T13:20:14Z)
• Learning to Exploit Temporal Structure for Biomedical Vision-Language Processing [53.89917396428747]
  Self-supervised learning in vision-language processing exploits semantic alignment between imaging and text modalities. We explicitly account for prior images and reports when available during both training and fine-tuning. Our approach, named BioViL-T, uses a CNN-Transformer hybrid multi-image encoder trained jointly with a text model.
  arXiv  Detail & Related papers  (2023-01-11T16:35:33Z)
• Generative Negative Text Replay for Continual Vision-Language Pretraining [95.2784858069843]
  Vision-language pre-training has attracted increasing attention recently. Massive data are usually collected in a streaming fashion. We propose a multi-modal knowledge distillation between images and texts to align the instance-wise prediction between old and new models.
  arXiv  Detail & Related papers  (2022-10-31T13:42:21Z)
• DenseCLIP: Language-Guided Dense Prediction with Context-Aware Prompting [91.56988987393483]
  We present a new framework for dense prediction by implicitly and explicitly leveraging the pre-trained knowledge from CLIP. Specifically, we convert the original image-text matching problem in CLIP to a pixel-text matching problem and use the pixel-text score maps to guide the learning of dense prediction models. Our method is model-agnostic, which can be applied to arbitrary dense prediction systems and various pre-trained visual backbones.
  arXiv  Detail & Related papers  (2021-12-02T18:59:32Z)
• FILIP: Fine-grained Interactive Language-Image Pre-Training [106.19474076935363]
  Fine-grained Interactive Language-Image Pre-training achieves finer-level alignment through a cross-modal late interaction mechanism. We construct a new large-scale image-text pair dataset called FILIP300M for pre-training. Experiments show that FILIP achieves state-of-the-art performance on multiple downstream vision-language tasks.
  arXiv  Detail & Related papers  (2021-11-09T17:15:38Z)
• FP-Age: Leveraging Face Parsing Attention for Facial Age Estimation in the Wild [50.8865921538953]
  We propose a method to explicitly incorporate facial semantics into age estimation. We design a face parsing-based network to learn semantic information at different scales. We show that our method consistently outperforms all existing age estimation methods.
  arXiv  Detail & Related papers  (2021-06-21T14:31:32Z)
• Hierarchical Attention-based Age Estimation and Bias Estimation [16.335191345543063]
  We propose a novel deep-learning approach for age estimation based on face images. Our scheme is shown to outperform contemporary schemes and provide a new state-of-the-art age estimation accuracy.
  arXiv  Detail & Related papers  (2021-03-17T19:41:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.