Quality-Aware Multimodal Biometric Recognition

• URL: http://arxiv.org/abs/2112.05827v1
• Date: Fri, 10 Dec 2021 20:48:55 GMT
• Title: Quality-Aware Multimodal Biometric Recognition
• Abstract summary: We develop a quality-aware framework for fusing representations of input modalities by weighting their importance using quality scores estimated in a weakly-supervised fashion. This framework utilizes two fusion blocks, each represented by a set of quality-aware and aggregation networks. We evaluate the performance by considering three multimodal datasets consisting of face, iris, and fingerprint modalities.
• Score: 30.322429033099688
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present a quality-aware multimodal recognition framework that combines representations from multiple biometric traits with varying quality and number of samples to achieve increased recognition accuracy by extracting complimentary identification information based on the quality of the samples. We develop a quality-aware framework for fusing representations of input modalities by weighting their importance using quality scores estimated in a weakly-supervised fashion. This framework utilizes two fusion blocks, each represented by a set of quality-aware and aggregation networks. In addition to architecture modifications, we propose two task-specific loss functions: multimodal separability loss and multimodal compactness loss. The first loss assures that the representations of modalities for a class have comparable magnitudes to provide a better quality estimation, while the multimodal representations of different classes are distributed to achieve maximum discrimination in the embedding space. The second loss, which is considered to regularize the network weights, improves the generalization performance by regularizing the framework. We evaluate the performance by considering three multimodal datasets consisting of face, iris, and fingerprint modalities. The efficacy of the framework is demonstrated through comparison with the state-of-the-art algorithms. In particular, our framework outperforms the rank- and score-level fusion of modalities of BIOMDATA by more than 30% for true acceptance rate at false acceptance rate of $10^{-4}$.

Related papers

• Two-Stage Multi-Modal Fusion with Adaptive Alignment for Action Quality Assessment [41.72676356491381]
  Action Quality Assessment (AQA) aims to evaluate how well a person performs a movement.<n>Unimodal approaches often struggle to capture subtle cues of movement quality in real-world settings.<n>We propose DualAlign, a two-stage multi-modal fusion framework with adaptive alignment.
  arXiv  Detail & Related papers  (2026-07-08T14:09:13Z)
• Measuring What Matters Beyond Text: Evaluating Multimodal Summaries by Quality, Alignment, and Diversity [12.669087812857533]
  We introduce MM-Eval, a unified evaluation framework that integrates assessments of textual quality, cross-modal alignment, and visual diversity.<n> MM-Eval comprises three components: (1) text quality, measured using OpenFActScore for factual consistency and G-Eval for coherence, relevance, and relevance; (2) image-text relevance, evaluated via an MLLM-as-a-judge approach; and (3) image-set diversity, quantified using Truncated CLIP Entropy.<n>Our analysis reveals a text-dominant hierarchy in this setting, where factual consistency acts as a critical determinant of perceived overall quality
  arXiv  Detail & Related papers  (2026-05-12T07:50:55Z)
• PIDNet: Progressive Implicit Decouple Network for Multimodal Action Quality Assessment [14.92667427129547]
  Action quality assessment (AQA) aims to automatically quantify the execution quality of human actions in videos and is valuable for applications such as competitive sports judging.<n>We propose a progressive implicit decoupling and fusion network (PIDNet) that integrates modality-specific information, cross-modal complementary cues, and global quality semantics for accurate assessment.<n>Experiments on the Rhythmic Gymnastics and Fis-V datasets show that PIDNet achieves highly competitive score correlation with favorable error control compared with existing unimodal and multimodal methods.
  arXiv  Detail & Related papers  (2026-05-09T13:33:16Z)
• Multi-Granularity Reasoning for Image Quality Assessment via Attribute-Aware Reinforcement Learning to Rank [12.385224959925523]
  We propose MG-IQA (Multi-Granularity IQA), a multi-granularity reasoning framework that extends RL2R to jointly assess overall image quality and fine-grained quality attributes.<n>Our approach introduces three key innovations: (1) an attribute-aware prompting strategy that elicits structured multi-attribute reasoning from VLMs; (2) a multi-dimensional Thurstone reward model that computes attribute-specific fidelity rewards for group relative policy optimization; and (3) a cross-domain alignment mechanism that enables stable joint training across synthetic distortion, authentic distortion, and AI-generated image datasets without perceptual scale re-alignment.
  arXiv  Detail & Related papers  (2026-04-07T16:07:55Z)
• Complementarity-Supervised Spectral-Band Routing for Multimodal Emotion Recognition [60.20529806857076]
  Multimodal emotion recognition fuses cues such as text, video, and audio to understand individual emotional states.<n>Prior methods face two main limitations: mechanically relying on independent unimodal performance, and coarse-grained fusion conflicting with the fine-grained representations required by emotion tasks.<n>We propose the Complementarity-Supervised Multi-Band Expert Network, named Atsuko, to model fine-grained complementary features via multi-scale band decomposition and expert collaboration.
  arXiv  Detail & Related papers  (2026-03-07T03:58:48Z)
• Addressing Missing and Noisy Modalities in One Solution: Unified Modality-Quality Framework for Low-quality Multimodal Data [15.181276098822737]
  We propose a unified modality-quality (UMQ) framework to enhance low-quality representations for multimodal affective computing.<n>UMQ consistently outperforms state-of-the-art baselines on multiple datasets under the settings of complete, missing, and noisy modalities.
  arXiv  Detail & Related papers  (2026-03-03T07:38:13Z)
• Detecting Deepfakes with Multivariate Soft Blending and CLIP-based Image-Text Alignment [4.34685509565816]
  The proliferation of highly realistic facial forgeries necessitates robust detection methods.<n>Existing approaches often suffer from limited accuracy and poor generalization due to significant distribution shifts among samples generated by diverse forgery techniques.<n>Our method leverages the multimodal alignment capabilities of CLIP to capture subtle forgery traces.
  arXiv  Detail & Related papers  (2026-02-14T09:53:35Z)
• Implicit Neural Representation-Based Continuous Single Image Super Resolution: An Empirical Study [50.15623093332659]
  Implicit neural representation (INR) has become the standard approach for arbitrary-scale image super-resolution (ASSR)<n>We compare existing techniques across diverse settings and present aggregated performance results on multiple image quality metrics.<n>We examine a new loss function that penalizes intensity variations while preserving edges, textures, and finer details during training.
  arXiv  Detail & Related papers  (2026-01-25T07:09:20Z)
• A Quality-Guided Mixture of Score-Fusion Experts Framework for Human Recognition [14.242680363313148]
  Whole-body biometric recognition is a challenging task that integrates various biometric modalities.<n>We present textbfQuality-guided textbfMixture of score-fusion textbfExperts (QME)<n>We introduce a novel pseudo-quality loss for quality estimation with a modality-specific Quality Estimator (QE) and a score triplet loss to improve the metric performance.
  arXiv  Detail & Related papers  (2025-07-31T18:00:01Z)
• M3-AGIQA: Multimodal, Multi-Round, Multi-Aspect AI-Generated Image Quality Assessment [65.3860007085689]
  M3-AGIQA is a comprehensive framework for AGI quality assessment. It includes a structured multi-round evaluation mechanism, where intermediate image descriptions are generated. Experiments conducted on multiple benchmark datasets demonstrate that M3-AGIQA achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-02-21T03:05:45Z)
• Multi-QuAD: Multi-Level Quality-Adaptive Dynamic Network for Reliable Multimodal Classification [57.08108545219043]
  Existing reliable multimodal classification methods fail to provide robust estimation of data quality.<n>New framework for reliable classification termed textitMulti-level Quality-Adaptive Dynamic multimodal network (Multi-QuAD) is proposed.<n>Experiments conducted on four datasets demonstrate that Multi-QuAD significantly outperforms state-of-the-art methods in classification performance and reliability.
  arXiv  Detail & Related papers  (2024-12-19T03:26:51Z)
• Multi-task Feature Enhancement Network for No-Reference Image Quality Assessment [4.4150617622399055]
  Multi-task strategies based No-Reference Image Quality Assessment (NR-IQA) methods encounter several challenges. Our framework consists of three key components: a high-frequency extraction network, a quality estimation network, and a distortion-aware network. Empirical results from five standard IQA databases confirm that our method achieves high performance and also exhibits robust generalization ability.
  arXiv  Detail & Related papers  (2024-11-12T05:10:32Z)
• No-Reference Image Quality Assessment with Global-Local Progressive Integration and Semantic-Aligned Quality Transfer [6.095342999639137]
  We develop a dual-measurement framework that combines vision Transformer (ViT)-based global feature extractor and convolutional neural networks (CNNs)-based local feature extractor. We introduce a semantic-aligned quality transfer method that extends the training data by automatically labeling the quality scores of diverse image content with subjective opinion scores.
  arXiv  Detail & Related papers  (2024-08-07T16:34:32Z)
• Q-Ground: Image Quality Grounding with Large Multi-modality Models [61.72022069880346]
  We introduce Q-Ground, the first framework aimed at tackling fine-scale visual quality grounding. Q-Ground combines large multi-modality models with detailed visual quality analysis. Central to our contribution is the introduction of the QGround-100K dataset.
  arXiv  Detail & Related papers  (2024-07-24T06:42:46Z)
• Adaptive Image Quality Assessment via Teaching Large Multimodal Model to Compare [99.57567498494448]
  We introduce Compare2Score, an all-around LMM-based no-reference IQA model. During training, we generate scaled-up comparative instructions by comparing images from the same IQA dataset. Experiments on nine IQA datasets validate that the Compare2Score effectively bridges text-defined comparative levels during training.
  arXiv  Detail & Related papers  (2024-05-29T17:26:09Z)
• How Quality Affects Deep Neural Networks in Fine-Grained Image Classification [0.799543372823325]
  We propose a No-Reference Image Quality Assessment (NRIQA) guided cut-off point selection (CPS) strategy to enhance the performance of a fine-grained classification system. We take the three most commonly adopted image augmentation configurations -- cropping, rotating, and blurring -- as the entry point. Concretely, the cut-off points yielded by those methods are aggregated via majority voting to inform the process of image subset selection.
  arXiv  Detail & Related papers  (2024-05-09T12:59:11Z)
• Multi-Modal Prompt Learning on Blind Image Quality Assessment [65.0676908930946]
  Image Quality Assessment (IQA) models benefit significantly from semantic information, which allows them to treat different types of objects distinctly. Traditional methods, hindered by a lack of sufficiently annotated data, have employed the CLIP image-text pretraining model as their backbone to gain semantic awareness. Recent approaches have attempted to address this mismatch using prompt technology, but these solutions have shortcomings. This paper introduces an innovative multi-modal prompt-based methodology for IQA.
  arXiv  Detail & Related papers  (2024-04-23T11:45:32Z)
• Context-Semantic Quality Awareness Network for Fine-Grained Visual Categorization [30.92656780805478]
  We propose a weakly supervised Context-Semantic Quality Awareness Network (CSQA-Net) for fine-grained visual categorization (FGVC) To model the spatial contextual relationship between rich part descriptors and global semantics, we develop a novel multi-part and multi-scale cross-attention (MPMSCA) module. We also propose a generic multi-level semantic quality evaluation module (MLSQE) to progressively supervise and enhance hierarchical semantics from different levels of the backbone network.
  arXiv  Detail & Related papers  (2024-03-15T13:40:44Z)
• Blind Image Quality Assessment via Vision-Language Correspondence: A Multitask Learning Perspective [93.56647950778357]
  Blind image quality assessment (BIQA) predicts the human perception of image quality without any reference information. We develop a general and automated multitask learning scheme for BIQA to exploit auxiliary knowledge from other tasks.
  arXiv  Detail & Related papers  (2023-03-27T07:58:09Z)
• Evaluating and Improving Factuality in Multimodal Abstractive Summarization [91.46015013816083]
  We propose CLIPBERTScore to leverage the robustness and strong factuality detection performance between image-summary and document-summary. We show that this simple combination of two metrics in the zero-shot achieves higher correlations than existing factuality metrics for document summarization. Our analysis demonstrates the robustness and high correlation of CLIPBERTScore and its components on four factuality metric-evaluation benchmarks.
  arXiv  Detail & Related papers  (2022-11-04T16:50:40Z)
• Iterative Optimization of Pseudo Ground-Truth Face Image Quality Labels [0.0]
  Face image quality assessment (FIQA) techniques provide sample quality information that can be used to reject poor quality data. We propose a quality label optimization approach, which incorporates sample-quality information from mated-pair similarities into quality predictions. We evaluate the proposed approach using three state-of-the-art FIQA methods over three diverse datasets.
  arXiv  Detail & Related papers  (2022-08-31T08:24:09Z)
• Trusted Multi-View Classification [76.73585034192894]
  We propose a novel multi-view classification method, termed trusted multi-view classification. It provides a new paradigm for multi-view learning by dynamically integrating different views at an evidence level. The proposed algorithm jointly utilizes multiple views to promote both classification reliability and robustness.
  arXiv  Detail & Related papers  (2021-02-03T13:30:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.