Person Recognition at Altitude and Range: Fusion of Face, Body Shape and Gait

• URL: http://arxiv.org/abs/2505.04616v1
• Date: Wed, 07 May 2025 17:58:25 GMT
• Title: Person Recognition at Altitude and Range: Fusion of Face, Body Shape and Gait
• Authors: Feng Liu, Nicholas Chimitt, Lanqing Guo, Jitesh Jain, Aditya Kane, Minchul Kim, Wes Robbins, Yiyang Su, Dingqiang Ye, Xingguang Zhang, Jie Zhu, Siddharth Satyakam, Christopher Perry, Stanley H. Chan, Arun Ross, Humphrey Shi, Zhangyang Wang, Anil Jain, Xiaoming Liu,
• Abstract summary: FarSight is an end-to-end system for person recognition that integrates biometric cues across face, gait, and body shape modalities.<n>FarSight incorporates novel algorithms across four core modules: multi-subject detection and tracking, recognition-aware video restoration, modality-specific biometric feature encoding, and quality-guided multi-modal fusion.
• Score: 70.00430652562012
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We address the problem of whole-body person recognition in unconstrained environments. This problem arises in surveillance scenarios such as those in the IARPA Biometric Recognition and Identification at Altitude and Range (BRIAR) program, where biometric data is captured at long standoff distances, elevated viewing angles, and under adverse atmospheric conditions (e.g., turbulence and high wind velocity). To this end, we propose FarSight, a unified end-to-end system for person recognition that integrates complementary biometric cues across face, gait, and body shape modalities. FarSight incorporates novel algorithms across four core modules: multi-subject detection and tracking, recognition-aware video restoration, modality-specific biometric feature encoding, and quality-guided multi-modal fusion. These components are designed to work cohesively under degraded image conditions, large pose and scale variations, and cross-domain gaps. Extensive experiments on the BRIAR dataset, one of the most comprehensive benchmarks for long-range, multi-modal biometric recognition, demonstrate the effectiveness of FarSight. Compared to our preliminary system, this system achieves a 34.1% absolute gain in 1:1 verification accuracy (TAR@0.1% FAR), a 17.8% increase in closed-set identification (Rank-20), and a 34.3% reduction in open-set identification errors (FNIR@1% FPIR). Furthermore, FarSight was evaluated in the 2025 NIST RTE Face in Video Evaluation (FIVE), which conducts standardized face recognition testing on the BRIAR dataset. These results establish FarSight as a state-of-the-art solution for operational biometric recognition in challenging real-world conditions.

Related papers

• Exploiting Multiple Representations: 3D Face Biometrics Fusion with Application to Surveillance [6.277064632667653]
  3D face reconstruction (3DFR) algorithms are based on specific assumptions tailored to the limits and characteristics of the different application scenarios.<n>In this study, we investigate how multiple state-of-the-art 3DFR algorithms can be used to generate a better representation of subjects.<n>We also explore how different parametric and non-parametric score-level fusion methods can exploit the unique strengths of multiple 3DFR algorithms to enhance biometric recognition robustness.
  arXiv  Detail & Related papers  (2025-04-26T10:21:46Z)
• Long-Range Biometric Identification in Real World Scenarios: A Comprehensive Evaluation Framework Based on Missions [11.557368031775717]
  This paper evaluates research solutions for identifying individuals at ranges and altitudes. By fusing face and body features, we propose developing robust biometric systems for effective long-range identification.
  arXiv  Detail & Related papers  (2024-09-03T02:17:36Z)
• IdentiFace : A VGG Based Multimodal Facial Biometric System [0.0]
  "IdentiFace" is a multimodal facial biometric system that combines the core of facial recognition with some of the most important soft biometric traits such as gender, face shape, and emotion. For the recognition problem, we acquired a 99.2% test accuracy for five classes with high intra-class variations using data collected from the FERET database. We were also able to achieve a testing accuracy of 88.03% in the face-shape problem using the celebrity face-shape dataset.
  arXiv  Detail & Related papers  (2024-01-02T14:36:28Z)
• Whole-body Detection, Recognition and Identification at Altitude and Range [57.445372305202405]
  We propose an end-to-end system evaluated on diverse datasets. Our approach involves pre-training the detector on common image datasets and fine-tuning it on BRIAR's complex videos and images. We conduct thorough evaluations under various conditions, such as different ranges and angles in indoor, outdoor, and aerial scenarios.
  arXiv  Detail & Related papers  (2023-11-09T20:20:23Z)
• FarSight: A Physics-Driven Whole-Body Biometric System at Large Distance and Altitude [67.55994773068191]
  This paper presents the end-to-end design, development and evaluation of FarSight. FarSight is an innovative software system designed for whole-body (fusion of face, gait and body shape) biometric recognition. We test FarSight's effectiveness using the newly acquired IARPA Biometric Recognition and Identification at Altitude and Range dataset.
  arXiv  Detail & Related papers  (2023-06-29T16:14:27Z)
• Benchmarking Quality-Dependent and Cost-Sensitive Score-Level Multimodal Biometric Fusion Algorithms [58.156733807470395]
  This paper reports a benchmarking study carried out within the framework of the BioSecure DS2 (Access Control) evaluation campaign. The campaign targeted the application of physical access control in a medium-size establishment with some 500 persons. To the best of our knowledge, this is the first attempt to benchmark quality-based multimodal fusion algorithms.
  arXiv  Detail & Related papers  (2021-11-17T13:39:48Z)
• A Synthesis-Based Approach for Thermal-to-Visible Face Verification [105.63410428506536]
  This paper presents an algorithm that achieves state-of-the-art performance on the ARL-VTF and TUFTS multi-spectral face datasets. We also present MILAB-VTF(B), a challenging multi-spectral face dataset composed of paired thermal and visible videos.
  arXiv  Detail & Related papers  (2021-08-21T17:59:56Z)
• Biometrics: Trust, but Verify [49.9641823975828]
  Biometric recognition has exploded into a plethora of different applications around the globe. There are a number of outstanding problems and concerns pertaining to the various sub-modules of biometric recognition systems.
  arXiv  Detail & Related papers  (2021-05-14T03:07:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.