DisguisOR: Holistic Face Anonymization for the Operating Room
- URL: http://arxiv.org/abs/2307.14241v1
- Date: Wed, 26 Jul 2023 15:10:54 GMT
- Title: DisguisOR: Holistic Face Anonymization for the Operating Room
- Authors: Lennart Bastian, Tony Danjun Wang, Tobias Czempiel, Benjamin Busam and
Nassir Navab
- Abstract summary: Existing automated 2D anonymization methods under-perform in Operating Rooms.
We propose to anonymize multi-view OR recordings using 3D data from multiple camera streams.
- Score: 43.68679886516574
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Purpose: Recent advances in Surgical Data Science (SDS) have contributed to
an increase in video recordings from hospital environments. While methods such
as surgical workflow recognition show potential in increasing the quality of
patient care, the quantity of video data has surpassed the scale at which
images can be manually anonymized. Existing automated 2D anonymization methods
under-perform in Operating Rooms (OR), due to occlusions and obstructions. We
propose to anonymize multi-view OR recordings using 3D data from multiple
camera streams. Methods: RGB and depth images from multiple cameras are fused
into a 3D point cloud representation of the scene. We then detect each
individual's face in 3D by regressing a parametric human mesh model onto
detected 3D human keypoints and aligning the face mesh with the fused 3D point
cloud. The mesh model is rendered into every acquired camera view, replacing
each individual's face. Results: Our method shows promise in locating faces at
a higher rate than existing approaches. DisguisOR produces geometrically
consistent anonymizations for each camera view, enabling more realistic
anonymization that is less detrimental to downstream tasks. Conclusion:
Frequent obstructions and crowding in operating rooms leaves significant room
for improvement for off-the-shelf anonymization methods. DisguisOR addresses
privacy on a scene level and has the potential to facilitate further research
in SDS.
Related papers
- 3D Face Tracking from 2D Video through Iterative Dense UV to Image Flow [15.479024531161476]
We propose a novel face tracker, FlowFace, that introduces an innovative 2D alignment network for dense per-vertex alignment.
Unlike prior work, FlowFace is trained on high-quality 3D scan annotations rather than weak supervision or synthetic data.
Our method exhibits superior performance on both custom and publicly available benchmarks.
arXiv Detail & Related papers (2024-04-15T14:20:07Z) - 3D Holistic OR Anonymization [0.0]
We propose a novel method to automatically anonymize multi-view RGB-D video recordings of operating rooms (OR)
Our anonymization method preserves the original data distribution by replacing the faces in each image with different faces.
In contrast to established anonymization methods, our approach localizes faces in 3D space first rather than in 2D space.
arXiv Detail & Related papers (2024-03-18T23:32:02Z) - FitDiff: Robust monocular 3D facial shape and reflectance estimation using Diffusion Models [79.65289816077629]
We present FitDiff, a diffusion-based 3D facial avatar generative model.
Our model accurately generates relightable facial avatars, utilizing an identity embedding extracted from an "in-the-wild" 2D facial image.
Being the first 3D LDM conditioned on face recognition embeddings, FitDiff reconstructs relightable human avatars, that can be used as-is in common rendering engines.
arXiv Detail & Related papers (2023-12-07T17:35:49Z) - Decaf: Monocular Deformation Capture for Face and Hand Interactions [77.75726740605748]
This paper introduces the first method that allows tracking human hands interacting with human faces in 3D from single monocular RGB videos.
We model hands as articulated objects inducing non-rigid face deformations during an active interaction.
Our method relies on a new hand-face motion and interaction capture dataset with realistic face deformations acquired with a markerless multi-view camera system.
arXiv Detail & Related papers (2023-09-28T17:59:51Z) - Generating 2D and 3D Master Faces for Dictionary Attacks with a
Network-Assisted Latent Space Evolution [68.8204255655161]
A master face is a face image that passes face-based identity authentication for a high percentage of the population.
We optimize these faces for 2D and 3D face verification models.
In 3D, we generate faces using the 2D StyleGAN2 generator and predict a 3D structure using a deep 3D face reconstruction network.
arXiv Detail & Related papers (2022-11-25T09:15:38Z) - 3D Human Pose Estimation in Multi-View Operating Room Videos Using
Differentiable Camera Projections [2.486571221735935]
We propose to directly optimise for localisation in 3D by training 2D CNNs end-to-end based on a 3D loss.
Using videos from the MVOR dataset, we show that this end-to-end approach outperforms optimisation in 2D space.
arXiv Detail & Related papers (2022-10-21T09:00:02Z) - Human Mesh Recovery from Multiple Shots [85.18244937708356]
We propose a framework for improved 3D reconstruction and mining of long sequences with pseudo ground truth 3D human mesh.
We show that the resulting data is beneficial in the training of various human mesh recovery models.
The tools we develop open the door to processing and analyzing in 3D content from a large library of edited media.
arXiv Detail & Related papers (2020-12-17T18:58:02Z) - DeepFaceFlow: In-the-wild Dense 3D Facial Motion Estimation [56.56575063461169]
DeepFaceFlow is a robust, fast, and highly-accurate framework for the estimation of 3D non-rigid facial flow.
Our framework was trained and tested on two very large-scale facial video datasets.
Given registered pairs of images, our framework generates 3D flow maps at 60 fps.
arXiv Detail & Related papers (2020-05-14T23:56:48Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.