Evaluation of facial landmark localization performance in a surgical setting

• URL: http://arxiv.org/abs/2507.18248v1
• Date: Thu, 24 Jul 2025 09:40:47 GMT
• Title: Evaluation of facial landmark localization performance in a surgical setting
• Authors: Ines Frajtag, Marko Švaco, Filip Šuligoj,
• Abstract summary: Many face detection algorithms have found applications in neurosurgery, ophthalmology, and plastic surgery.<n>A common challenge in using these algorithms is variable lighting conditions and the flexibility of detection positions to identify and precisely localize patients.<n>The proposed experiment tests the MediaPipe algorithm for detecting facial landmarks in a controlled setting, using a robotic arm that automatically adjusts positions while the surgical light and the phantom remain in a fixed position.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The use of robotics, computer vision, and their applications is becoming increasingly widespread in various fields, including medicine. Many face detection algorithms have found applications in neurosurgery, ophthalmology, and plastic surgery. A common challenge in using these algorithms is variable lighting conditions and the flexibility of detection positions to identify and precisely localize patients. The proposed experiment tests the MediaPipe algorithm for detecting facial landmarks in a controlled setting, using a robotic arm that automatically adjusts positions while the surgical light and the phantom remain in a fixed position. The results of this study demonstrate that the improved accuracy of facial landmark detection under surgical lighting significantly enhances the detection performance at larger yaw and pitch angles. The increase in standard deviation/dispersion occurs due to imprecise detection of selected facial landmarks. This analysis allows for a discussion on the potential integration of the MediaPipe algorithm into medical procedures.

Related papers

• Stable Tracking of Eye Gaze Direction During Ophthalmic Surgery [6.887111029199582]
  This study proposes an innovative eye localization and tracking method that combines machine learning with traditional algorithms.<n>Our proposed method has an average estimation error of 0.58 degrees for eye orientation estimation and 2.08-degree average control error for the robotic arm's movement.
  arXiv  Detail & Related papers  (2025-07-01T10:28:40Z)
• Benchmarking Laparoscopic Surgical Image Restoration and Beyond [54.28852320829451]
  In laparoscopic surgery, a clear and high-quality visual field is critical for surgeons to make accurate decisions.<n> persistent visual degradation, including smoke generated by energy devices, lens fogging from thermal gradients, and lens contamination pose risks to patient safety.<n>We introduce a real-world open-source surgical image restoration dataset covering laparoscopic environments, called SurgClean.
  arXiv  Detail & Related papers  (2025-05-25T14:17:56Z)
• Adapting Visual-Language Models for Generalizable Anomaly Detection in Medical Images [68.42215385041114]
  This paper introduces a novel lightweight multi-level adaptation and comparison framework to repurpose the CLIP model for medical anomaly detection. Our approach integrates multiple residual adapters into the pre-trained visual encoder, enabling a stepwise enhancement of visual features across different levels. Our experiments on medical anomaly detection benchmarks demonstrate that our method significantly surpasses current state-of-the-art models.
  arXiv  Detail & Related papers  (2024-03-19T09:28:19Z)
• End-to-End assessment of AR-assisted neurosurgery systems [0.5892638927736115]
  We classify different techniques for assessing an AR-assisted neurosurgery system and propose a new technique to systematize the assessment procedure. We found that although the system can undergo registration and tracking errors, physical feedback can significantly reduce the error caused by hologram displacement. The lack of visual feedback on the hologram does not have a significant effect on the user 3D perception.
  arXiv  Detail & Related papers  (2023-11-03T13:41:44Z)
• Gravity Network for end-to-end small lesion detection [50.38534263407915]
  This paper introduces a novel one-stage end-to-end detector specifically designed to detect small lesions in medical images. Precise localization of small lesions presents challenges due to their appearance and the diverse contextual backgrounds in which they are found. We refer to this new architecture as GravityNet, and the novel anchors as gravity points since they appear to be "attracted" by the lesions.
  arXiv  Detail & Related papers  (2023-09-22T14:02:22Z)
• Detecting the Sensing Area of A Laparoscopic Probe in Minimally Invasive Cancer Surgery [6.0097646269887965]
  In surgical oncology, it is challenging for surgeons to identify lymph nodes and completely resect cancer. A novel tethered laparoscopic gamma detector is used to localize a preoperatively injected radiotracer. Gamma activity visualization is challenging to present to the operator because the probe is non-imaging and it does not visibly indicate the activity on the tissue surface.
  arXiv  Detail & Related papers  (2023-07-07T15:33:49Z)
• Next-generation Surgical Navigation: Marker-less Multi-view 6DoF Pose Estimation of Surgical Instruments [64.59698930334012]
  We present a multi-camera capture setup consisting of static and head-mounted cameras.<n>Second, we publish a multi-view RGB-D video dataset of ex-vivo spine surgeries, captured in a surgical wet lab and a real operating theatre.<n>Third, we evaluate three state-of-the-art single-view and multi-view methods for the task of 6DoF pose estimation of surgical instruments.
  arXiv  Detail & Related papers  (2023-05-05T13:42:19Z)
• A Temporal Learning Approach to Inpainting Endoscopic Specularities and Its effect on Image Correspondence [13.25903945009516]
  We propose using a temporal generative adversarial network (GAN) to inpaint the hidden anatomy under specularities. This is achieved using in-vivo data of gastric endoscopy (Hyper-Kvasir) in a fully unsupervised manner. We also assess the effect of our method in computer vision tasks that underpin 3D reconstruction and camera motion estimation.
  arXiv  Detail & Related papers  (2022-03-31T13:14:00Z)
• Unsupervised deep learning techniques for powdery mildew recognition based on multispectral imaging [63.62764375279861]
  This paper presents a deep learning approach to automatically recognize powdery mildew on cucumber leaves. We focus on unsupervised deep learning techniques applied to multispectral imaging data. We propose the use of autoencoder architectures to investigate two strategies for disease detection.
  arXiv  Detail & Related papers  (2021-12-20T13:29:13Z)
• Searching for Efficient Architecture for Instrument Segmentation in Robotic Surgery [58.63306322525082]
  Most applications rely on accurate real-time segmentation of high-resolution surgical images. We design a light-weight and highly-efficient deep residual architecture which is tuned to perform real-time inference of high-resolution images.
  arXiv  Detail & Related papers  (2020-07-08T21:38:29Z)
• Real-Time Face and Landmark Localization for Eyeblink Detection [1.617280000789628]
  A face- and landmark-detection algorithm have been carefully combined in order to provide fully automated eyelid tracking. Our experimental evaluation reveals an overall runtime application of 0.533 ms per frame. This is 1,101$times$ faster than the sequential implementation and well within the real-time requirements of eyeblink conditioning in humans.
  arXiv  Detail & Related papers  (2020-06-01T09:46:25Z)
• Robust Medical Instrument Segmentation Challenge 2019 [56.148440125599905]
  Intraoperative tracking of laparoscopic instruments is often a prerequisite for computer and robotic-assisted interventions. Our challenge was based on a surgical data set comprising 10,040 annotated images acquired from a total of 30 surgical procedures. The results confirm the initial hypothesis, namely that algorithm performance degrades with an increasing domain gap.
  arXiv  Detail & Related papers  (2020-03-23T14:35:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.