A Dual-Feature Extractor Framework for Accurate Back Depth and Spine Morphology Estimation from Monocular RGB Images

• URL: http://arxiv.org/abs/2507.22691v1
• Date: Wed, 30 Jul 2025 13:55:37 GMT
• Title: A Dual-Feature Extractor Framework for Accurate Back Depth and Spine Morphology Estimation from Monocular RGB Images
• Authors: Yuxin Wei, Yue Zhang, Moxin Zhao, Chang Shi, Jason P. Y. Cheung, Teng Zhang, Nan Meng,
• Abstract summary: We propose a novel pipeline to accurately estimate the depth information of the unclothed back.<n>We then estimate spine morphology by integrating both depth and surface information.<n>This integrated approach enhances the accuracy of spine curve generation, achieving an impressive performance of up to 97%.
• Score: 15.19284295210246
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Scoliosis is a prevalent condition that impacts both physical health and appearance, with adolescent idiopathic scoliosis (AIS) being the most common form. Currently, the main AIS assessment tool, X-rays, poses significant limitations, including radiation exposure and limited accessibility in poor and remote areas. To address this problem, the current solutions are using RGB images to analyze spine morphology. However, RGB images are highly susceptible to environmental factors, such as lighting conditions, compromising model stability and generalizability. Therefore, in this study, we propose a novel pipeline to accurately estimate the depth information of the unclothed back, compensating for the limitations of 2D information, and then estimate spine morphology by integrating both depth and surface information. To capture the subtle depth variations of the back surface with precision, we design an adaptive multiscale feature learning network named Grid-Aware Multiscale Adaptive Network (GAMA-Net). This model uses dual encoders to extract both patch-level and global features, which are then interacted by the Patch-Based Hybrid Attention (PBHA) module. The Adaptive Multiscale Feature Fusion (AMFF) module is used to dynamically fuse information in the decoder. As a result, our depth estimation model achieves remarkable accuracy across three different evaluation metrics, with scores of nearly 78.2%, 93.6%, and 97.5%, respectively. To further validate the effectiveness of the predicted depth, we integrate both surface and depth information for spine morphology estimation. This integrated approach enhances the accuracy of spine curve generation, achieving an impressive performance of up to 97%.

Related papers

• Deep Supervised LSTM for 3D morphology estimation from Multi-View RGB Images of Wheat Spikes [0.0]
  Estimating morphological traits from two-dimensional RGB images presents inherent challenges.<n>We propose a neural network approach for volume estimation in 2D images.<n>Our deep supervised model achieves a mean absolute percentage error (MAPE) of 6.46% on six-view indoor images.
  arXiv  Detail & Related papers  (2025-06-22T15:02:18Z)
• Lightweight RGB-D Salient Object Detection from a Speed-Accuracy Tradeoff Perspective [54.91271106816616]
  Current RGB-D methods usually leverage large-scale backbones to improve accuracy but sacrifice efficiency.<n>We propose a Speed-Accuracy Tradeoff Network (SATNet) for Lightweight RGB-D SOD from three fundamental perspectives.<n> Concerning depth quality, we introduce the Depth Anything Model to generate high-quality depth maps.<n>For modality fusion, we propose a Decoupled Attention Module (DAM) to explore the consistency within and between modalities.<n>For feature representation, we develop a Dual Information Representation Module (DIRM) with a bi-directional inverted framework.
  arXiv  Detail & Related papers  (2025-05-07T19:37:20Z)
• Enhanced Encoder-Decoder Architecture for Accurate Monocular Depth Estimation [0.0]
  This paper introduces a novel deep learning-based approach using an enhanced encoder-decoder architecture.<n>It incorporates multi-scale feature extraction to enhance depth prediction accuracy across various object sizes and distances.<n> Experimental results on the KITTI dataset show that our model achieves a significantly faster inference time of 0.019 seconds.
  arXiv  Detail & Related papers  (2024-10-15T13:46:19Z)
• Adapting Visual-Language Models for Generalizable Anomaly Detection in Medical Images [68.42215385041114]
  This paper introduces a novel lightweight multi-level adaptation and comparison framework to repurpose the CLIP model for medical anomaly detection. Our approach integrates multiple residual adapters into the pre-trained visual encoder, enabling a stepwise enhancement of visual features across different levels. Our experiments on medical anomaly detection benchmarks demonstrate that our method significantly surpasses current state-of-the-art models.
  arXiv  Detail & Related papers  (2024-03-19T09:28:19Z)
• Confidence-Aware RGB-D Face Recognition via Virtual Depth Synthesis [48.59382455101753]
  2D face recognition encounters challenges in unconstrained environments due to varying illumination, occlusion, and pose. Recent studies focus on RGB-D face recognition to improve robustness by incorporating depth information. In this work, we first construct a diverse depth dataset generated by 3D Morphable Models for depth model pre-training. Then, we propose a domain-independent pre-training framework that utilizes readily available pre-trained RGB and depth models to separately perform face recognition without needing additional paired data for retraining.
  arXiv  Detail & Related papers  (2024-03-11T09:12:24Z)
• Multi-task learning with cross-task consistency for improved depth estimation in colonoscopy [0.2995885872626565]
  We develop a novel multi-task learning (MTL) approach with a shared encoder and two decoders, namely a surface normal decoder and a depth estimator. We demonstrate an improvement of 14.17% on relative error and 10.4% on $delta_1$ accuracy over the most accurate baseline state-of-the-art BTS approach.
  arXiv  Detail & Related papers  (2023-11-30T16:13:17Z)
• MMA-Net: Multiple Morphology-Aware Network for Automated Cobb Angle Measurement [6.8243631770391735]
  We introduce a novel framework that improves Cobb angle measurement accuracy by integrating multiple spine morphology as attention information. We evaluate our method on the AASCE challenge dataset and achieve superior performance with the SMAPE of 7.28% and the MAE of 3.18deg.
  arXiv  Detail & Related papers  (2023-09-25T01:56:53Z)
• Affinity Feature Strengthening for Accurate, Complete and Robust Vessel Segmentation [48.638327652506284]
  Vessel segmentation is crucial in many medical image applications, such as detecting coronary stenoses, retinal vessel diseases and brain aneurysms. We present a novel approach, the affinity feature strengthening network (AFN), which jointly models geometry and refines pixel-wise segmentation features using a contrast-insensitive, multiscale affinity approach.
  arXiv  Detail & Related papers  (2022-11-12T05:39:17Z)
• Dual Swin-Transformer based Mutual Interactive Network for RGB-D Salient Object Detection [67.33924278729903]
  In this work, we propose Dual Swin-Transformer based Mutual Interactive Network. We adopt Swin-Transformer as the feature extractor for both RGB and depth modality to model the long-range dependencies in visual inputs. Comprehensive experiments on five standard RGB-D SOD benchmark datasets demonstrate the superiority of the proposed DTMINet method.
  arXiv  Detail & Related papers  (2022-06-07T08:35:41Z)
• HiMODE: A Hybrid Monocular Omnidirectional Depth Estimation Model [3.5290359800552946]
  HiMODE is a novel monocular omnidirectional depth estimation model based on a CNN+Transformer architecture. We show that HiMODE can achieve state-of-the-art performance for 360deg monocular depth estimation.
  arXiv  Detail & Related papers  (2022-04-11T11:11:43Z)
• Fader Networks for domain adaptation on fMRI: ABIDE-II study [68.5481471934606]
  We use 3D convolutional autoencoders to build the domain irrelevant latent space image representation and demonstrate this method to outperform existing approaches on ABIDE data.
  arXiv  Detail & Related papers  (2020-10-14T16:50:50Z)
• DONet: Dual Objective Networks for Skin Lesion Segmentation [77.9806410198298]
  We propose a simple yet effective framework, named Dual Objective Networks (DONet), to improve the skin lesion segmentation. Our DONet adopts two symmetric decoders to produce different predictions for approaching different objectives. To address the challenge of large variety of lesion scales and shapes in dermoscopic images, we additionally propose a recurrent context encoding module (RCEM)
  arXiv  Detail & Related papers  (2020-08-19T06:02:46Z)
• A Deep Convolutional Neural Network for the Detection of Polyps in Colonoscopy Images [12.618653234201089]
  We propose a deep convolutional neural network based model for the computerized detection of polyps within colonoscopy images. Data augmentation techniques such as photometric and geometric distortions are adapted to overcome the obstacles faced in polyp detection.
  arXiv  Detail & Related papers  (2020-08-15T13:55:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.