At FullTilt: Real-Time Open-Set 3D Macromolecule Detection Directly from Tilted 2D Projections

• URL: http://arxiv.org/abs/2604.10766v1
• Date: Sun, 12 Apr 2026 18:28:57 GMT
• Title: At FullTilt: Real-Time Open-Set 3D Macromolecule Detection Directly from Tilted 2D Projections
• Abstract summary: VRAM constraints prohibit processing an entire 3D tomogram, forcing current methods to rely on slow sliding-window inference over extracted subvolumes.<n>We propose FullTilt, an end-to-end framework that redefines 3D detection by operating directly on aligned 2D tilt-series.<n>FullTilt achieves state-of-the-art zero-shot performance while drastically reducing VRAM requirements, paving the way for rapid, large-scale visual analysis.
• Score: 2.9465623430708905
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Open-set 3D macromolecule detection in cryogenic electron tomography eliminates the need for target-specific model retraining. However, strict VRAM constraints prohibit processing an entire 3D tomogram, forcing current methods to rely on slow sliding-window inference over extracted subvolumes. To overcome this, we propose FullTilt, an end-to-end framework that redefines 3D detection by operating directly on aligned 2D tilt-series. Because a tilt-series contains significantly fewer images than slices in a reconstructed tomogram, FullTilt eliminates redundant volumetric computation, accelerating inference by orders of magnitude. To process the entire tilt-series simultaneously, we introduce a tilt-series encoder to efficiently fuse cross-view information. We further propose a multiclass visual prompt encoder for flexible prompting, a tilt-aware query initializer to effectively anchor 3D queries, and an auxiliary geometric primitives module to enhance the model's understanding of multi-view geometry while improving robustness to adverse imaging artifacts. Extensive evaluations on three real-world datasets demonstrate that FullTilt achieves state-of-the-art zero-shot performance while drastically reducing runtime and VRAM requirements, paving the way for rapid, large-scale visual proteomics analysis. All code and data will be publicly available upon publication.

Related papers

• Wavelet-Optimized Pseudo-3D Accelerated Diffusion Model for Truncated Computed Laminography [12.168086376276934]
  Field-of-view (FOV) limitations inevitably lead to truncation of projected data.<n>Existing deep learning methods rely on 2D architectures that lack rigorous data consistency constraints.<n>We propose a novel wavelet-optimized pseudo-3D accelerated diffusion model for CL truncation reconstruction (CL-DM)
  arXiv  Detail & Related papers  (2026-06-30T08:24:57Z)
• MB-Loc: Multi-planar Bird's-eye-view Localization in outdoor LiDAR scenes [2.3666547827177733]
  We present MB-Loc, a lightweight and viewpoint-robust SCR framework for global LiDAR localization.<n>By slicing the point-cloud along the Z-axis and mapping signed depths into discrete 2D planes, MB-Loc retains essential 3D geometric structures while exploiting the computational tractability of standard 2D CNNs.<n>We perform extensive experiments on the publicly available NCLT dataset and demonstrate that our proposed method outperforms the current state-of-the-art.
  arXiv  Detail & Related papers  (2026-06-07T17:25:06Z)
• TriCLIP-3D: A Unified Parameter-Efficient Framework for Tri-Modal 3D Visual Grounding based on CLIP [52.79100775328595]
  3D visual grounding allows an embodied agent to understand visual information in real-world 3D environments based on human instructions.<n>Existing 3D visual grounding methods rely on separate encoders for different modalities.<n>We propose a unified 2D pre-trained multi-modal network to process all three modalities.
  arXiv  Detail & Related papers  (2025-07-20T10:28:06Z)
• RAPS-3D: Efficient interactive segmentation for 3D radiological imaging [5.8497833718980345]
  Adapting 2D models to 3D typically involves autoregressive strategies, where predictions are propagated slice by slice.<n>We present a simplified 3D promptable segmentation method, inspired by SegVol, designed to reduce inference time and eliminate prompt management complexities.
  arXiv  Detail & Related papers  (2025-07-10T13:08:57Z)
• VideoLifter: Lifting Videos to 3D with Fast Hierarchical Stereo Alignment [54.66217340264935]
  VideoLifter is a novel video-to-3D pipeline that leverages a local-to-global strategy on a fragment basis.<n>It significantly accelerates the reconstruction process, reducing training time by over 82% while holding better visual quality than current SOTA methods.
  arXiv  Detail & Related papers  (2025-01-03T18:52:36Z)
• PF3plat: Pose-Free Feed-Forward 3D Gaussian Splatting [54.7468067660037]
  PF3plat sets a new state-of-the-art across all benchmarks, supported by comprehensive ablation studies validating our design choices.<n>Our framework capitalizes on fast speed, scalability, and high-quality 3D reconstruction and view synthesis capabilities of 3DGS.
  arXiv  Detail & Related papers  (2024-10-29T15:28:15Z)
• DM3D: Distortion-Minimized Weight Pruning for Lossless 3D Object Detection [42.07920565812081]
  We propose a novel post-training weight pruning scheme for 3D object detection. It determines redundant parameters in the pretrained model that lead to minimal distortion in both locality and confidence. This framework aims to minimize detection distortion of network output to maximally maintain detection precision.
  arXiv  Detail & Related papers  (2024-07-02T09:33:32Z)
• Instant Multi-View Head Capture through Learnable Registration [62.70443641907766]
  Existing methods for capturing datasets of 3D heads in dense semantic correspondence are slow. We introduce TEMPEH to directly infer 3D heads in dense correspondence from calibrated multi-view images. Predicting one head takes about 0.3 seconds with a median reconstruction error of 0.26 mm, 64% lower than the current state-of-the-art.
  arXiv  Detail & Related papers  (2023-06-12T21:45:18Z)
• Parallel Diffusion Model-based Sparse-view Cone-beam Breast CT [7.712142153700843]
  We transform the cutting-edge Denoising Diffusion Probabilistic Model (DDPM) into a parallel framework for sub-volume-based sparse-view breast CT image reconstruction. Our experimental findings reveal that this method delivers competitive reconstruction performance at half to one-third of the standard radiation doses.
  arXiv  Detail & Related papers  (2023-03-22T18:55:43Z)
• Weakly Supervised Volumetric Image Segmentation with Deformed Templates [80.04326168716493]
  We propose an approach that is truly weakly-supervised in the sense that we only need to provide a sparse set of 3D point on the surface of target objects. We will show that it outperforms a more traditional approach to weak-supervision in 3D at a reduced supervision cost.
  arXiv  Detail & Related papers  (2021-06-07T22:09:34Z)
• Reinforced Axial Refinement Network for Monocular 3D Object Detection [160.34246529816085]
  Monocular 3D object detection aims to extract the 3D position and properties of objects from a 2D input image. Conventional approaches sample 3D bounding boxes from the space and infer the relationship between the target object and each of them, however, the probability of effective samples is relatively small in the 3D space. We propose to start with an initial prediction and refine it gradually towards the ground truth, with only one 3d parameter changed in each step. This requires designing a policy which gets a reward after several steps, and thus we adopt reinforcement learning to optimize it.
  arXiv  Detail & Related papers  (2020-08-31T17:10:48Z)
• Range Conditioned Dilated Convolutions for Scale Invariant 3D Object Detection [41.59388513615775]
  This paper presents a novel 3D object detection framework that processes LiDAR data directly on its native representation: range images. Benefiting from the compactness of range images, 2D convolutions can efficiently process dense LiDAR data of a scene.
  arXiv  Detail & Related papers  (2020-05-20T09:24:43Z)
• Lightweight Multi-View 3D Pose Estimation through Camera-Disentangled Representation [57.11299763566534]
  We present a solution to recover 3D pose from multi-view images captured with spatially calibrated cameras. We exploit 3D geometry to fuse input images into a unified latent representation of pose, which is disentangled from camera view-points. Our architecture then conditions the learned representation on camera projection operators to produce accurate per-view 2d detections.
  arXiv  Detail & Related papers  (2020-04-05T12:52:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.