Larger than memory image processing

• URL: http://arxiv.org/abs/2601.18407v1
• Date: Mon, 26 Jan 2026 12:02:41 GMT
• Title: Larger than memory image processing
• Abstract summary: This report addresses larger-than-memory image analysis for petascale datasets such as 1.4 PB electronmicroscopy volumes and 150 TB human-organ atlases.<n>We show that structuring analysis as streaming passes over data is crucial.<n>For 3D volumes, two representations are popular: stacks of 2D slices (e.g., directories or multi-page TIFF) and 3D chunked layouts (e.g., Zarr/HDF5).<n>We show how the slice-based streaming architecture can be built on top of either image representation in a manner that minimizes disk I/O.
• Score: 0.7161783472741748
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This report addresses larger-than-memory image analysis for petascale datasets such as 1.4 PB electron-microscopy volumes and 150 TB human-organ atlases. We argue that performance is fundamentally I/O-bound. We show that structuring analysis as streaming passes over data is crucial. For 3D volumes, two representations are popular: stacks of 2D slices (e.g., directories or multi-page TIFF) and 3D chunked layouts (e.g., Zarr/HDF5). While for a few algorithms, chunked layout on disk is crucial to keep disk I/O at a minimum, we show how the slice-based streaming architecture can be built on top of either image representation in a manner that minimizes disk I/O. This is in particular advantageous for algorithms relying on neighbouring values, since the slicing streaming architecture is 1D, which implies that there are only 2 possible sweeping orders, both of which are aligned with the order in which images are read from the disk. This is in contrast to 3D chunks, in which any sweep cannot be done without accessing each chunk at least 9 times. We formalize this with sweep-based execution (natural 2D/3D orders), windowed operations, and overlap-aware tiling to minimize redundant access. Building on these principles, we introduce a domain-specific language (DSL) that encodes algorithms with intrinsic knowledge of their optimal streaming and memory use; the DSL performs compile-time and run-time pipeline analyses to automatically select window sizes, fuse stages, tee and zip streams, and schedule passes for limited-RAM machines, yielding near-linear I/O scans and predictable memory footprints. The approach integrates with existing tooling for segmentation and morphology but reframes pre/post-processing as pipelines that privilege sequential read/write patterns, delivering substantial throughput gains for extremely large images without requiring full-volume residency in memory.

Related papers

• Memory-efficient GPU pipelines for real-time non-line-of-sight reconstruction [0.30556942817031457]
  Non-line-of-sight (NLOS) imaging reconstructs scenes hidden around a corner from indirect light recorded by a single-photon avalanche diode (SPAD)<n>A single reconstruction is a large inverse problem: billions of photon timestamps must be binned, moved through memory, transformed and inverted.<n>We rebuild the execution of two established wave-based algorithms, f-k migration and phasor-fields, for both streaming and offline processing.
  arXiv  Detail & Related papers  (2026-08-28T10:51:37Z)
• LightSplat: Fast and Memory-Efficient Open-Vocabulary 3D Scene Understanding in Five Seconds [11.158005062855372]
  LightSplat is a training-free framework that injects compact 2-byte semantic indices into 3D representations from multi-view images.<n>LightSplat achieves state-of-the-art performance with up to 50-400x speedup and 64x lower memory, enabling scalable language-driven 3D understanding.
  arXiv  Detail & Related papers  (2026-03-25T10:12:22Z)
• S2GS: Streaming Semantic Gaussian Splatting for Online Scene Understanding and Reconstruction [57.07346645250984]
  Streaming Semantic Gaussian Splatting (S2GS) is a strictly causal, incremental 3D Gaussian semantic field framework.<n>It does not leverage future frames and continuously updates scene geometry, appearance, and instance-level semantics without reprocessing historical frames.<n>S2GS matches or outperforms strong offline baselines on joint reconstruction-and-understanding benchmarks.
  arXiv  Detail & Related papers  (2026-03-15T05:48:55Z)
• ImLoc: Revisiting Visual Localization with Image-based Representation [61.282162006394934]
  We propose to augment each image with estimated depth maps to capture the geometric structure.<n>This representation is easy to build and maintain, but achieves highest accuracy in challenging conditions.<n>Our method achieves a new state-of-the-art accuracy on various standard benchmarks and outperforms existing memory-efficient methods at comparable map sizes.
  arXiv  Detail & Related papers  (2026-01-07T18:51:51Z)
• DistZO2: High-Throughput and Memory-Efficient Zeroth-Order Fine-tuning LLMs with Distributed Parallel Computing [4.589472292598182]
  Fine-tuning large language models (LLMs) remains resource-intensive due to their sheer scale.<n>We present DistZO2, a memory-efficient framework for distributed zeroth-order fine-tuning of LLMs.
  arXiv  Detail & Related papers  (2025-07-03T22:53:34Z)
• LODGE: Level-of-Detail Large-Scale Gaussian Splatting with Efficient Rendering [75.67501939005119]
  We present a novel level-of-detail (LOD) method for 3D Gaussian Splatting on memory-constrained devices.<n>Our approach iteratively selects optimal subsets of Gaussians based on camera distance.<n>Our method achieves state-of-the-art performance on both outdoor (Hierarchical 3DGS) and indoor (Zip-NeRF) datasets.
  arXiv  Detail & Related papers  (2025-05-29T06:50:57Z)
• vTensor: Flexible Virtual Tensor Management for Efficient LLM Serving [53.972175896814505]
  Large Language Models (LLMs) are widely used across various domains, processing millions of daily requests. Large Language Models (LLMs) are widely used across various domains, processing millions of daily requests.
  arXiv  Detail & Related papers  (2024-07-22T14:37:58Z)
• ALSTER: A Local Spatio-Temporal Expert for Online 3D Semantic Reconstruction [62.599588577671796]
  We propose an online 3D semantic segmentation method that incrementally reconstructs a 3D semantic map from a stream of RGB-D frames. Unlike offline methods, ours is directly applicable to scenarios with real-time constraints, such as robotics or mixed reality.
  arXiv  Detail & Related papers  (2023-11-29T20:30:18Z)
• High-performance Data Management for Whole Slide Image Analysis in Digital Pathology [20.02220555234364]
  This paper details our endeavors in tackling this data access challenge by implementing the Adaptable IO System version 2 (ADIOS2) Under the CPU scenario, ADIOS2 showcases an impressive two-fold speed-up compared to the brute-force approach. From what we know, this appears to be among the initial instances, if any, of utilizing ADIOS2 within the field of digital pathology.
  arXiv  Detail & Related papers  (2023-08-10T16:33:59Z)
• NumS: Scalable Array Programming for the Cloud [82.827921577004]
  We present NumS, an array programming library which optimize NumPy-like expressions on task-based distributed systems. This is achieved through a novel scheduler called Load Simulated Hierarchical Scheduling (LSHS) We show that LSHS enhances performance on Ray by decreasing network load by a factor of 2x, requiring 4x less memory, and reducing execution time by 10x on the logistic regression problem.
  arXiv  Detail & Related papers  (2022-06-28T20:13:40Z)
• Memory-efficient Segmentation of High-resolution Volumetric MicroCT Images [11.723370840090453]
  We propose a memory-efficient network architecture for 3D high-resolution image segmentation. The network incorporates both global and local features via a two-stage U-net-based cascaded framework. Experiments show that it outperforms state-of-the-art 3D segmentation methods in terms of both segmentation accuracy and memory efficiency.
  arXiv  Detail & Related papers  (2022-05-31T16:42:48Z)
• Parallel Discrete Convolutions on Adaptive Particle Representations of Images [2.362412515574206]
  We present data structures and algorithms for native implementations of discrete convolution operators over Adaptive Particle Representations. The APR is a content-adaptive image representation that locally adapts the sampling resolution to the image signal. We show that APR convolution naturally leads to scale-adaptive algorithms that efficiently parallelize on multi-core CPU and GPU architectures.
  arXiv  Detail & Related papers  (2021-12-07T09:40:05Z)
• ASH: A Modern Framework for Parallel Spatial Hashing in 3D Perception [91.24236600199542]
  ASH is a modern and high-performance framework for parallel spatial hashing on GPU. ASH achieves higher performance, supports richer functionality, and requires fewer lines of code. ASH and its example applications are open sourced in Open3D.
  arXiv  Detail & Related papers  (2021-10-01T16:25:40Z)
• SRH-Net: Stacked Recurrent Hourglass Network for Stereo Matching [33.66537830990198]
  We decouple the 4D cubic cost volume used by 3D convolutional filters into sequential cost maps along the direction of disparity. A novel recurrent module, Stacked Recurrent Hourglass (SRH), is proposed to process each cost map. The proposed architecture is implemented in an end-to-end pipeline and evaluated on public datasets.
  arXiv  Detail & Related papers  (2021-05-25T00:10:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.