Neural Space-filling Curves

• URL: http://arxiv.org/abs/2204.08453v1
• Date: Mon, 18 Apr 2022 17:59:01 GMT
• Title: Neural Space-filling Curves
• Authors: Hanyu Wang, Kamal Gupta, Larry Davis, Abhinav Shrivastava
• Abstract summary: We present a data-driven approach to infer a context-based scan order for a set of images. Our work learns a spatially coherent linear ordering of pixels from the dataset of images using a graph-based neural network. We show the advantage of using Neural SFCs in downstream applications such as image compression.
• Score: 47.852964985588486
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present Neural Space-filling Curves (SFCs), a data-driven approach to infer a context-based scan order for a set of images. Linear ordering of pixels forms the basis for many applications such as video scrambling, compression, and auto-regressive models that are used in generative modeling for images. Existing algorithms resort to a fixed scanning algorithm such as Raster scan or Hilbert scan. Instead, our work learns a spatially coherent linear ordering of pixels from the dataset of images using a graph-based neural network. The resulting Neural SFC is optimized for an objective suitable for the downstream task when the image is traversed along with the scan line order. We show the advantage of using Neural SFCs in downstream applications such as image compression. Code and additional results will be made available at https://hywang66.github.io/publication/neuralsfc.

Related papers

• Image-GS: Content-Adaptive Image Representation via 2D Gaussians [55.15950594752051]
  We propose Image-GS, a content-adaptive image representation. Using anisotropic 2D Gaussians as the basis, Image-GS shows high memory efficiency, supports fast random access, and offers a natural level of detail stack. General efficiency and fidelity of Image-GS are validated against several recent neural image representations and industry-standard texture compressors. We hope this research offers insights for developing new applications that require adaptive quality and resource control, such as machine perception, asset streaming, and content generation.
  arXiv  Detail & Related papers  (2024-07-02T00:45:21Z)
• N-BVH: Neural ray queries with bounding volume hierarchies [51.430495562430565]
  In 3D computer graphics, the bulk of a scene's memory usage is due to polygons and textures. We devise N-BVH, a neural compression architecture designed to answer arbitrary ray queries in 3D. Our method provides faithful approximations of visibility, depth, and appearance attributes.
  arXiv  Detail & Related papers  (2024-05-25T13:54:34Z)
• A Novel Algorithm for Exact Concave Hull Extraction [0.0]
  Region extraction is necessary in a wide range of applications, from object detection in autonomous driving to analysis of subcellular morphology in cell biology. There exist two main approaches: convex hull extraction, for which exact and efficient algorithms exist and concave hulls, which are better at capturing real-world shapes but do not have a single solution. In this study, we present a novel algorithm that can provide concave hulls with maximal (i.e. pixel-perfect) resolution and is tunable for speed-efficiency tradeoffs.
  arXiv  Detail & Related papers  (2022-06-23T05:26:48Z)
• Variable Bitrate Neural Fields [75.24672452527795]
  We present a dictionary method for compressing feature grids, reducing their memory consumption by up to 100x. We formulate the dictionary optimization as a vector-quantized auto-decoder problem which lets us learn end-to-end discrete neural representations in a space where no direct supervision is available.
  arXiv  Detail & Related papers  (2022-06-15T17:58:34Z)
• FSOINet: Feature-Space Optimization-Inspired Network for Image Compressive Sensing [11.352530132548912]
  We propose the idea of achieving information flow phase by phase in feature space and design a Feature-Space Optimization-Inspired Network (dubbed FSOINet) Experiments show that the proposed FSOINet outperforms the existing state-of-the-art methods by a large margin both quantitatively and qualitatively.
  arXiv  Detail & Related papers  (2022-04-12T03:30:22Z)
• Oscillatory Neural Network as Hetero-Associative Memory for Image Edge Detection [0.0]
  We propose a novel image processing method by using ONNs as a hetero-associative memory for image edge detection. This work is the first to explore ONNs as hetero-associative memory for image processing applications.
  arXiv  Detail & Related papers  (2022-02-25T08:09:29Z)
• Neural Knitworks: Patched Neural Implicit Representation Networks [1.0470286407954037]
  We propose Knitwork, an architecture for neural implicit representation learning of natural images that achieves image synthesis. To the best of our knowledge, this is the first implementation of a coordinate-based patch tailored for synthesis tasks such as image inpainting, super-resolution, and denoising. The results show that modeling natural images using patches, rather than pixels, produces results of higher fidelity.
  arXiv  Detail & Related papers  (2021-09-29T13:10:46Z)
• OSLO: On-the-Sphere Learning for Omnidirectional images and its application to 360-degree image compression [59.58879331876508]
  We study the learning of representation models for omnidirectional images and propose to use the properties of HEALPix uniform sampling of the sphere to redefine the mathematical tools used in deep learning models for omnidirectional images. Our proposed on-the-sphere solution leads to a better compression gain that can save 13.7% of the bit rate compared to similar learned models applied to equirectangular images.
  arXiv  Detail & Related papers  (2021-07-19T22:14:30Z)
• Neural Sparse Representation for Image Restoration [116.72107034624344]
  Inspired by the robustness and efficiency of sparse coding based image restoration models, we investigate the sparsity of neurons in deep networks. Our method structurally enforces sparsity constraints upon hidden neurons. Experiments show that sparse representation is crucial in deep neural networks for multiple image restoration tasks.
  arXiv  Detail & Related papers  (2020-06-08T05:15:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.