Implicit Neural Representation Facilitates Unified Universal Vision Encoding

• URL: http://arxiv.org/abs/2601.14256v1
• Date: Tue, 20 Jan 2026 18:59:57 GMT
• Title: Implicit Neural Representation Facilitates Unified Universal Vision Encoding
• Authors: Matthew Gwilliam, Xiao Wang, Xuefeng Hu, Zhenheng Yang,
• Abstract summary: A first-of-its-kind model learns representations which are simultaneously useful for recognition and generation.<n>We train our model as a hyper-network for implicit neural representation, which learns to map images to model weights for fast, accurate reconstruction.<n>The model also learns an unprecedented compressed embedding space with outstanding performance for various visual tasks.
• Score: 11.947746726150001
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Models for image representation learning are typically designed for either recognition or generation. Various forms of contrastive learning help models learn to convert images to embeddings that are useful for classification, detection, and segmentation. On the other hand, models can be trained to reconstruct images with pixel-wise, perceptual, and adversarial losses in order to learn a latent space that is useful for image generation. We seek to unify these two directions with a first-of-its-kind model that learns representations which are simultaneously useful for recognition and generation. We train our model as a hyper-network for implicit neural representation, which learns to map images to model weights for fast, accurate reconstruction. We further integrate our INR hyper-network with knowledge distillation to improve its generalization and performance. Beyond the novel training design, the model also learns an unprecedented compressed embedding space with outstanding performance for various visual tasks. The complete model competes with state-of-the-art results for image representation learning, while also enabling generative capabilities with its high-quality tiny embeddings. The code is available at https://github.com/tiktok/huvr.

Related papers

• GMAIL: Generative Modality Alignment for generated Image Learning [51.071351994330605]
  We propose a novel framework for discriminative use of generated images, coined GMAIL, that explicitly treats generated images as a separate modality from real images.<n>Our framework can be easily incorporated with various vision-language models, and we demonstrate its efficacy throughout extensive experiments.
  arXiv  Detail & Related papers  (2026-02-17T05:40:25Z)
• GloTok: Global Perspective Tokenizer for Image Reconstruction and Generation [51.95701097588426]
  We introduce a Global Perspective Tokenizer (GloTok) to model a more uniform semantic distribution of tokenized features.<n>A residual learning module is proposed to recover the fine-grained details to minimize the reconstruction error caused by quantization.<n>Experiments on the standard ImageNet-1k benchmark clearly show that our proposed method achieves state-of-the-art reconstruction performance and generation quality.
  arXiv  Detail & Related papers  (2025-11-18T06:40:26Z)
• Learned representation-guided diffusion models for large-image generation [58.192263311786824]
  We introduce a novel approach that trains diffusion models conditioned on embeddings from self-supervised learning (SSL) Our diffusion models successfully project these features back to high-quality histopathology and remote sensing images. Augmenting real data by generating variations of real images improves downstream accuracy for patch-level and larger, image-scale classification tasks.
  arXiv  Detail & Related papers  (2023-12-12T14:45:45Z)
• Sequential Modeling Enables Scalable Learning for Large Vision Models [120.91839619284431]
  We introduce a novel sequential modeling approach which enables learning a Large Vision Model (LVM) without making use of any linguistic data. We define a common format, "visual sentences", in which we can represent raw images and videos as well as annotated data sources.
  arXiv  Detail & Related papers  (2023-12-01T18:59:57Z)
• GPT4Image: Large Pre-trained Models Help Vision Models Learn Better on Perception Task [47.1857510710807]
  We present a new learning framework, dubbed GPT4Image, where the knowledge of the large pre-trained models are extracted to help CNNs and ViTs learn better representations.<n>We conduct extensive experiments to verify the effectiveness of the proposed algorithm on various visual perception tasks.
  arXiv  Detail & Related papers  (2023-06-01T14:02:45Z)
• Not All Image Regions Matter: Masked Vector Quantization for Autoregressive Image Generation [78.13793505707952]
  Existing autoregressive models follow the two-stage generation paradigm that first learns a codebook in the latent space for image reconstruction and then completes the image generation autoregressively based on the learned codebook. We propose a novel two-stage framework, which consists of Masked Quantization VAE (MQ-VAE) Stack model from modeling redundancy.
  arXiv  Detail & Related papers  (2023-05-23T02:15:53Z)
• MAGE: MAsked Generative Encoder to Unify Representation Learning and Image Synthesis [33.46831766206675]
  MAsked Generative (MAGE) is first framework to unify SOTA image generation and self-supervised representation learning. Inspired by previous generative models, MAGE uses semantic tokens learned by a vector-quantized GAN at inputs and outputs. On ImageNet-1K, a single MAGE ViT-L model obtains 9.10 FID in the task of class-unconditional image generation.
  arXiv  Detail & Related papers  (2022-11-16T18:59:02Z)
• Meta Internal Learning [88.68276505511922]
  Internal learning for single-image generation is a framework, where a generator is trained to produce novel images based on a single image. We propose a meta-learning approach that enables training over a collection of images, in order to model the internal statistics of the sample image more effectively. Our results show that the models obtained are as suitable as single-image GANs for many common image applications.
  arXiv  Detail & Related papers  (2021-10-06T16:27:38Z)
• Distilling Visual Priors from Self-Supervised Learning [24.79633121345066]
  Convolutional Neural Networks (CNNs) are prone to overfit small training datasets. We present a novel two-phase pipeline that leverages self-supervised learning and knowledge distillation to improve the generalization ability of CNN models for image classification under the data-deficient setting.
  arXiv  Detail & Related papers  (2020-08-01T13:07:18Z)
• Text-to-Image Generation with Attention Based Recurrent Neural Networks [1.2599533416395765]
  We develop a tractable and stable caption-based image generation model. Experimentations are performed on Microsoft datasets. Results show that the proposed model performs better than contemporary approaches.
  arXiv  Detail & Related papers  (2020-01-18T12:19:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.