Towards Task-Compatible Compressible Representations

• URL: http://arxiv.org/abs/2405.10244v3
• Date: Mon, 15 Jul 2024 14:58:07 GMT
• Title: Towards Task-Compatible Compressible Representations
• Authors: Anderson de Andrade, Ivan Bajić,
• Abstract summary: We investigate an issue in multi-task learnable compression, in which a representation learned for one task does not positively contribute to the rate-distortion performance of a different task. In learnable scalable coding, previous work increased the utilization of side-information for input reconstruction by also rewarding input reconstruction when learning this shared representation. We perform experiments using representations trained for object detection on COCO 2017 and depth estimation on the Cityscapes dataset, and use them to assist in image reconstruction and semantic segmentation tasks.
• Score: 0.7980273012483663
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We identify an issue in multi-task learnable compression, in which a representation learned for one task does not positively contribute to the rate-distortion performance of a different task as much as expected, given the estimated amount of information available in it. We interpret this issue using the predictive $\mathcal{V}$-information framework. In learnable scalable coding, previous work increased the utilization of side-information for input reconstruction by also rewarding input reconstruction when learning this shared representation. We evaluate the impact of this idea in the context of input reconstruction more rigorously and extended it to other computer vision tasks. We perform experiments using representations trained for object detection on COCO 2017 and depth estimation on the Cityscapes dataset, and use them to assist in image reconstruction and semantic segmentation tasks. The results show considerable improvements in the rate-distortion performance of the assisted tasks. Moreover, using the proposed representations, the performance of the base tasks are also improved. Results suggest that the proposed method induces simpler representations that are more compatible with downstream processes.

Related papers

• Unveiling Backbone Effects in CLIP: Exploring Representational Synergies and Variances [49.631908848868505]
  Contrastive Language-Image Pretraining (CLIP) stands out as a prominent method for image representation learning. We investigate the differences in CLIP performance among various neural architectures. We propose a simple, yet effective approach to combine predictions from multiple backbones, leading to a notable performance boost of up to 6.34%.
  arXiv  Detail & Related papers  (2023-12-22T03:01:41Z)
• Revisiting Image Reconstruction for Semi-supervised Semantic Segmentation [16.27277238968567]
  We revisit the idea of using image reconstruction as an auxiliary task and incorporate it with a modern semi-supervised semantic segmentation framework. Surprisingly, we discover that such an old idea in semi-supervised learning can produce results competitive with state-of-the-art semantic segmentation algorithms.
  arXiv  Detail & Related papers  (2023-03-17T06:31:06Z)
• Composite Learning for Robust and Effective Dense Predictions [81.2055761433725]
  Multi-task learning promises better model generalization on a target task by jointly optimizing it with an auxiliary task. We find that jointly training a dense prediction (target) task with a self-supervised (auxiliary) task can consistently improve the performance of the target task, while eliminating the need for labeling auxiliary tasks.
  arXiv  Detail & Related papers  (2022-10-13T17:59:16Z)
• Task Formulation Matters When Learning Continually: A Case Study in Visual Question Answering [58.82325933356066]
  Continual learning aims to train a model incrementally on a sequence of tasks without forgetting previous knowledge. We present a detailed study of how different settings affect performance for Visual Question Answering.
  arXiv  Detail & Related papers  (2022-09-30T19:12:58Z)
• An Empirical Investigation of Representation Learning for Imitation [76.48784376425911]
  Recent work in vision, reinforcement learning, and NLP has shown that auxiliary representation learning objectives can reduce the need for large amounts of expensive, task-specific data. We propose a modular framework for constructing representation learning algorithms, then use our framework to evaluate the utility of representation learning for imitation.
  arXiv  Detail & Related papers  (2022-05-16T11:23:42Z)
• Robust Representation Learning via Perceptual Similarity Metrics [18.842322467828502]
  Contrastive Input Morphing (CIM) is a representation learning framework that learns input-space transformations of the data. We show that CIM is complementary to other mutual information-based representation learning techniques.
  arXiv  Detail & Related papers  (2021-06-11T21:45:44Z)
• Learning to Relate Depth and Semantics for Unsupervised Domain Adaptation [87.1188556802942]
  We present an approach for encoding visual task relationships to improve model performance in an Unsupervised Domain Adaptation (UDA) setting. We propose a novel Cross-Task Relation Layer (CTRL), which encodes task dependencies between the semantic and depth predictions. Furthermore, we propose an Iterative Self-Learning (ISL) training scheme, which exploits semantic pseudo-labels to provide extra supervision on the target domain.
  arXiv  Detail & Related papers  (2021-05-17T13:42:09Z)
• Return-Based Contrastive Representation Learning for Reinforcement Learning [126.7440353288838]
  We propose a novel auxiliary task that forces the learnt representations to discriminate state-action pairs with different returns. Our algorithm outperforms strong baselines on complex tasks in Atari games and DeepMind Control suite.
  arXiv  Detail & Related papers  (2021-02-22T13:04:18Z)
• Towards Generalising Neural Implicit Representations [15.728196666021665]
  We argue that training neural representations for both reconstruction tasks, alongside conventional tasks, can produce more general encodings. Our approach learns feature rich encodings that produce high quality results for each task. We also reformulate the segmentation task, creating a more representative challenge for implicit representation contexts.
  arXiv  Detail & Related papers  (2021-01-29T17:20:22Z)
• Analyzing Visual Representations in Embodied Navigation Tasks [45.35107294831313]
  We use the recently proposed projection weighted Canonical Correlation Analysis (PWCCA) to measure the similarity of visual representations learned in the same environment by performing different tasks. We then empirically demonstrate that visual representations learned on one task can be effectively transferred to a different task.
  arXiv  Detail & Related papers  (2020-03-12T19:43:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.