Task-aware Distributed Source Coding under Dynamic Bandwidth
- URL: http://arxiv.org/abs/2305.15523v3
- Date: Fri, 13 Oct 2023 17:47:41 GMT
- Title: Task-aware Distributed Source Coding under Dynamic Bandwidth
- Authors: Po-han Li, Sravan Kumar Ankireddy, Ruihan Zhao, Hossein Nourkhiz
Mahjoub, Ehsan Moradi-Pari, Ufuk Topcu, Sandeep Chinchali, Hyeji Kim
- Abstract summary: We propose a distributed compression framework composed of independent encoders and a joint decoder, which we call neural distributed principal component analysis (NDPCA)
NDPCA flexibly compresses data from multiple sources to any available bandwidth with a single model, reducing computing and storage overhead.
Experiments show that NDPCA improves the success rate of multi-view robotic arm manipulation by 9% and the accuracy of object detection tasks on satellite imagery by 14%.
- Score: 24.498190179263837
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Efficient compression of correlated data is essential to minimize
communication overload in multi-sensor networks. In such networks, each sensor
independently compresses the data and transmits them to a central node due to
limited communication bandwidth. A decoder at the central node decompresses and
passes the data to a pre-trained machine learning-based task to generate the
final output. Thus, it is important to compress the features that are relevant
to the task. Additionally, the final performance depends heavily on the total
available bandwidth. In practice, it is common to encounter varying
availability in bandwidth, and higher bandwidth results in better performance
of the task. We design a novel distributed compression framework composed of
independent encoders and a joint decoder, which we call neural distributed
principal component analysis (NDPCA). NDPCA flexibly compresses data from
multiple sources to any available bandwidth with a single model, reducing
computing and storage overhead. NDPCA achieves this by learning low-rank task
representations and efficiently distributing bandwidth among sensors, thus
providing a graceful trade-off between performance and bandwidth. Experiments
show that NDPCA improves the success rate of multi-view robotic arm
manipulation by 9% and the accuracy of object detection tasks on satellite
imagery by 14% compared to an autoencoder with uniform bandwidth allocation.
Related papers
- Distributed Semantic Segmentation with Efficient Joint Source and Task Decoding [18.8803233019656]
Deep neural networks (DNNs) execute one part of the network on edge devices and the other part on a large-scale cloud platform.
In this work, we propose joint source and task decoding, as it allows for a smaller network size in the cloud.
We demonstrate the effectiveness of our method by achieving a distributed semantic segmentation SOTA over a wide range of intersections.
arXiv Detail & Related papers (2024-07-15T20:20:04Z) - A Point-Based Approach to Efficient LiDAR Multi-Task Perception [49.91741677556553]
PAttFormer is an efficient multi-task architecture for joint semantic segmentation and object detection in point clouds.
Unlike other LiDAR-based multi-task architectures, our proposed PAttFormer does not require separate feature encoders for task-specific point cloud representations.
Our evaluations show substantial gains from multi-task learning, improving LiDAR semantic segmentation by +1.7% in mIou and 3D object detection by +1.7% in mAP.
arXiv Detail & Related papers (2024-04-19T11:24:34Z) - Enabling robust sensor network design with data processing and
optimization making use of local beehive image and video files [0.0]
We of er a revolutionary paradigm that uses cutting-edge edge computing techniques to optimize data transmission and storage.
Our approach encompasses data compression for images and videos, coupled with a data aggregation technique for numerical data.
A key aspect of our approach is its ability to operate in resource-constrained environments.
arXiv Detail & Related papers (2024-02-26T15:27:47Z) - Attention-based Feature Compression for CNN Inference Offloading in Edge
Computing [93.67044879636093]
This paper studies the computational offloading of CNN inference in device-edge co-inference systems.
We propose a novel autoencoder-based CNN architecture (AECNN) for effective feature extraction at end-device.
Experiments show that AECNN can compress the intermediate data by more than 256x with only about 4% accuracy loss.
arXiv Detail & Related papers (2022-11-24T18:10:01Z) - Bandwidth-efficient distributed neural network architectures with
application to body sensor networks [73.02174868813475]
This paper describes a conceptual design methodology to design distributed neural network architectures.
We show that the proposed framework enables up to a factor 20 in bandwidth reduction with minimal loss.
While the application focus of this paper is on wearable brain-computer interfaces, the proposed methodology can be applied in other sensor network-like applications as well.
arXiv Detail & Related papers (2022-10-14T12:35:32Z) - An Adaptive Device-Edge Co-Inference Framework Based on Soft
Actor-Critic [72.35307086274912]
High-dimension parameter model and large-scale mathematical calculation restrict execution efficiency, especially for Internet of Things (IoT) devices.
We propose a new Deep Reinforcement Learning (DRL)-Soft Actor Critic for discrete (SAC-d), which generates the emphexit point, emphexit point, and emphcompressing bits by soft policy iterations.
Based on the latency and accuracy aware reward design, such an computation can well adapt to the complex environment like dynamic wireless channel and arbitrary processing, and is capable of supporting the 5G URL
arXiv Detail & Related papers (2022-01-09T09:31:50Z) - Federated Split Vision Transformer for COVID-19 CXR Diagnosis using
Task-Agnostic Training [28.309185925167565]
Federated learning enables neural network training for COVID-19 diagnosis on chest X-ray (CXR) images without collecting patient CXR data across multiple hospitals.
We show that Vision Transformer, a recently developed deep learning architecture with straightforward decomposable configuration, is ideally suitable for split learning without sacrificing performance.
Our results affirm the suitability of Transformer for collaborative learning in medical imaging and pave the way forward for future real-world implementations.
arXiv Detail & Related papers (2021-11-02T02:54:30Z) - Supervised Compression for Resource-constrained Edge Computing Systems [26.676557573171618]
Full-scale deep neural networks are often too resource-intensive in terms of energy and storage.
This paper adopts ideas from knowledge distillation and neural image compression to compress intermediate feature representations more efficiently.
It achieves better supervised rate-distortion performance while also maintaining smaller end-to-end latency.
arXiv Detail & Related papers (2021-08-21T11:10:29Z) - End-to-end optimized image compression for machines, a study [3.0448872422956437]
An increasing share of image and video content is analyzed by machines rather than viewed by humans.
Conventional coding tools are challenging to specialize for machine tasks as they were originally designed for human perception.
neural network based codecs can be jointly trained end-to-end with any convolutional neural network (CNN)-based task model.
arXiv Detail & Related papers (2020-11-10T20:10:43Z) - PowerGossip: Practical Low-Rank Communication Compression in
Decentralized Deep Learning [62.440827696638664]
We introduce a simple algorithm that directly compresses the model differences between neighboring workers.
Inspired by the PowerSGD for centralized deep learning, this algorithm uses power steps to maximize the information transferred per bit.
arXiv Detail & Related papers (2020-08-04T09:14:52Z) - Joint Parameter-and-Bandwidth Allocation for Improving the Efficiency of
Partitioned Edge Learning [73.82875010696849]
Machine learning algorithms are deployed at the network edge for training artificial intelligence (AI) models.
This paper focuses on the novel joint design of parameter (computation load) allocation and bandwidth allocation.
arXiv Detail & Related papers (2020-03-10T05:52:15Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.