Multiscale Feature Importance-based Bit Allocation for End-to-End Feature Coding for Machines

• URL: http://arxiv.org/abs/2503.19278v1
• Date: Tue, 25 Mar 2025 02:19:04 GMT
• Title: Multiscale Feature Importance-based Bit Allocation for End-to-End Feature Coding for Machines
• Abstract summary: Feature Coding for Machines (FCM) aims to compress intermediate features effectively for remote intelligent analytics.<n>We propose a Multiscale Feature Importance-based Bit Allocation (MFIBA) for end-to-end FCM.
• Score: 2.5667927142354907
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Feature Coding for Machines (FCM) aims to compress intermediate features effectively for remote intelligent analytics, which is crucial for future intelligent visual applications. In this paper, we propose a Multiscale Feature Importance-based Bit Allocation (MFIBA) for end-to-end FCM. First, we find that the importance of features for machine vision tasks varies with the scales, object size, and image instances. Based on this finding, we propose a Multiscale Feature Importance Prediction (MFIP) module to predict the importance weight for each scale of features. Secondly, we propose a task loss-rate model to establish the relationship between the task accuracy losses of using compressed features and the bitrate of encoding these features. Finally, we develop a MFIBA for end-to-end FCM, which is able to assign coding bits of multiscale features more reasonably based on their importance. Experimental results demonstrate that when combined with a retained Efficient Learned Image Compression (ELIC), the proposed MFIBA achieves an average of 38.202% bitrate savings in object detection compared to the anchor ELIC. Moreover, the proposed MFIBA achieves an average of 17.212% and 36.492% feature bitrate savings for instance segmentation and keypoint detection, respectively. When the proposed MFIBA is applied to the LIC-TCM, it achieves an average of 18.103%, 19.866% and 19.597% bit rate savings on three machine vision tasks, respectively, which validates the proposed MFIBA has good generalizability and adaptability to different machine vision tasks and FCM base codecs.

Related papers

• Towards Large Model Feature Coding [90.89390892560094]
  We present a benchmark and evaluation framework for large model feature coding (LaMoFC)<n>We first build the feature dataset LaMoFCBench, covering diverse task requirements across 4 categories and 16 scenarios.<n>We then specify representative split points according to practical application scenarios to extract intermediate features.
  arXiv  Detail & Related papers  (2026-05-20T13:39:48Z)
• CI-ICM: Channel Importance-driven Learned Image Coding for Machines [56.1188469769241]
  We propose a Channel Importance-driven learned Image Coding for Machines (CI-ICM) to maximize the performance of machine vision tasks.<n>The proposed CI-ICM achieves BD-mAP@50:95 gains of 16.25%$ in object detection and 13.72%$ in instance segmentation over the established baseline.
  arXiv  Detail & Related papers  (2026-04-07T02:36:47Z)
• Efficient Leaf Disease Classification and Segmentation using Midpoint Normalization Technique and Attention Mechanism [0.0]
  We introduce a transformative two-stage methodology, Mid Point Normalization (MPN) for intelligent image preprocessing.<n>Our classification pipeline achieves 93% accuracy while maintaining exceptional class-wise balance.<n>For segmentation tasks, we seamlessly integrate identical attention blocks within U-Net architecture using MPN-enhanced inputs.
  arXiv  Detail & Related papers  (2025-05-27T15:14:04Z)
• Benchmarking Feature Upsampling Methods for Vision Foundation Models using Interactive Segmentation [24.531539125814877]
  Vision Foundation Models (VFMs) are large-scale, pre-trained models that serve as general-purpose backbones for various computer vision tasks.<n>One way to tackle this limitation is by employing a task-agnostic feature upsampling module that refines VFM features resolution.<n>Our benchmarking experiments show that selecting appropriate upsampling strategies significantly improves VFM features quality.
  arXiv  Detail & Related papers  (2025-05-04T11:59:26Z)
• Task-Oriented Feature Compression for Multimodal Understanding via Device-Edge Co-Inference [54.53508601749513]
  We propose a task-oriented feature compression (TOFC) method for multimodal understanding in a device-edge co-inference framework.<n>To enhance compression efficiency, multiple entropy models are adaptively selected based on the characteristics of the visual features.<n>Results show that TOFC achieves up to 52% reduction in data transmission overhead and 63% reduction in system latency.
  arXiv  Detail & Related papers  (2025-03-17T08:37:22Z)
• Resource-Efficient Affordance Grounding with Complementary Depth and Semantic Prompts [21.435113588059924]
  Affordance refers to the functional properties that an agent perceives and utilizes from its environment.<n>Existing multimodal affordance methods face limitations in extracting useful information.<n>This paper proposes the BiT-Align image-depth-text affordance mapping framework.
  arXiv  Detail & Related papers  (2025-03-04T13:20:42Z)
• Towards Economical Inference: Enabling DeepSeek's Multi-Head Latent Attention in Any Transformer-based LLMs [74.74225314708225]
  Multi-head Latent Attention (MLA) is an innovative architecture designed to ensure efficient and economical inference.<n>This paper proposes the first data-efficient fine-tuning method for transitioning from Multi-Head Attention to MLA.
  arXiv  Detail & Related papers  (2025-02-20T18:50:42Z)
• MALT: Multi-scale Action Learning Transformer for Online Action Detection [6.819772961626024]
  Online action detection (OAD) aims to identify ongoing actions from streaming video in real-time, without access to future frames. We propose a multi-scale action learning transformer (MALT) that includes a novel recurrent decoder (used for feature fusion) We also introduce an explicit frame scoring mechanism employing sparse attention, which filters irrelevant frames more efficiently.
  arXiv  Detail & Related papers  (2024-05-31T15:03:35Z)
• LLMC: Benchmarking Large Language Model Quantization with a Versatile Compression Toolkit [55.73370804397226]
  Quantization, a key compression technique, can effectively mitigate these demands by compressing and accelerating large language models. We present LLMC, a plug-and-play compression toolkit, to fairly and systematically explore the impact of quantization. Powered by this versatile toolkit, our benchmark covers three key aspects: calibration data, algorithms (three strategies), and data formats.
  arXiv  Detail & Related papers  (2024-05-09T11:49:05Z)
• MSCMNet: Multi-scale Semantic Correlation Mining for Visible-Infrared Person Re-Identification [18.40238463083768]
  MSCMNet is proposed to comprehensively exploit semantic features at multiple scales. It simultaneously reduces modality information loss as small as possible in feature extraction. Extensive experiments on the SYSU-MM01, RegDB, and LLCM datasets demonstrate that the proposed MSCMNet achieves the greatest accuracy.
  arXiv  Detail & Related papers  (2023-11-24T10:23:57Z)
• Tuning Pre-trained Model via Moment Probing [62.445281364055795]
  We propose a novel Moment Probing (MP) method to explore the potential of LP. MP performs a linear classification head based on the mean of final features. Our MP significantly outperforms LP and is competitive with counterparts at less training cost.
  arXiv  Detail & Related papers  (2023-07-21T04:15:02Z)
• BOtied: Multi-objective Bayesian optimization with tied multivariate ranks [33.414682601242006]
  In this paper, we show a natural connection between non-dominated solutions and the extreme quantile of the joint cumulative distribution function. Motivated by this link, we propose the Pareto-compliant CDF indicator and the associated acquisition function, BOtied. Our experiments on a variety of synthetic and real-world problems demonstrate that BOtied outperforms state-of-the-art MOBO acquisition functions.
  arXiv  Detail & Related papers  (2023-06-01T04:50:06Z)
• SafeSpace MFNet: Precise and Efficient MultiFeature Drone Detection Network [12.221253007423394]
  MultiFeatureNet (MFNet) is a solution that enhances feature representation by capturing the most concentrated feature maps. MFNet-FA is a technique that adaptively weights different channels of the input feature maps. The proposed MFNet-L with the focus module (Ablation study 2) demonstrates the most remarkable classification outcomes.
  arXiv  Detail & Related papers  (2022-11-30T06:56:39Z)
• Transformer-based Context Condensation for Boosting Feature Pyramids in Object Detection [77.50110439560152]
  Current object detectors typically have a feature pyramid (FP) module for multi-level feature fusion (MFF) We propose a novel and efficient context modeling mechanism that can help existing FPs deliver better MFF results. In particular, we introduce a novel insight that comprehensive contexts can be decomposed and condensed into two types of representations for higher efficiency.
  arXiv  Detail & Related papers  (2022-07-14T01:45:03Z)
• S$^2$-FPN: Scale-ware Strip Attention Guided Feature Pyramid Network for Real-time Semantic Segmentation [6.744210626403423]
  This paper presents a new model to achieve a trade-off between accuracy/speed for real-time road scene semantic segmentation. Specifically, we proposed a lightweight model named Scale-aware Strip Attention Guided Feature Pyramid Network (S$2$-FPN) Our network consists of three main modules: Attention Pyramid Fusion (APF) module, Scale-aware Strip Attention Module (SSAM), and Global Feature Upsample (GFU) module.
  arXiv  Detail & Related papers  (2022-06-15T05:02:49Z)
• EPMF: Efficient Perception-aware Multi-sensor Fusion for 3D Semantic Segmentation [62.210091681352914]
  We study multi-sensor fusion for 3D semantic segmentation for many applications, such as autonomous driving and robotics. In this work, we investigate a collaborative fusion scheme called perception-aware multi-sensor fusion (PMF) We propose a two-stream network to extract features from the two modalities separately. The extracted features are fused by effective residual-based fusion modules.
  arXiv  Detail & Related papers  (2021-06-21T10:47:26Z)
• CARAFE++: Unified Content-Aware ReAssembly of FEatures [132.49582482421246]
  We propose unified Content-Aware ReAssembly of FEatures (CARAFE++), a universal, lightweight and highly effective operator to fulfill this goal. CARAFE++ generates adaptive kernels on-the-fly to enable instance-specific content-aware handling. It shows consistent and substantial gains across all the tasks with negligible computational overhead.
  arXiv  Detail & Related papers  (2020-12-07T07:34:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.