Foundation Models for Generalizable Semantic and Goal-Oriented Communication

• URL: http://arxiv.org/abs/2609.07853v1
• Date: Mon, 07 Sep 2026 18:12:39 GMT
• Title: Foundation Models for Generalizable Semantic and Goal-Oriented Communication
• Abstract summary: We introduce Foundation Model-Guided Semantic and Goal-Oriented Communication (FMSGOC)<n>It improves rate efficiency by concentrating bits on sparse, goal-aligned anchors and relying on generative foundation-model priors to reconstruct the masked regions.<n>It maintains high semantic fidelity (cosine similarity 0.87-0.90 on CIFAR-10), remains robust on previously unseen inputs (0.83-0.86 on ImageNet), and shows good perceptual similarity (0.1278/0.1558, CIFAR-10/ImageNet)
• Score: 42.78591555984395
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Semantic and goal-oriented communication is increasingly studied for 6G, but generalization beyond seen data remains a key weakness under tight rate budgets. Many existing systems overfit their training data and degrade sharply at very low bit rates because they attempt to compress the entire signal. We introduce Foundation Model-Guided Semantic and Goal-Oriented Communication (FMSGOC), a framework that uses broad visual-linguistic Foundation Model priors to mitigate overfitting. It further improves rate efficiency by concentrating bits on sparse, goal-aligned anchors and relying on generative foundation-model priors to reconstruct the masked regions. By decoupling what to send from how to reconstruct, a vision-language foundation model selects and transmits a sparse set of semantic anchors, while a pretrained diffusion model, fine-tuned for masked completion, reconstructs the image at the receiver. In our experiments, FMSGOC reaches 0.039 bits per pixel (BPP), maintains high semantic fidelity (cosine similarity 0.87-0.90 on CIFAR-10), remains robust on previously unseen inputs (0.83-0.86 on ImageNet), and shows good perceptual similarity (0.1278/0.1558, CIFAR-10/ImageNet), outperforming strong end-to-end baselines at lower bit rates.

Related papers

• A Frozen Pixel-Space Diffusion Model Can Guide Itself with Its Own Samples [53.8898272536272]
  Pixel-space diffusion models aim to learn an end-to-end generator directly over raw pixels.<n>Recent work improves pixel diffusion through alternative prediction targets, training objectives, and architectures.<n>We show there is a cheaper, complementary strategy: textbfa frozen, pretrained pixel diffusion model can guide itself
  arXiv  Detail & Related papers  (2026-07-31T07:52:08Z)
• ZID-Net: Zero-Inference Diffusion Prior Decoupling Network for Single Image Dehazing [10.634107398163676]
  Single image dehazing is often constrained by a trade-off between restoration quality and computational efficiency.<n>ZID-Net is a novel framework that explicitly decouples diffusion supervision from feed-forward inference.<n>ZID-Net achieves 40.75 dB PSNR on the synthetic RESIDE dataset and outperforms existing methods with a 1.13 dB gain on real-world datasets.
  arXiv  Detail & Related papers  (2026-04-26T13:44:25Z)
• VFM-Recon: Unlocking Cross-Domain Scene-Level Neural Reconstruction with Scale-Aligned Foundation Priors [49.39553550491549]
  We present VFMRecon, the first attempt to bridge transferable VFM priors with scaleconsistent requirements in scene-level neural reconstruction.<n> Specifically, we first introduce a lightweight scale alignment stage that restores multiview scale coherence.<n>We then integrate pretrained VFM features into the neural volumetric reconstruction pipeline via lightweight task-specific adapters.
  arXiv  Detail & Related papers  (2026-03-13T05:00:44Z)
• CORE: Context-Robust Remasking for Diffusion Language Models [51.59514489363897]
  We propose Context-Robust Remasking (CORE), a training-free framework for inference-time revision.<n>Rather than trusting static token probabilities, CORE identifies context-brittle tokens by probing their sensitivity to targeted masked-context perturbations.<n>On LLaDA-8B-Base, CORE delivers consistent improvements across reasoning and code benchmarks, outperforming compute-matched baselines and improving MBPP by up to 9.2 percentage points.
  arXiv  Detail & Related papers  (2026-02-04T00:12:30Z)
• Deep Leakage with Generative Flow Matching Denoiser [54.05993847488204]
  We introduce a new deep leakage (DL) attack that integrates a generative Flow Matching (FM) prior into the reconstruction process.<n>Our approach consistently outperforms state-of-the-art attacks across pixel-level, perceptual, and feature-based similarity metrics.
  arXiv  Detail & Related papers  (2026-01-21T14:51:01Z)
• Model Inversion Attacks Through Target-Specific Conditional Diffusion Models [54.69008212790426]
  Model inversion attacks (MIAs) aim to reconstruct private images from a target classifier's training set, thereby raising privacy concerns in AI applications. Previous GAN-based MIAs tend to suffer from inferior generative fidelity due to GAN's inherent flaws and biased optimization within latent space. We propose Diffusion-based Model Inversion (Diff-MI) attacks to alleviate these issues.
  arXiv  Detail & Related papers  (2024-07-16T06:38:49Z)
• AttenScribble: Attentive Similarity Learning for Scribble-Supervised Medical Image Segmentation [5.8447004333496855]
  In this paper, we present a straightforward yet effective scribble supervised learning framework. We create a pluggable spatial self-attention module which could be attached on top of any internal feature layers of arbitrary fully convolutional network (FCN) backbone. This attentive similarity leads to a novel regularization loss that imposes consistency between segmentation prediction and visual affinity.
  arXiv  Detail & Related papers  (2023-12-11T18:42:18Z)
• Contrastive Adapters for Foundation Model Group Robustness [11.961187296775185]
  Existing methods to improve robustness require retraining, which can be prohibitively expensive on large foundation models. We propose contrastive adapting, which trains adapters with contrastive learning to bring sample embeddings close to both their ground-truth class embeddings and other sample embeddings in the same class. Our approach consistently improves group robustness, raising worst-group accuracy by 8.5 to 56.0 pp over zero-shot.
  arXiv  Detail & Related papers  (2022-07-14T19:40:55Z)
• Adaptive Clustering of Robust Semantic Representations for Adversarial Image Purification [0.9203366434753543]
  We propose a robust defense against adversarial attacks, which is model agnostic and generalizable to unseen adversaries. In this paper, we extract the latent representations for each class and adaptively cluster the latent representations that share a semantic similarity. We adversarially train a new model constraining the latent space representation to minimize the distance between the adversarial latent representation and the true cluster distribution.
  arXiv  Detail & Related papers  (2021-04-05T21:07:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.