Semantic Prioritization in Visual Counterfactual Explanations with Weighted Segmentation and Auto-Adaptive Region Selection

• URL: http://arxiv.org/abs/2511.12992v1
• Date: Mon, 17 Nov 2025 05:34:10 GMT
• Title: Semantic Prioritization in Visual Counterfactual Explanations with Weighted Segmentation and Auto-Adaptive Region Selection
• Authors: Lintong Zhang, Kang Yin, Seong-Whan Lee,
• Abstract summary: This study introduces an innovative methodology named as Weighted Semantic Map with Auto-adaptive Candidate Editing Network (WSAE-Net)<n>The generation of the weighted semantic map is designed to maximize the reduction of non-semantic feature units that need to be computed.<n>The auto-adaptive candidate editing sequences are designed to determine the optimal computational order among the feature units to be processed.
• Score: 50.68751788132789
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In the domain of non-generative visual counterfactual explanations (CE), traditional techniques frequently involve the substitution of sections within a query image with corresponding sections from distractor images. Such methods have historically overlooked the semantic relevance of the replacement regions to the target object, thereby impairing the model's interpretability and hindering the editing workflow. Addressing these challenges, the present study introduces an innovative methodology named as Weighted Semantic Map with Auto-adaptive Candidate Editing Network (WSAE-Net). Characterized by two significant advancements: the determination of an weighted semantic map and the auto-adaptive candidate editing sequence. First, the generation of the weighted semantic map is designed to maximize the reduction of non-semantic feature units that need to be computed, thereby optimizing computational efficiency. Second, the auto-adaptive candidate editing sequences are designed to determine the optimal computational order among the feature units to be processed, thereby ensuring the efficient generation of counterfactuals while maintaining the semantic relevance of the replacement feature units to the target object. Through comprehensive experimentation, our methodology demonstrates superior performance, contributing to a more lucid and in-depth understanding of visual counterfactual explanations.

Related papers

• Masked Feature Modeling Enhances Adaptive Segmentation [9.279607578922683]
  Masked Feature Modeling (MFM) is a novel auxiliary task that performs feature masking and reconstruction directly in the feature space.<n>MFM aligns its learning target with the main segmentation task, ensuring compatibility with standard architectures like DeepLab and DAFormer.<n>To facilitate effective reconstruction, we introduce a lightweight auxiliary module, Rebuilder, which is trained jointly but discarded during inference.
  arXiv  Detail & Related papers  (2025-09-17T08:16:05Z)
• Seeing What Matters: Empowering CLIP with Patch Generation-to-Selection [54.21851618853518]
  We present a concise yet effective approach called Patch Generation-to-Selection to enhance CLIP's training efficiency.<n>Our approach, CLIP-PGS, sets new state-of-the-art results in zero-shot classification and retrieval tasks.
  arXiv  Detail & Related papers  (2025-03-21T12:10:38Z)
• PseudoNeg-MAE: Self-Supervised Point Cloud Learning using Conditional Pseudo-Negative Embeddings [55.55445978692678]
  PseudoNeg-MAE enhances global feature representation of point cloud masked autoencoders by making them both discriminative and sensitive to transformations.<n>We propose a novel loss that explicitly penalizes invariant collapse, enabling the network to capture richer transformation cues while preserving discriminative representations.
  arXiv  Detail & Related papers  (2024-09-24T07:57:21Z)
• Utilizing Graph Generation for Enhanced Domain Adaptive Object Detection [1.9846381198210663]
  The problem of Domain Adaptive in the field of Object Detection involves the transfer of object detection models from labeled source domains to unannotated target domains. Recent advancements in this field aim to address domain discrepancies by aligning pixel-pairs across domains within a non-Euclidean graphical space. Despite their remarkable achievements, these methods often use coarse semantic representations to model graphs.
  arXiv  Detail & Related papers  (2024-04-23T03:11:08Z)
• Continual-MAE: Adaptive Distribution Masked Autoencoders for Continual Test-Time Adaptation [49.827306773992376]
  Continual Test-Time Adaptation (CTTA) is proposed to migrate a source pre-trained model to continually changing target distributions. Our proposed method attains state-of-the-art performance in both classification and segmentation CTTA tasks.
  arXiv  Detail & Related papers  (2023-12-19T15:34:52Z)
• Incremental Image Labeling via Iterative Refinement [4.7590051176368915]
  In particular, the existence of the semantic gap problem leads to a many-to-many mapping between the information extracted from an image and its linguistic description. This unavoidable bias further leads to poor performance on current computer vision tasks. We introduce a Knowledge Representation (KR)-based methodology to provide guidelines driving the labeling process.
  arXiv  Detail & Related papers  (2023-04-18T13:37:22Z)
• Labeling Where Adapting Fails: Cross-Domain Semantic Segmentation with Point Supervision via Active Selection [81.703478548177]
  Training models dedicated to semantic segmentation require a large amount of pixel-wise annotated data. Unsupervised domain adaptation approaches aim at aligning the feature distributions between the labeled source and the unlabeled target data. Previous works attempted to include human interactions in this process under the form of sparse single-pixel annotations in the target data. We propose a new domain adaptation framework for semantic segmentation with annotated points via active selection.
  arXiv  Detail & Related papers  (2022-06-01T01:52:28Z)
• Generalizing Interactive Backpropagating Refinement for Dense Prediction [0.0]
  We introduce a set of G-BRS layers that enable both global and localized refinement for a range of dense prediction tasks. Our method can successfully generalize and significantly improve performance of existing pretrained state-of-the-art models with only a few clicks.
  arXiv  Detail & Related papers  (2021-12-21T03:52:08Z)
• Latent Space Regularization for Unsupervised Domain Adaptation in Semantic Segmentation [14.050836886292869]
  We introduce feature-level space-shaping regularization strategies to reduce the domain discrepancy in semantic segmentation. We verify the effectiveness of such methods in the autonomous driving setting.
  arXiv  Detail & Related papers  (2021-04-06T16:07:22Z)
• Efficient Semantic Image Synthesis via Class-Adaptive Normalization [116.63715955932174]
  Class-adaptive normalization (CLADE) is a lightweight but equally-effective variant that is only adaptive to semantic class. We introduce intra-class positional map encoding calculated from semantic layouts to modulate the normalization parameters of CLADE. The proposed CLADE can be generalized to different SPADE-based methods while achieving comparable generation quality compared to SPADE.
  arXiv  Detail & Related papers  (2020-12-08T18:59:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.