Deep spatial context: when attention-based models meet spatial regression

• URL: http://arxiv.org/abs/2401.10044v2
• Date: Sun, 10 Mar 2024 16:37:32 GMT
• Title: Deep spatial context: when attention-based models meet spatial regression
• Authors: Paulina Tomaszewska, El\.zbieta Sienkiewicz, Mai P. Hoang, Przemys{\l}aw Biecek
• Abstract summary: 'Deep spatial context' (DSCon) method serves for investigation of the attention-based vision models using the concept of spatial context. It was inspired by histopathologists, however, the method can be applied to various domains.
• Score: 8.90978723839271
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose 'Deep spatial context' (DSCon) method, which serves for investigation of the attention-based vision models using the concept of spatial context. It was inspired by histopathologists, however, the method can be applied to various domains. The DSCon allows for a quantitative measure of the spatial context's role using three Spatial Context Measures: $SCM_{features}$, $SCM_{targets}$, $SCM_{residuals}$ to distinguish whether the spatial context is observable within the features of neighboring regions, their target values (attention scores) or residuals, respectively. It is achieved by integrating spatial regression into the pipeline. The DSCon helps to verify research questions. The experiments reveal that spatial relationships are much bigger in the case of the classification of tumor lesions than normal tissues. Moreover, it turns out that the larger the size of the neighborhood taken into account within spatial regression, the less valuable contextual information is. Furthermore, it is observed that the spatial context measure is the largest when considered within the feature space as opposed to the targets and residuals.

Related papers

• QueST: Querying Functional and Structural Niches on Spatial Transcriptomics Data via Contrastive Subgraph Embedding [6.579597615392464]
  QueST is a novel niche representation learning model designed for querying spatial niches across multiple samples. We evaluate QueST on established benchmarks using human and mouse datasets. QueST offers a specialized model for spatial niche queries, paving the way for deeper insights into the patterns and mechanisms of cell spatial organization across tissues.
  arXiv  Detail & Related papers  (2024-10-14T16:01:27Z)
• SpaRC and SpaRP: Spatial Reasoning Characterization and Path Generation for Understanding Spatial Reasoning Capability of Large Language Models [70.01883340129204]
  spatial reasoning is a crucial component of both biological and artificial intelligence. We present a comprehensive study of the capability of current state-of-the-art large language models (LLMs) on spatial reasoning.
  arXiv  Detail & Related papers  (2024-06-07T01:06:34Z)
• SpatialRGPT: Grounded Spatial Reasoning in Vision Language Models [68.13636352687257]
  We introduce Spatial Region GPT (SpatialRGPT) to enhance VLMs' spatial perception and reasoning capabilities. During inference, when provided with user-specified region proposals, SpatialRGPT can accurately perceive their relative directions and distances. Our results demonstrate that SpatialRGPT significantly enhances performance in spatial reasoning tasks, both with and without local region prompts.
  arXiv  Detail & Related papers  (2024-06-03T17:59:06Z)
• Reframing Spatial Reasoning Evaluation in Language Models: A Real-World Simulation Benchmark for Qualitative Reasoning [4.422649561583363]
  We present a novel benchmark for assessing spatial reasoning in language models (LMs) It is grounded in realistic 3D simulation data, offering a series of diverse room layouts with various objects and their spatial relationships. A key contribution is our logic-based consistency-checking tool, which enables the assessment of multiple plausible solutions.
  arXiv  Detail & Related papers  (2024-05-23T21:22:00Z)
• Towards Spatially-Lucid AI Classification in Non-Euclidean Space: An Application for MxIF Oncology Data [3.0566763412020714]
  Given multi-category point sets from different place-types, our goal is to develop a spatially-lucid classifier. This problem is important for many applications, such as oncology, for analyzing immune-tumor relationships and designing new immunotherapies. We explore a spatial ensemble framework that explicitly uses different training strategies, including weighted-distance learning rate and spatial domain adaptation.
  arXiv  Detail & Related papers  (2024-02-22T21:22:21Z)
• Deep Spatial Domain Generalization [8.102110157532556]
  We develop the spatial graph neural network that handles spatial data as a graph and learns the spatial embedding on each node. The proposed method infers the spatial embedding of an unseen location during the test phase and decodes the parameters of the downstream-task model directly on the target location.
  arXiv  Detail & Related papers  (2022-10-03T06:16:20Z)
• Analyzing the Latent Space of GAN through Local Dimension Estimation [4.688163910878411]
  style-based GANs (StyleGANs) in high-fidelity image synthesis have motivated research to understand the semantic properties of their latent spaces. We propose a local dimension estimation algorithm for arbitrary intermediate layers in a pre-trained GAN model. Our proposed metric, called Distortion, measures an inconsistency of intrinsic space on the learned latent space.
  arXiv  Detail & Related papers  (2022-05-26T06:36:06Z)
• Incorporating Texture Information into Dimensionality Reduction for High-Dimensional Images [65.74185962364211]
  We present a method for incorporating neighborhood information into distance-based dimensionality reduction methods. Based on a classification of different methods for comparing image patches, we explore a number of different approaches.
  arXiv  Detail & Related papers  (2022-02-18T13:17:43Z)
• The Geometry of Robust Value Functions [119.94715309072983]
  We introduce a new perspective that enables us to characterize both the non-robust and robust value space. We show that the robust value space is determined by a set conic hypersurfaces, each which contains the robust values of all policies that agree on one state.
  arXiv  Detail & Related papers  (2022-01-30T22:12:17Z)
• SIRI: Spatial Relation Induced Network For Spatial Description Resolution [64.38872296406211]
  We propose a novel relationship induced (SIRI) network for language-guided localization. We show that our method is around 24% better than the state-of-the-art method in terms of accuracy, measured by an 80-pixel radius. Our method also generalizes well on our proposed extended dataset collected using the same settings as Touchdown.
  arXiv  Detail & Related papers  (2020-10-27T14:04:05Z)
• From Spatial Relations to Spatial Configurations [64.21025426604274]
  spatial relation language is able to represent a large, comprehensive set of spatial concepts crucial for reasoning. We show how we extend the capabilities of existing spatial representation languages with the fine-grained decomposition of semantics.
  arXiv  Detail & Related papers  (2020-07-19T02:11:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.