Hasse Diagrams for Attention: A Partial Order Framework for Designing Transformer Masks

• URL: http://arxiv.org/abs/2606.09951v1
• Date: Mon, 08 Jun 2026 09:27:47 GMT
• Title: Hasse Diagrams for Attention: A Partial Order Framework for Designing Transformer Masks
• Abstract summary: This paper develops a complete theoretical framework for arbitrary attention masks.<n>We show that the information flow of a Transformer converges to a Hasse diagram -- a directed acyclic graph representing a partial order.<n>Applying the framework, we design two novel masks: a block-generation attention mask that ensures training-inference consistency, and a fully supervised bidirectional attention mask.
• Score: 9.276851330300621
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: During the training of large Transformer models, attention masks regulate the scope and direction of information flow across a sequence. Numerous mask variants exist, and operators such as FlexAttention already support arbitrary attention masks. Nevertheless, a systematic formal analysis of the information-flow structure induced by arbitrary masks has been missing. This paper develops a complete theoretical framework. We prove that, with sufficient depth, the information flow of a multi-layer Transformer converges to a Hasse diagram -- a directed acyclic graph representing a partial order. Building on this, we recast the design of parallel training tasks as the problem of finding a minimal common supergraph of Hasse diagrams, and we establish a criterion for the minimal common supergraph. This yields a constructive method to derive attention masks directly from a family of tasks. Applying the framework, we design two novel masks: a block-generation attention mask that ensures training-inference consistency (Block Two-Stream Attention), and a fully supervised bidirectional attention mask (Butterfly Attention). These results demonstrate the framework's capacity to discover new structures.

Related papers

• Unifying and Enhancing Graph Transformers via a Hierarchical Mask Framework [18.725415922303632]
  We propose a unified hierarchical mask framework that reveals an underlying equivalence between model architecture and attention mask construction.<n>This framework enables a consistent modeling paradigm by capturing diverse interactions through carefully designed attention masks.<n>We introduce M3Dphormer, a Mixture-of-Experts-based Graph Transformer with Multi-Level Masking and Dual Attention Computation.
  arXiv  Detail & Related papers  (2025-10-21T17:22:32Z)
• SeeDiff: Off-the-Shelf Seeded Mask Generation from Diffusion Models [16.109077391631917]
  We show that cross-attention alone provides very coarse object localization, which however can provide initial seeds.<n>We also observe that a simple-text-guided synthetic image often has a uniform background, which is easier to find correspondences.<n>Our proposed method, dubbed SeeDiff, generates high-quality masks off-the-shelf from Stable Diffusion.
  arXiv  Detail & Related papers  (2025-07-26T05:44:00Z)
• Evolved Hierarchical Masking for Self-Supervised Learning [49.77271430882176]
  Existing Masked Image Modeling methods apply fixed mask patterns to guide the self-supervised training.<n>This paper introduces an evolved hierarchical masking method to pursue general visual cues modeling in self-supervised learning.
  arXiv  Detail & Related papers  (2025-04-12T09:40:14Z)
• Adapting Vision-Language Model with Fine-grained Semantics for Open-Vocabulary Segmentation [42.020470627552136]
  Open-vocabulary segmentation is primarily bottlenecked by mask classification, not mask generation.<n>We propose a novel Fine-grained Semantic Adaptation (FISA) method to address this limitation.<n>FISA enhances the extracted visual features with fine-grained semantic awareness by explicitly integrating this crucial semantic information early in the visual encoding process.
  arXiv  Detail & Related papers  (2024-09-24T17:50:28Z)
• Efficiently Dispatching Flash Attention For Partially Filled Attention Masks [29.36452085947087]
  Transformers are widely used across various applications, many of which yield sparse or partially filled attention matrices. We introduce Binary Block Masking, a highly efficient modification that enhances Flash Attention by making it mask-aware. Our experiments on attention masks derived from real-world scenarios demonstrate up to a 9x runtime improvement.
  arXiv  Detail & Related papers  (2024-09-23T15:11:07Z)
• ColorMAE: Exploring data-independent masking strategies in Masked AutoEncoders [53.3185750528969]
  Masked AutoEncoders (MAE) have emerged as a robust self-supervised framework. We introduce a data-independent method, termed ColorMAE, which generates different binary mask patterns by filtering random noise. We demonstrate our strategy's superiority in downstream tasks compared to random masking.
  arXiv  Detail & Related papers  (2024-07-17T22:04:00Z)
• UGMAE: A Unified Framework for Graph Masked Autoencoders [67.75493040186859]
  We propose UGMAE, a unified framework for graph masked autoencoders. We first develop an adaptive feature mask generator to account for the unique significance of nodes. We then design a ranking-based structure reconstruction objective joint with feature reconstruction to capture holistic graph information.
  arXiv  Detail & Related papers  (2024-02-12T19:39:26Z)
• A Simple Latent Diffusion Approach for Panoptic Segmentation and Mask Inpainting [2.7563282688229664]
  This work builds upon Stable Diffusion and proposes a latent diffusion approach for panoptic segmentation. Our training consists of two steps: (1) training a shallow autoencoder to project the segmentation masks to latent space; (2) training a diffusion model to allow image-conditioned sampling in latent space.
  arXiv  Detail & Related papers  (2024-01-18T18:59:19Z)
• CL-MAE: Curriculum-Learned Masked Autoencoders [49.24994655813455]
  We propose a curriculum learning approach that updates the masking strategy to continually increase the complexity of the self-supervised reconstruction task. We train our Curriculum-Learned Masked Autoencoder (CL-MAE) on ImageNet and show that it exhibits superior representation learning capabilities compared to MAE.
  arXiv  Detail & Related papers  (2023-08-31T09:13:30Z)
• Mask-free OVIS: Open-Vocabulary Instance Segmentation without Manual Mask Annotations [86.47908754383198]
  Open-Vocabulary (OV) methods leverage large-scale image-caption pairs and vision-language models to learn novel categories. Our method generates pseudo-mask annotations by leveraging the localization ability of a pre-trained vision-language model for objects present in image-caption pairs. Our method trained with just pseudo-masks significantly improves the mAP scores on the MS-COCO dataset and OpenImages dataset.
  arXiv  Detail & Related papers  (2023-03-29T17:58:39Z)
• MaskSketch: Unpaired Structure-guided Masked Image Generation [56.88038469743742]
  MaskSketch is an image generation method that allows spatial conditioning of the generation result using a guiding sketch as an extra conditioning signal during sampling. We show that intermediate self-attention maps of a masked generative transformer encode important structural information of the input image. Our results show that MaskSketch achieves high image realism and fidelity to the guiding structure.
  arXiv  Detail & Related papers  (2023-02-10T20:27:02Z)
• Differentiable Soft-Masked Attention [115.5770357189209]
  "Differentiable Soft-Masked Attention" is used for the task of WeaklySupervised Video Object. We develop a transformer-based network for training, but can also benefit from cycle consistency training on a video with just one annotated frame.
  arXiv  Detail & Related papers  (2022-06-01T02:05:13Z)
• GANSeg: Learning to Segment by Unsupervised Hierarchical Image Generation [16.900404701997502]
  We propose a GAN-based approach that generates images conditioned on latent masks. We show that such mask-conditioned image generation can be learned faithfully when conditioning the masks in a hierarchical manner. It also lets us generate image-mask pairs for training a segmentation network, which outperforms the state-of-the-art unsupervised segmentation methods on established benchmarks.
  arXiv  Detail & Related papers  (2021-12-02T07:57:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.