2D Rotary Position Embedding for Scene Text Recognition with Transformers

• URL: http://arxiv.org/abs/2607.13458v1
• Date: Wed, 15 Jul 2026 05:38:24 GMT
• Title: 2D Rotary Position Embedding for Scene Text Recognition with Transformers
• Abstract summary: Axial 2D extensions of Rotary Position Embedding (RoPE) exist for vision Transformers, but they assume roughly square, isotropic image content and apply the rotation only within encoder self-attention.<n>We introduce 2D-RoPE-STR, which adapts axial 2D-RoPE to this setting through (1) an anisotropic row/column dimension allocation matched to the aspect ratio of text, and (2) an extension of the rotary coupling into encoder-decoder cross-attention.
• Score: 0.40611352512781856
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Scene Text Recognition (STR) remains challenging due to the diversity of text appearances, including curvature, rotation, and perspective distortion. Recent Transformer-based approaches perform well but usually rely on one-dimensional positional encodings that ignore the 2D spatial structure of text images. Axial 2D extensions of Rotary Position Embedding (RoPE) exist for vision Transformers, but they assume roughly square, isotropic image content and apply the rotation only within encoder self-attention. Scene text violates both assumptions: crops are markedly anisotropic, and STR models are encoder-decoder, so the decoder must relate its queries to the encoder's 2D layout through cross-attention. We introduce 2D-RoPE-STR, which adapts axial 2D-RoPE to this setting through (1) an anisotropic row/column dimension allocation matched to the aspect ratio of text, and (2) an extension of the rotary coupling into encoder-decoder cross-attention, letting autoregressive decoding steps attend to encoder tokens by their 2D layout, a setting not addressed by prior encoder-only formulations. Both changes are essentially parameter-free and require no architectural redesign beyond the positional-encoding module. We further introduce a diagnostic protocol (a controlled ablation pair isolating only the positional encoding, an image-level net-win disagreement analysis, and encoder attention visualization) that identifies where and why relative 2D position helps: curved, rotated, and perspective-distorted layouts where reading order departs from a straight horizontal line. On six standard benchmarks (IIIT5K, SVT, ICDAR 2013, ICDAR 2015, CUTE80, SVTP), gains concentrate on exactly these irregular layouts, with ablations isolating each design choice against 1D RoPE and 2D sinusoidal and learnable alternatives.

Related papers

• Embedding Rotation Invariance for Provable Multi-Oriented Scene Text Recognition [49.665177053925184]
  Multi-oriented text is a major challenge for scene text recognition.<n>Existing rotation-aware methods explicitly estimate text orientation.<n>We incorporate rotation invariance into the STR framework to address these limitations.
  arXiv  Detail & Related papers  (2026-08-11T09:06:46Z)
• RayPE: Ray-Space Positional Encoding for 3D-Aware Video Generation [43.253222907352146]
  RayPE is a positional-encoding extension that injects per-token 6D Plucker coordinates additively into the queries and keys of self-attention.<n>The resulting attention score decomposes into a content term, a geometry term, and two content and geometry cross-terms.
  arXiv  Detail & Related papers  (2026-06-25T17:51:02Z)
• URoPE: Universal Relative Position Embedding across Geometric Spaces [55.651792747815854]
  URoPE is a universal extension of Rotary Position Embedding (RoPE) to cross-view or cross-dimensional geometric spaces.<n>For each key/value image patch, URoPE samples 3D points along the corresponding camera ray at predefined depth anchors.<n>Standard 2D RoPE can then be applied using the projected pixel coordinates.
  arXiv  Detail & Related papers  (2026-04-20T18:52:03Z)
• Spiral RoPE: Rotate Your Rotary Positional Embeddings in the 2D Plane [49.14270539697387]
  Spiral RoPE is a simple yet effective extension that enables multi-directional positional encoding.<n>Across a wide range of vision tasks including classification, segmentation, and generation, Spiral RoPE consistently improves performance.
  arXiv  Detail & Related papers  (2026-02-03T07:56:58Z)
• SeqPE: Transformer with Sequential Position Encoding [76.22159277300891]
  SeqPE represents each $n$-dimensional position index as a symbolic sequence and employs a lightweight sequential position encoder to learn their embeddings.<n> Experiments across language modeling, long-context question answering, and 2D image classification demonstrate that SeqPE not only surpasses strong baselines in perplexity, exact match (EM) and accuracy--but also enables seamless generalization to multi-dimensional inputs without requiring manual architectural redesign.
  arXiv  Detail & Related papers  (2025-06-16T09:16:40Z)
• NDC-Scene: Boost Monocular 3D Semantic Scene Completion in Normalized Device Coordinates Space [77.6067460464962]
  Monocular 3D Semantic Scene Completion (SSC) has garnered significant attention in recent years due to its potential to predict complex semantics and geometry shapes from a single image, requiring no 3D inputs. We identify several critical issues in current state-of-the-art methods, including the Feature Ambiguity of projected 2D features in the ray to the 3D space, the Pose Ambiguity of the 3D convolution, and the Imbalance in the 3D convolution across different depth levels. We devise a novel Normalized Device Coordinates scene completion network (NDC-Scene) that directly extends the 2
  arXiv  Detail & Related papers  (2023-09-26T02:09:52Z)
• 3D-Aware Encoding for Style-based Neural Radiance Fields [50.118687869198716]
  We learn an inversion function to project an input image to the latent space of a NeRF generator and then synthesize novel views of the original image based on the latent code. Compared with GAN inversion for 2D generative models, NeRF inversion not only needs to 1) preserve the identity of the input image, but also 2) ensure 3D consistency in generated novel views. We propose a two-stage encoder for style-based NeRF inversion.
  arXiv  Detail & Related papers  (2022-11-12T06:14:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.