Coupled Continuous-Discrete Generation for Scene Text Image Super-Resolution

• URL: http://arxiv.org/abs/2608.04525v1
• Date: Wed, 05 Aug 2026 06:57:30 GMT
• Title: Coupled Continuous-Discrete Generation for Scene Text Image Super-Resolution
• Abstract summary: Scene text image super-resolution (STISR) aims to recover visually plausible appearance while preserving character semantics from degraded inputs.<n>We present DualTSR, a unified framework that formulates STISR as coupled continuous-discrete generation.<n> Conditional flow matching restores continuous image latents, while absorbing-state discrete diffusion reconstructs text tokens.
• Score: 51.31474187379532
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Scene text image super-resolution (STISR) aims to recover visually plausible appearance while preserving character semantics from degraded inputs. Existing STISR systems often rely on externally generated priors or separate image and text models, resulting in error propagation and costly multi-stage inference. We present DualTSR, a unified framework that formulates STISR as coupled continuous-discrete generation. Conditional flow matching restores continuous image latents, while absorbing-state discrete diffusion reconstructs text tokens. Both processes share a multimodal transformer backbone, allowing the evolving image and text states to interact throughout generation without an external OCR prior at inference. On CTR-TSR, DualTSR achieves the best FID, LPIPS, ACC, and NED among the compared methods at both X2 and X4. On an aligned RealCE subset, it obtains the best FID, ACC, and NED with competitive LPIPS. Compared with DiffTSR at X4, DualTSR improves ACC by 12.78 percentage points while reducing the parameter count from 1.23B to 203M and end-to-end latency from 13.3s to 132ms. These results establish DualTSR as an accurate and efficient method for STISR.

Related papers

• FLAT: Resampling Image and Text into 1D Flexible-Length Aligned Transmodal Tokens for Retrieval and Generation [29.59094476146176]
  Flat is a representation pre-training framework that jointly optimize a shared multimodal encoder alongside downstream text-to-image (T2I) and image-to-text (I2T) decoders.<n>A single pre-training stage allows FLAT to perform cross-modal retrieval and generation across variable prefix K, achieving a T2I GenEval score of 71.1.
  arXiv  Detail & Related papers  (2026-09-15T03:35:59Z)
• Rarity-Aware Discrete Diffusion with Spatially Consistent Decoding for Photo-Realistic Image Super-Resolution [80.28905986756685]
  DiMOO-SR is a rarity-aware multimodal discrete diffusion framework for photo-realistic image Super-Resolution.<n>Experiments on widely used real-world SR benchmarks demonstrate that DiMOO-SR achieves competitive perceptual quality with only a few parallel decoding steps.
  arXiv  Detail & Related papers  (2026-07-20T07:01:01Z)
• LBTCap: A Lightweight Bilateral Transformer for Real-Time Remote Sensing Image Change Captioning [44.00535468968033]
  Remote sensing image change captioning (RSICC) generates descriptions of semantic changes between paired remote sensing images (RSIs)<n>LBTCap is a lightweight RSICC framework built on a bilateral Transformer that jointly models pre- and post-change features for efficient processing of paired RSIs.<n> experiments show that LBTCap matches or closely approaches the accuracy of state-of-the-art methods while using far fewer parameters and running at markedly higher inference speed.
  arXiv  Detail & Related papers  (2026-07-03T13:38:38Z)
• VOSR: A Vision-Only Generative Model for Image Super-Resolution [28.65862989487937]
  We propose a Vision-Only generative framework for generative image super-resolution (SR)<n>We first extract semantically rich and spatially grounded features from the LR input using a pretrained vision encoder as visual semantic guidance.<n>Our results, for the first time, show that high-quality generative SR can be achieved without multimodal pretraining.
  arXiv  Detail & Related papers  (2026-04-03T17:50:29Z)
• DualTSR: Unified Dual-Diffusion Transformer for Scene Text Image Super-Resolution [52.962204748003394]
  Scene Text Image Super-Resolution aims to restore high-resolution details in low-resolution text images.<n>Existing methods, however, often depend on external Optical Character Recognition (OCR) models for textual priors.<n>We introduce DualTSR, a unified end-to-end framework that addresses both issues.
  arXiv  Detail & Related papers  (2026-03-15T03:50:47Z)
• Few-Step Diffusion via Score identity Distillation [67.07985339442703]
  Diffusion distillation has emerged as a promising strategy for accelerating text-to-image (T2I) diffusion models.<n>Existing methods rely on real or teacher-synthesized images to perform well when distilling high-resolution T2I diffusion models.<n>We propose two new guidance strategies: Zero-CFG, which disables CFG in the teacher and removes text conditioning in the fake score network, and Anti-CFG, which applies negative CFG in the fake score network.
  arXiv  Detail & Related papers  (2025-05-19T03:45:16Z)
• Improving Consistency in Diffusion Models for Image Super-Resolution [28.945663118445037]
  We observe two kinds of inconsistencies in diffusion-based methods.<n>We introduce ConsisSR to handle both semantic and training-inference consistencies.<n>Our method demonstrates state-of-the-art performance among existing diffusion models.
  arXiv  Detail & Related papers  (2024-10-17T17:41:52Z)
• Noisy-Correspondence Learning for Text-to-Image Person Re-identification [50.07634676709067]
  We propose a novel Robust Dual Embedding method (RDE) to learn robust visual-semantic associations even with noisy correspondences. Our method achieves state-of-the-art results both with and without synthetic noisy correspondences on three datasets.
  arXiv  Detail & Related papers  (2023-08-19T05:34:13Z)
• Audio-visual Multi-channel Recognition of Overlapped Speech [79.21950701506732]
  This paper presents an audio-visual multi-channel overlapped speech recognition system featuring tightly integrated separation front-end and recognition back-end. Experiments suggest that the proposed multi-channel AVSR system outperforms the baseline audio-only ASR system by up to 6.81% (26.83% relative) and 22.22% (56.87% relative) absolute word error rate (WER) reduction on overlapped speech constructed using either simulation or replaying of the lipreading sentence 2 dataset respectively.
  arXiv  Detail & Related papers  (2020-05-18T10:31:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.