COREY: Entropy-Guided Runtime Chunk Scheduling for Selective Scan Kernels

• URL: http://arxiv.org/abs/2604.10597v2
• Date: Sat, 18 Apr 2026 23:16:35 GMT
• Title: COREY: Entropy-Guided Runtime Chunk Scheduling for Selective Scan Kernels
• Abstract summary: Prototype scheduler uses activation entropy estimated via fixed-width histograms as a runtime signal for chunk-size selection.<n>COREY is positioned as a Concept and Feasibility contribution.<n>This work does not yet include a fully integrated end-to-end run connecting Tier 2a and Tier 2b.
• Score: 11.316541559874864
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Mamba selective state space models (SSMs) provide linear-time sequence modeling but are often limited by memory bandwidth in practice, where selective state updates are executed as fragmented kernels with repeated intermediate tensor materialization. We present COREY, a prototype scheduler that uses activation entropy estimated via fixed-width histograms as a runtime signal for chunk-size selection at the kernel-invocation level. COREY is positioned as a Concept and Feasibility contribution: a single-parameter runtime auto-tuner built on an existing Triton selective-scan kernel rather than a new fused implementation. Evidence is organized in three tiers. Tier 1 (Python cost model) shows that entropy-guided grouping reduces surrogate latency and DRAM traffic. Tier 2a (real-checkpoint inline hook) demonstrates that entropy computation and chunk selection can run on the critical path of model.generate(); on Mamba-370M (RTX 3070, n=5), measured overhead is 8.3 percent with full instrumentation and estimated about 2 percent with sparse sampling. Tier 2b (kernel-level scan benchmark) shows that, under a principled calibration where H_ref equals log(K), COREY selects the same chunk as a one-time-profile oracle without offline sweeps and achieves up to 4.41x speedup over static chunk-64. This work does not yet include a fully integrated end-to-end run connecting Tier 2a and Tier 2b, which remains key future work. Across 80 LongBench prompts, entropy distributions are stable, supporting COREY as a practical runtime auto-tuner within a single regime. Code and data: https://github.com/mabo1215/COREY_Transformer/.

Related papers

• BoltNet: An Ultra-Lightweight Convolutional Network for On-Device Plant Species Identification [3.860597465886636]
  BoltNet is an ultra-lightweight fully convolutional architecture combining a Spatial Redistribution Bottleneck and Logit PreSampling.<n>On Pl@ntNet300K, BoltNet reaches 0.682 F1-score with 341K parameters (1.37 MB), the highest F1-score among evaluated models below 2 MB.<n>BoltNet is the most consistently efficient model, with the best FPS/W on the GPU and NPU and second-best on the CPU.
  arXiv  Detail & Related papers  (2026-08-12T09:34:52Z)
• Chimera: Designing and Chinchilla-Scaling Hybrid Visual Diffusion Transformers [54.01042826416009]
  Visual generation requires high-resolution images, long videos, and multimodal context.<n>We introduce Chimera, a hybrid visual diffusion backbone with a principled scaling recipe.<n>We show that the dense backbone is 1.7x as compute-efficient as a matched full-attention Wan2.1 2B baseline.
  arXiv  Detail & Related papers  (2026-07-30T17:58:02Z)
• Decomposing Runtime, Kernel, and Quantization Speedups via a Matched FP16 Intermediate: A Hardware-Conditioned Case Study on Four NVIDIA RTX A5000 GPUs [0.614481021961242]
  Quantization extends concurrent users roughly four times past a reproducible half precision memory cliff.<n>A matched intermediate stack keeps the faster runtime without the quantized kernel splits the full speedup into a runtime part and a kernel and quantization part.<n>Differences in sampling mode and prompt pool between the two stacks are documented as threats to validity.
  arXiv  Detail & Related papers  (2026-07-13T10:31:54Z)
• PersistentKV: Page-Aware Decode Scheduling for Long-Context LLM Serving on Commodity GPUs [0.8460698440162889]
  We present PersistentKV, a block-table decode attention engine and page-aware scheduling study.<n>Per PersistentKV maps work by KV-head group, executes directly over native page tables, and adds a compact workqueue schedule.<n>We show that five held-out seeds improve mean wall decode-token throughput by 1.04x to 1.08x on bimodal, uniform, and Zipf-like workloads.
  arXiv  Detail & Related papers  (2026-06-25T06:56:43Z)
• StereoFactory: A Unified Merging Framework for Robust Stereo Matching [61.973843344605655]
  Stereo matching has advanced through foundation models trained on large-scale datasets, yet this paradigm suffers from a scalability bottleneck.<n>Model merging offers a scalable post-hoc alternative by integrating knowledge from specialized models after source checkpoints are available.<n>We propose StereoFactory, a coarse-to-fine evolutionary framework for adaptive model merging.
  arXiv  Detail & Related papers  (2026-06-16T03:36:59Z)
• KForge: LLM-Driven Cross-Platform Kernel Generation for AI Accelerators [5.703659894138959]
  Production inference increasingly targets a heterogeneous mix of accelerators.<n>This creates a systems challenge: each pipeline now requires high-performance kernels across a growing set of hardware backends and programming models.<n>We present KForge, a cross-platform framework built around an iterative refinement loop driven by two collaborating LLM-based agents.
  arXiv  Detail & Related papers  (2026-06-01T23:48:55Z)
• Spatial Autoregressive Modeling of DINOv3 Embeddings for Unsupervised Anomaly Detection [15.896078006029475]
  DINO models provide rich patch-level representations that have recently enabled strong performance in unsupervised anomaly detection (UAD)<n>Most existing methods extract patch embeddings from normal'' images and model them independently, ignoring spatial and neighborhood relationships between patches.<n>We propose a framework that explicitly models spatial and contextual dependencies between patch embeddings using a 2D autoregressive (AR) model.
  arXiv  Detail & Related papers  (2026-03-03T13:30:33Z)
• Robot-DIFT: Distilling Diffusion Features for Geometrically Consistent Visuomotor Control [31.523644271297616]
  A key bottleneck in generalizable robot manipulation is a structural mismatch between current visual backbones and the physical requirements of closed-loop control.<n>We propose Robot-DIFT, a framework that decouples the source of geometric information from the process of inference via Manifold Distillation.<n>By distilling a frozen diffusion teacher into a deterministic Spatial-Semantic Feature Pyramid Network (S2-FPN), we retain the rich geometric priors of the generative model while ensuring temporal stability, real-time execution, and robustness against drift.
  arXiv  Detail & Related papers  (2026-02-12T13:30:24Z)
• LINA: Linear Autoregressive Image Generative Models with Continuous Tokens [56.80443965097921]
  Autoregressive models with continuous tokens form a promising paradigm for visual generation, especially for text-to-image (T2I) synthesis.<n>We study how to design compute-efficient linear attention within this framework.<n>We present LINA, a simple and compute-efficient T2I model built entirely on linear attention, capable of generating high-fidelity 1024x1024 images from user instructions.
  arXiv  Detail & Related papers  (2026-01-30T06:44:33Z)
• S$^3$-Attention:Attention-Aligned Endogenous Retrieval for Memory-Bounded Long-Context Inference [11.779449360037518]
  We present S3-Attention, a memory-first inference-time framework that treats long-context processing as attention-aligned endogenous retrieval.<n>S3-Attention decodes transient key and query projections into top-k sparse feature identifiers using lightweight sparse autoencoders.<n>It constructs a CPU-based inverted index mapping features to token positions or spans during a single streaming scan.
  arXiv  Detail & Related papers  (2026-01-25T05:25:22Z)
• Disentangling History and Propagation Dependencies in Cross-Subject Knee Contact Stress Prediction Using a Shared MeshGraphNet Backbone [0.8283940114367679]
  It remains unclear whether the dominant source of prediction uncertainty arises from temporal history dependence or spatial propagation dependence.<n>A dataset of running trials from nine subjects was constructed using an OpenSim-FEBio workflow.<n>Models incorporating history encoding significantly outperformed the baseline MGN and ModMGN in global accuracy and spatial consistency.
  arXiv  Detail & Related papers  (2026-01-13T08:15:57Z)
• AutoSAGE: Input-Aware CUDA Scheduling for Sparse GNN Aggregation (SpMM/SDDMM) and CSR Attention [52.20940151628735]
  AutoSAGE is an input-aware scheduler that chooses tiling and mapping per input.<n>On Reddit OGBN-Products it achieves up to 4.7x kernel-level speedups.
  arXiv  Detail & Related papers  (2025-11-17T18:25:51Z)
• The Curious Case of In-Training Compression of State Space Models [49.819321766705514]
  State Space Models (SSMs) tackle long sequence modeling tasks efficiently, offer both parallelizable training and fast inference.<n>Key design challenge is striking the right balance between maximizing expressivity and limiting this computational burden.<n>Our approach, textscCompreSSM, applies to Linear Time-Invariant SSMs such as Linear Recurrent Units, but is also extendable to selective models.
  arXiv  Detail & Related papers  (2025-10-03T09:02:33Z)
• Network-Optimised Spiking Neural Network for Event-Driven Networking [2.5941336499463383]
  Spiking neural networks offer event-driven computation suited to time-critical networking tasks such as anomaly detection, local routing control, and congestion management at the edge.<n>We introduce Network-Optimised Spiking (NOS), a compact two-variable unit whose state encodes normalised queue occupancy and a recovery resource.<n>We provide guidance for data-driven initialisation, surrogate-gradient training with a homotopy on reset sharpness, and explicit stability checks with topology of bounds for resource constrained deployments.
  arXiv  Detail & Related papers  (2025-09-27T22:31:24Z)
• ATTS: Asynchronous Test-Time Scaling via Conformal Prediction [112.54016379556073]
  Large language models (LLMs) benefit from test-time scaling but are often hampered by high inference latency.<n>We introduce ATTS (Asynchronous Test-Time Scaling), a statistically guaranteed adaptive scaling framework.<n>We show that ATTS delivers up to 56.7x speedup in test-time scaling and a 4.14x throughput improvement.
  arXiv  Detail & Related papers  (2025-09-18T16:55:09Z)
• Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free [81.65559031466452]
  We conduct experiments to investigate gating-augmented softmax attention variants.<n>We find that a simple modification-applying a head-specific sigmoid gate after the Scaled Dot-Product Attention (SDPA)-consistently improves performance.
  arXiv  Detail & Related papers  (2025-05-10T17:15:49Z)
• Online Dense Point Tracking with Streaming Memory [54.22820729477756]
  Dense point tracking is a challenging task requiring the continuous tracking of every point in the initial frame throughout a substantial portion of a video.<n>Recent point tracking algorithms usually depend on sliding windows for indirect information propagation from the first frame to the current one.<n>We present a lightweight and fast model with textbfStreaming memory for dense textbfPOint textbfTracking and online video processing.
  arXiv  Detail & Related papers  (2025-03-09T06:16:49Z)
• Two-stream Beats One-stream: Asymmetric Siamese Network for Efficient Visual Tracking [54.124445709376154]
  We propose a novel asymmetric Siamese tracker named textbfAsymTrack for efficient tracking.<n>Building on this architecture, we devise an efficient template modulation mechanism to inject crucial cues into the search features.<n>Experiments demonstrate that AsymTrack offers superior speed-precision trade-offs across different platforms.
  arXiv  Detail & Related papers  (2025-03-01T14:44:54Z)
• SHERL: Synthesizing High Accuracy and Efficient Memory for Resource-Limited Transfer Learning [63.93193829913252]
  We propose an innovative METL strategy called SHERL for resource-limited scenarios. In the early route, intermediate outputs are consolidated via an anti-redundancy operation. In the late route, utilizing minimal late pre-trained layers could alleviate the peak demand on memory overhead.
  arXiv  Detail & Related papers  (2024-07-10T10:22:35Z)
• DA-Flow: Dual Attention Normalizing Flow for Skeleton-based Video Anomaly Detection [52.74152717667157]
  We propose a lightweight module called Dual Attention Module (DAM) for capturing cross-dimension interaction relationships in-temporal skeletal data. It employs the frame attention mechanism to identify the most significant frames and the skeleton attention mechanism to capture broader relationships across fixed partitions with minimal parameters and flops.
  arXiv  Detail & Related papers  (2024-06-05T06:18:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.