Daedalus-150M: A Convolution-Attention Hybrid Designed for CPU Inference

• URL: http://arxiv.org/abs/2608.20210v1
• Date: Thu, 20 Aug 2026 16:09:43 GMT
• Title: Daedalus-150M: A Convolution-Attention Hybrid Designed for CPU Inference
• Abstract summary: Small language models are usually built like large ones and then squeezed onto a CPU afterwards.<n>We did the opposite: we fixed the target first, one user, one token at a time, 4-bit weights, ordinary CPU, and chose the architecture to suit it.<n>The model scores 47.31 on a five-task benchmark against a bar of 42.20 that was fixed before training began.<n>It beats GPT-2 124M, Pythia-160M, OPT-125M and GPT-neo-125M, all trained on three to six times more data, and exceeds MobileLLM-125M'
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Small language models are usually built like large ones and then squeezed onto a CPU afterwards. We did the opposite: we fixed the target first, one user, one token at a time, 4-bit weights, ordinary CPU, and chose the architecture to suit it. The result keeps full attention in only 6 of its 18 blocks. The other 12 use short convolutions whose memory is two timesteps wide no matter how long the conversation gets, so two thirds of the network never re-reads a growing cache. Trained from scratch on 59.9B tokens, the model scores 47.31 on a five-task benchmark against a bar of 42.20 that was fixed before training began. It beats GPT-2 124M, Pythia-160M, OPT-125M and GPT-neo-125M, all trained on three to six times more data, and exceeds MobileLLM-125M's published score despite that model seeing a trillion tokens. Validation bits-per-byte is 0.8685. To check the architecture rather than the training recipe, we trained a conventional all-attention model of the same size on the same data, and wrote down the winning condition before scoring either. The hybrid won the chosen quality metric by 0.81%, matched it on downstream tasks, produced a 6.3% smaller 4-bit file, and decoded 1.76x faster at 2048 tokens of context, 2.08x against an external model of similar size. In every measurement the speed advantage is near zero at an empty context and grows with length, which is what the mechanism predicts and what a merely leaner model would not show. A simple bandwidth calculation predicts only 1.17x, so memory volume alone does not explain the gap. We also report what did not work: an unmitigated 4-bit quality cost, roughly half the convolution channels ending up inert and impossible to remove, and a vocabulary larger than this model size warrants.

Related papers

• Breaking the Token Ceiling: Distilling Smaller, Stronger Byte Models [67.72077328700088]
  We present the first large scale study of overtraining decoder-only dense transformer models varying two dimensions simultaneously.<n>We find that Token-1B models outperform byte models (End-Of-Token-1B and Bytes-1B) in the low-FLOP regime but eventually plateau.<n>By operating over a small vocabulary of 256 bytes instead of on the order of 100K tokens, they circumvent the need for top-kruncation during logit dumping.
  arXiv  Detail & Related papers  (2026-09-11T00:17:22Z)
• What actually runs: a measurement study of language model placement and decode speed on the Apple Neural Engine [0.0]
  We train matched models across size and precision, with quantized checkpoints byte-identical in structure to their fp16 counterparts.<n>We read the ANE's memory-controller byte counters during inference, establishing what actually ran rather than what the compiler intended.
  arXiv  Detail & Related papers  (2026-08-22T21:22:24Z)
• Opt.Gear Technical Report [3.573729952898132]
  OptGear is a foundation model designed for efficient on-device deployment, real-tim inference, and strong task capability.<n>It includes a dense model (1M, 270M, and 1B) with a context length of 64K.<n>OptGear-1M is the first generative language model to achieve 20 TPS with W4A32 quantization on the ARM Cortex-M7 CPU.
  arXiv  Detail & Related papers  (2026-08-02T06:43:25Z)
• A Modern Multimodal Assistant on a 6 GB 2011 GPU: Stage-Validated, All-GPU CUDA Inference for Fermi [0.0]
  This report keeps the hardware and asks what a model that fits can do.<n>We deploy Mini-V-4.6, a modern multimodal assistant pairing a SigLIP2 encoder and window-attention merger.<n>The system answers an image question end-to-end in 1.7s.
  arXiv  Detail & Related papers  (2026-07-16T04:48:44Z)
• Smarter and Cheaper at Once: Byte-Exact KV-Cache Grafting Turns a Frozen Small Model into a Verified-Knowledge Flywheel [0.0]
  We make a frozen small language model both more capable and dramatically cheaper at once, without changing any weights.<n>We show that own-position graft is the numerically exact operating point on a model with floating-point rotary encoding.<n>We describe the system at the behavior level; the engine is proprietary, and every reported number is backed by committed input and output so the scoring can be re-checked without it.
  arXiv  Detail & Related papers  (2026-07-15T23:55:48Z)
• Variable Bit-width Quantization: Learning Per-Group Precision for "Bigger-but-Smaller" Language Models [0.0]
  We introduce Variable Bit-width Quantization (VBQ)<n>Training-time method in which each contiguous group of 64 weights learns its own resolution from 1,2,4,8 bits.<n>As quality-per-byte, VBQ is 3.9-8.4x more efficient than FP16.
  arXiv  Detail & Related papers  (2026-07-03T02:42:04Z)
• Chronicals: A High-Performance Framework for LLM Fine-Tuning with 3.51x Speedup over Unsloth [0.0]
  We present Chronicals, an open-source training framework achieving 3.5x speedup over Unsloth.<n>We provide complete mathematical foundations: online softmax correctness, FlashAttention IO complexity O(N2 d2 M-1), LoRA+ learning rate gradient approximations.
  arXiv  Detail & Related papers  (2026-01-06T00:00:55Z)
• Test-Time Training Done Right [61.8429380523577]
  Test-Time Training (TTT) models context by adapting part of the model's weights (referred to as fast weights) during inference.<n>Existing TTT methods struggled to show effectiveness in handling long-context data.<n>We develop Large Chunk Test-Time Training (LaCT) which improves hardware utilization by orders of magnitude.
  arXiv  Detail & Related papers  (2025-05-29T17:50:34Z)
• QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Models [64.34635279436054]
  Mixture-of-Experts (MoE) architectures offer a general solution to the high inference costs of large language models (LLMs) via sparse routing. We present a solution to this memory problem, in form of a new compression and execution framework called QMoE.
  arXiv  Detail & Related papers  (2023-10-25T17:24:53Z)
• QLoRA: Efficient Finetuning of Quantized LLMs [66.58009990713134]
  We present QLoRA, an efficient finetuning approach that reduces memory usage enough to finetune a 65B parameter model on a single 48GB GPU. QLoRA backpropagates through a frozen, 4-bit quantized pretrained language model into Low Rank Adapters(LoRA) Our best model family, which we name Guanaco, outperforms all previous openly released models on the Vicuna benchmark.
  arXiv  Detail & Related papers  (2023-05-23T17:50:33Z)
• The case for 4-bit precision: k-bit Inference Scaling Laws [75.4335600212427]
  Quantization methods reduce the number of bits required to represent each parameter in a model. The final model size depends on both the number of parameters of the original model and the rate of compression. We run more than 35,000 zero-shot experiments with 16-bit inputs and k-bit parameters to examine which quantization methods improve scaling for 3 to 8-bit precision.
  arXiv  Detail & Related papers  (2022-12-19T18:48:33Z)
• Combined Scaling for Zero-shot Transfer Learning [146.0851484769142]
  We present a combined scaling method - named BASIC - that achieves 85.7% top-1 accuracy on the ImageNet ILSVRC-2012 validation set. This accuracy surpasses best published similar models - CLIP and ALIGN - by 9.3%. Our model also shows significant improvements in robustness benchmarks.
  arXiv  Detail & Related papers  (2021-11-19T05:25:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.