GPT-OSS-20B: A Comprehensive Deployment-Centric Analysis of OpenAI's Open-Weight Mixture of Experts Model

• URL: http://arxiv.org/abs/2508.16700v2
• Date: Sun, 31 Aug 2025 03:40:19 GMT
• Title: GPT-OSS-20B: A Comprehensive Deployment-Centric Analysis of OpenAI's Open-Weight Mixture of Experts Model
• Abstract summary: GPT-OSS-20B (Mixture-of-Experts; 20.9B total, approx. 3.61B active) is evaluated against dense baselines Qwen3-32B and Yi-34B.<n>With only 17.3% of parameters active (3.61B of 20.9B), GPT-OSS-20B provides about 31.8% higher decode throughput and 25.8% lower energy per 1000 generated tokens.
• Score: 7.2566101637741545
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present a single-GPU (H100, bf16) evaluation of GPT-OSS-20B (Mixture-of-Experts; 20.9B total, approx. 3.61B active) against dense baselines Qwen3-32B and Yi-34B across multiple dimensions. We measure true time-to-first-token (TTFT), full-decode throughput (TPOT), end-to-end latency percentiles, peak VRAM with past key values (PKV) held, and energy via a consistent nvidia-smi-based sampler. At a 2048-token context with 64-token decode, GPT-OSS-20B delivers higher decode throughput and tokens per Joule than dense baselines Qwen3-32B and Yi-34B, while substantially reducing peak VRAM and energy per 1000 generated tokens; its TTFT is higher due to MoE routing overhead. With only 17.3% of parameters active (3.61B of 20.9B), GPT-OSS-20B provides about 31.8% higher decode throughput and 25.8% lower energy per 1000 generated tokens than Qwen3-32B at 2048/64, while using 31.7% less peak VRAM. Normalized by active parameters, GPT-OSS-20B shows markedly stronger per-active-parameter efficiency (APE), underscoring MoE's deployment advantages. We do not evaluate accuracy; this is a deployment-focused study. We release code and consolidated results to enable replication and extension.

Related papers

• SymbolicLight V2: Hybrid Neuromorphic Architecture and Sparse Execution for Low-Energy Language Inference [8.419155861590548]
  SymbolicLight V2 combines sparse event computation with continuous-state processing.<n>We implement SymbolicLight V2 on an Alveo U50C FPGA using digital fixed-point arithmetic.<n>On four Cortex-A76 cores of a ROCK 5T, complete requests reach 65.4 tokens/s at 9.80 W and 0.151 J per generated token at the adapter's AC input.
  arXiv  Detail & Related papers  (2026-09-09T06:18:49Z)
• GreenBench: Benchmarking Energy Efficiency and Carbon Footprint of Open-Source LLM Inference on Apple Silicon [0.0]
  This paper presents GreenBench, a benchmarking framework that evaluates the energy efficiency, throughput, and carbon footprint of five open-source Large Language Models (LLMs)<n>We find that the Apple M4 Pro draws only 0.47 W of CPU+GPU package power during sustained inference, with total system power of 8-12 W.<n>Smaller models (3-3.8B) deliver 2.6-4.2x higher throughput and up to 62% less energy per token than larger models (7-9B)<n>We provide per-token energy at package and system levels with CO2 estimates for India and US grids.
  arXiv  Detail & Related papers  (2026-08-24T04:46:26Z)
• RoPE-Aware Bit Allocation for KV-Cache Quantization [52.099459337231345]
  Block-GTQ is a bit allocator for key-cache quantization built on TurboQuant-MSE.<n>It better preserves RoPE query-key logits on a ten-model diagnostic panel.<n>It runs 1.34x faster than fp16 FlashAttention2 at 128K context.
  arXiv  Detail & Related papers  (2026-06-23T00:17:48Z)
• From Compression to Deployment: Real-Time and Energy-Efficient FastGRNN on Ultra-Constrained Microcontrollers [0.0]
  We present an end-to-end open-source reproduction of FastGRNN, a compact gated cell, deployed on two bare-metal targets.<n>The model occupies 566 bytes of weights and achieves macro F1 = 0.918 (seed 0; five-seed Q15 mean 0.853+-0.107) on the HAPT test set.<n>Both platforms sustain real-time 50 Hz streaming inference (9.21 ms per sample on Arduino; 13 ms on MSP430), where a 256-entry sigmoid/tanh look-up table delivers a 30.5x speedup.
  arXiv  Detail & Related papers  (2026-06-15T19:49:38Z)
• BWTA: Accurate and Efficient Binarized Transformer by Algorithm-Hardware Co-design [71.97035034203275]
  We analyze zero-point distortion in binarization and propose a Binary Weights & Ternary Activations (BWTA) quantization scheme.<n>For training, we propose Smooth Multi-Stage Quantization, combining a Levelwise Degradation Strategy and a MagnitudeAlignment Projection Factor.<n>Experiments show that BWTA approaches full-precision performance for Transformer-based models, with an average 3.5% drop on GLUE and less than 2% drop on five tasks.
  arXiv  Detail & Related papers  (2026-04-05T04:25:07Z)
• Once-for-All Channel Mixers (HYPERTINYPW): Generative Compression for TinyML [0.0]
  We present HYPER-TINYPW, a compression-as-generation approach that replaces most stored PW weights with generated weights.<n>A shared micro-MLP synthesizes PW kernels once at load time from tiny per-layer codes, caches them, and executes them with standard integer operators.<n>It preserves commodity MCU runtimes and adds only a one-off cost.
  arXiv  Detail & Related papers  (2026-03-26T01:08:52Z)
• Step 3.5 Flash: Open Frontier-Level Intelligence with 11B Active Parameters [169.7981969517903]
  Step 3.5 Flash bridges frontier-level agentic intelligence and computational efficiency.<n>We focus on what matters most when building agents: sharp reasoning and fast, reliable execution.
  arXiv  Detail & Related papers  (2026-02-11T07:53:51Z)
• STEP3-VL-10B Technical Report [115.89015065130127]
  STEP3-VL-10B is a lightweight foundation model designed to redefine the trade-off between compact efficiency and frontier-level multimodal intelligence.<n>We implement Parallel Coordinated Reasoning (PaCoRe) to scale test-time compute, allocating resources to scalable perceptual reasoning.<n>It records 92.2% on MMBench and 80.11% on MMMU, while excelling in complex reasoning with 94.43% on AIME2025 and 75.95% on MathVision.
  arXiv  Detail & Related papers  (2026-01-14T17:58:24Z)
• Energy-Efficient Deep Learning for Traffic Classification on Microcontrollers [1.3124513975412255]
  We present a practical deep learning (DL) approach for energy-efficient traffic classification on resource-limited microcontrollers.<n>We develop a lightweight 1D-CNN, optimized via hardware-aware neural architecture search (HW-NAS), which achieves 96.59% accuracy on the ISCX VPN-Non-VPN dataset.<n>We evaluate real-world inference performance on two microcontrollers.
  arXiv  Detail & Related papers  (2025-06-12T16:10:22Z)
• Energy-Efficient NTT Sampler for Kyber Benchmarked on FPGA [0.8049701904919515]
  Kyber is a lattice-based key encapsulation mechanism selected for standardization by the NIST Post-Quantum Cryptography (PQC) project.<n>A critical component of Kyber's key generation process is the sampling of matrix elements from a uniform distribution over the ring Rq.<n>Existing approaches to this sampling rely on rejection sampling.<n>We propose Modified SampleNTT, which substantially educes the average number of bits required from SHAKE-128 to generate elements in Rq.
  arXiv  Detail & Related papers  (2025-05-03T10:54:01Z)
• TeLLMe: An Energy-Efficient Ternary LLM Accelerator for Prefilling and Decoding on Edge FPGAs [5.889337608109388]
  TeLLMe is the first ternary LLM accelerator for low-power FPGAs.<n>It supports both prefill and autoregressive decoding using 1.58-bit weights and 8-bit activations.<n>Under a 7W power budget, TeLLMe delivers up to 9 tokens/s throughput over 1,024-token contexts.
  arXiv  Detail & Related papers  (2025-04-22T21:00:58Z)
• Fast Inference with Kronecker-Sparse Matrices [4.387337528923525]
  Existing GPU kernels for KS matrix multiplication suffer from high data movement costs.<n>We propose a fused, output-stationary GPU kernel that eliminates these overheads.<n>We demonstrate in FP32 end-to-end latency reductions of up to 22% in ViT-S/16 and 16% in GPT-2 medium.
  arXiv  Detail & Related papers  (2024-05-23T19:36:10Z)
• DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model [118.06260386652778]
  We present DeepSeek-V2, a strong Mixture-of-Experts (MoE) language model characterized by economical training and efficient inference. DeepSeek-V2 adopts innovative architectures including Multi-head Latent Attention (MLA) and DeepSeekMoE. Compared with DeepSeek 67B, DeepSeek-V2 achieves significantly stronger performance, and meanwhile saves 42.5% of training costs.
  arXiv  Detail & Related papers  (2024-05-07T15:56:43Z)
• Shallow Cross-Encoders for Low-Latency Retrieval [69.06104373460597]
  Cross-Encoders based on large transformer models (such as BERT or T5) are computationally expensive and allow for scoring only a small number of documents within a reasonably small latency window. We show that weaker shallow transformer models (i.e., transformers with a limited number of layers) actually perform better than full-scale models when constrained to these practical low-latency settings.
  arXiv  Detail & Related papers  (2024-03-29T15:07:21Z)
• From PEFT to DEFT: Parameter Efficient Finetuning for Reducing Activation Density in Transformers [52.199303258423306]
  We propose a novel density loss that encourages higher activation sparsity in pre-trained models. Our proposed method, textbfDEFT, can consistently reduce activation density by up to textbf44.94% on RoBERTa$_mathrmLarge$ and by textbf53.19% (encoder density) and textbf90.60% (decoder density) on Flan-T5$_mathrmXXL$.
  arXiv  Detail & Related papers  (2024-02-02T21:25:46Z)
• EfficientBERT: Progressively Searching Multilayer Perceptron via Warm-up Knowledge Distillation [82.3956677850676]
  Pre-trained language models have shown remarkable results on various NLP tasks. Due to their bulky size and slow inference speed, it is hard to deploy them on edge devices. In this paper, we have a critical insight that improving the feed-forward network (FFN) in BERT has a higher gain than improving the multi-head attention (MHA)
  arXiv  Detail & Related papers  (2021-09-15T11:25:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.