React-ing to Grace Hopper 200: Five Open-Weights Coding Models, One React Native App, One GH200, One Weekend

• URL: http://arxiv.org/abs/2604.17187v1
• Date: Sun, 19 Apr 2026 01:21:02 GMT
• Title: React-ing to Grace Hopper 200: Five Open-Weights Coding Models, One React Native App, One GH200, One Weekend
• Abstract summary: We evaluate five state-of-the-art open-weights coding language models -- Kimi-K2.5 (at Q3 and Q4 quantizations), GLM-5.1, Qwen3-Coder-480B, and DeepSeek-V3.2 -- on a single React Native application generation task on NVIDIA GH200 576 GB hardware.<n>We find that SWE-Bench rankings do not predict task performance.
• Score: 0.40611352512781873
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We evaluate five state-of-the-art open-weights coding language models -- Kimi-K2.5 (at Q3 and Q4 quantizations), GLM-5.1, Qwen3-Coder-480B, and DeepSeek-V3.2 -- on a single multi-file React Native application generation task on NVIDIA GH200 576 GB hardware. The task specifies authentication, per-user per-day counting, and web compatibility, and is evaluated on whether the generated project runs out-of-the-box and on feature-level correctness. We find that SWE-Bench rankings do not predict task performance: Kimi-K2.5 at aggressive 3-bit quantization (UD-Q3_K_XL, 480 GB) produces the most complete and specification-compliant output, outranking models with substantially higher SWE-Bench Pro scores. We document three novel deployment findings: (1) default temperature=0 in coding tools causes sampling hangs with reasoning-model architectures, (2) reasoning-model thinking traces can leak through integration tools' file-path parsers, and (3) web-platform adaptation of native-mobile APIs is a universal training-data gap across every model tested. We also map the hardware-tier structure of April 2026 open-weights coding models, identifying two architectural schools and showing that the efficiency school (10-15 B active parameters) delivers equivalent SWE-Bench results at roughly 1/7th the hardware cost of the scale school (32-40 B active parameters).

Related papers

• StepX-Edge: An On-Device UI Vision-Language Model via Architecture-Training-Deployment Co-Design [8.502778898017556]
  We present StepX-Edge, a on-device UI vision-language model that resolves tension through three co-grained co-grained architecture, training, and deployment.<n>For training, the five-stage StepX-Curriculum framework is designed around our observation of mutual-promotion effects among UI subtasks.<n>For deployment, a module-wise differentiated two-stage PTQ-to-QAT quantization scheme keeps the post-quantization accuracy loss within 1%.
  arXiv  Detail & Related papers  (2026-07-20T14:14:44Z)
• HyMobileAgent: Data-Environment Co-Scaling for Efficient GUI Agents [57.53434680014667]
  HyMobileAgent is a vision-native foundation model built on Hy3.0-VL-A3B.<n>We develop a joint data and environment centric scaling framework to address the key bottlenecks of mobile interaction.
  arXiv  Detail & Related papers  (2026-07-16T04:12:42Z)
• RadLite: Multi-Task LoRA Fine-Tuning of Small Language Models for CPU-Deployable Radiology AI [1.5161700234179216]
  Large language models (LLMs) show promise in radiology but their deployment is limited by computational requirements that preclude use in resource-constrained clinical environments.<n>We investigate whether small language models (SLMs) of 3-4 billion parameters can achieve strong multi-task radiology performance through LoRA fine-tuning, enabling deployment on consumer-grade CPUs.<n>Our work demonstrates that small, efficiently fine-tuned models can serve as practical multi-task radiology AI assistants deployable entirely on consumer hardware without GPU requirements.
  arXiv  Detail & Related papers  (2026-05-01T05:39:08Z)
• MinerU2.5-Pro: Pushing the Limits of Data-Centric Document Parsing at Scale [92.09717763663873]
  We present MinerU2.5-Pro, which advances the state of the art purely through data engineering and training strategy design.<n>At its core is a Data Engine co-designed around coverage, informativeness, and annotation accuracy.<n>We show that MinerU2.5-Pro achieves 95.69 on OmniDocBench v1.6, improving over the same-architecture baseline by 2.71 points and surpassing all existing methods.
  arXiv  Detail & Related papers  (2026-04-06T15:44:18Z)
• Hydra: Unifying Document Retrieval and Generation in a Single Vision-Language Model [0.0]
  Hydra is a dual-head approach that provides both ColBERT-style late-interaction retrieval and autoregressive generation from a single vision-language model (VLM)<n>A single LoRA adapter, trained only for retrieval, is toggled at inference; disabling it recovers the base model's generation quality.<n>On ViDoRe V1, Hydra (4B) is within 1 percentage point of a controlled single-head baseline in a single training run.
  arXiv  Detail & Related papers  (2026-03-30T15:17:41Z)
• David vs. Goliath: A comparative study of different-sized LLMs for code generation in the domain of automotive scenario generation [1.6752458252726459]
  NL-to-Scenic generation with large language models (LLMs) suffers from scarce data, limited metrics.<n>We introduce NL2Scenic, an open dataset and framework with 146 NL/Scenic pairs, a difficulty-stratified 30-case test split, and an Example Retriever.<n>We evaluate 13 models: four proprietary (GPT-4o, GPT-5, Claude-Sonnet-4, Gemini-2.5-pro) and nine open-source code models (Qwen2.5Coder 0.5B-32B; CodeLlama 7B/13B/34B)
  arXiv  Detail & Related papers  (2025-10-15T21:37:02Z)
• Skywork-SWE: Unveiling Data Scaling Laws for Software Engineering in LLMs [19.766885088032932]
  Software engineering (SWE) has emerged as a crucial testbed for next-generation LLM agents.<n>Most existing datasets are limited to only a few thousand GitHub-sourced instances.<n>We propose an incremental, automated data-curation pipeline that systematically scales both the volume and diversity of SWE datasets.
  arXiv  Detail & Related papers  (2025-06-24T03:53:36Z)
• MonkeyOCR: Document Parsing with a Structure-Recognition-Relation Triplet Paradigm [60.14048367611333]
  MonkeyOCR is a vision-language model for document parsing.<n>It advances the state of the art by leveraging a Structure-Recognition-Relation (SRR) triplet paradigm.
  arXiv  Detail & Related papers  (2025-06-05T16:34:57Z)
• Qwen2.5 Technical Report [122.13958993185952]
  We introduce Qwen2.5, a comprehensive series of large language models (LLMs) designed to meet diverse needs.<n>Compared to previous iterations, Qwen 2.5 has been significantly improved during both the pre-training and post-training stages.<n>Open-weight offerings include base and instruction-tuned models, with quantized versions available.<n>For hosted solutions, the proprietary models currently include two mixture-of-experts (MoE) variants: Qwen2.5-Turbo and Qwen2.5-Plus.
  arXiv  Detail & Related papers  (2024-12-19T17:56:09Z)
• Qwen2.5-Coder Technical Report [105.131580912726]
  We introduce the Qwen2.5-Coder series, a significant upgrade from its predecessor, CodeQwen1.5. As a code-specific model, Qwen2.5-Coder is built upon the Qwen2.5 architecture and continues pretrained on a vast corpus of over 5.5 trillion tokens.
  arXiv  Detail & Related papers  (2024-09-18T17:57:57Z)
• Disentangle Your Dense Object Detector [82.22771433419727]
  Deep learning-based dense object detectors have achieved great success in the past few years and have been applied to numerous multimedia applications such as video understanding. However, the current training pipeline for dense detectors is compromised to lots of conjunctions that may not hold. We propose Disentangled Dense Object Detector (DDOD), in which simple and effective disentanglement mechanisms are designed and integrated into the current state-of-the-art detectors.
  arXiv  Detail & Related papers  (2021-07-07T00:52:16Z)
• When Liebig's Barrel Meets Facial Landmark Detection: A Practical Model [87.25037167380522]
  We propose a model that is accurate, robust, efficient, generalizable, and end-to-end trainable. In order to achieve a better accuracy, we propose two lightweight modules. DQInit dynamically initializes the queries of decoder from the inputs, enabling the model to achieve as good accuracy as the ones with multiple decoder layers. QAMem is designed to enhance the discriminative ability of queries on low-resolution feature maps by assigning separate memory values to each query rather than a shared one.
  arXiv  Detail & Related papers  (2021-05-27T13:51:42Z)
• GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding [46.74457030177477]
  We show how to scale up multilingual neural machine translation Transformer model with Sparsely-Gated Mixture-of-Experts beyond 600 billion parameters using automatic sharding. We demonstrate such a giant model can efficiently be trained on 2048 TPU v3 accelerators in 4 days to achieve far superior quality for translation from 100 languages to English.
  arXiv  Detail & Related papers  (2020-06-30T10:42:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.