Hardware-Software Co-Design for Float16 On-Device Training on RISC-V Single-Core

• URL: http://arxiv.org/abs/2607.21130v1
• Date: Thu, 23 Jul 2026 10:06:11 GMT
• Title: Hardware-Software Co-Design for Float16 On-Device Training on RISC-V Single-Core
• Abstract summary: This work proposes an open-source framework to enable complete on-device training on resource-constrained RISC-V single-core.<n>Our approach allows memory footprint reduction by about 50% as compared to using float32 and with minimal model performance degradation.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: By leveraging standard RISC-V extensions, namely Zfh (scalar float16) and Zvfh (vector float16), this work proposes an open-source framework to enable complete on-device training on resource-constrained RISC-V single-core. Our approach allows memory footprint reduction by about 50% as compared to using float32 and with minimal model performance degradation. We also facilitate transfer learning and fine-tuning scenarios by incorporating layer-freezing capabilities. Our work builds onto AIfES, an open-source, modular and generic DNN training and inference framework for embedded systems that can be extended with custom hardware-specific functions. The benefits of float16 is further emphasized by outlining the low area overhead of Zfh on a RV64GC super-scalar out-of-order FPGA softcore (+1.15% LUT6 and +0.05% FF at 175MHz). Finally, we discuss the architecture of a Zvfh implementation within the same RISC-V core.

Related papers

• SLAI T-Rex: Full-Parameter Post-training of the DeepSeek-V4 Family on Ascend SuperPOD [89.88607039727783]
  This report presents an end-to-end optimization practice on the Ascend NPU SuperPOD.<n>We develop a hierarchical optimization framework spanning model-level parallelism, computation-communication orchestration, and low-level kernel execution.<n>The resulting system achieves 34.22% Model FLOPs Utilization (MFU) with a 2.93x improvement over the open-source baseline recipe.
  arXiv  Detail & Related papers  (2026-07-22T13:49:17Z)
• HORCRUX: A Complete PQC RISC-V eXtension Architecture [37.36871652478481]
  This work presents a compact RISC-V extension for Post-Quantum Cryptography (PQC) called HORCRUX.<n>HorCRUX addresses the difficult trade-off between crypto-agility, high performance, and low resource consumption in constrained environments.<n>The extension's modular structure maintains backward compatibility with standard RISC-V cores, offering a scalable solution for deploying PQC on constrained embedded systems.
  arXiv  Detail & Related papers  (2026-07-15T15:21:26Z)
• VFA: Relieving Vector Operations in Flash Attention with Global Maximum Pre-computation [5.279829639786756]
  FlashAttention-style online softmax enables exact attention computation with linear memory.<n>Non-matmul components of online softmax can become vector or SIMD limited and dominate latency.<n>This paper proposes Vector Relieved Flash Attention (VFA), a hardware-friendly method that reduces rowmax-driven updates of the running maximum.
  arXiv  Detail & Related papers  (2026-04-14T14:28:50Z)
• BWTA: Accurate and Efficient Binarized Transformer by Algorithm-Hardware Co-design [71.97035034203275]
  We analyze zero-point distortion in binarization and propose a Binary Weights & Ternary Activations (BWTA) quantization scheme.<n>For training, we propose Smooth Multi-Stage Quantization, combining a Levelwise Degradation Strategy and a MagnitudeAlignment Projection Factor.<n>Experiments show that BWTA approaches full-precision performance for Transformer-based models, with an average 3.5% drop on GLUE and less than 2% drop on five tasks.
  arXiv  Detail & Related papers  (2026-04-05T04:25:07Z)
• Scaling DoRA: High-Rank Adaptation via Factored Norms and Fused Kernels [83.99688944263843]
  Weight-De Low-Rank Adaptation (DoRA) extends LoRA by decoupling weight magnitude from direction.<n>At d_in = 8192 and rank r = 384, a single module's norm requires about 512 MB of transient working memory in bf16.<n>A factored norm decomposes the squared norm into base, cross, and Gram terms computable through O(d_out r + r2) intermediates, eliminating the dense product.
  arXiv  Detail & Related papers  (2026-03-23T17:57:24Z)
• Speeding Up MACE: Low-Precision Tricks for Equivarient Force Fields [51.95157731126864]
  Machine-learning force fields can deliver accurate molecular dynamics (MD) at high computational cost.<n>This thesis aims to make MACE cheaper and faster by identifying computational bottlenecks and evaluating low-precision execution policies.
  arXiv  Detail & Related papers  (2025-10-23T14:02:34Z)
• Decentor-V: Lightweight ML Training on Low-Power RISC-V Edge Devices [2.2332974094443174]
  L-SGD, a lightweight variant of gradient descent, has enabled neural network training on Arm Cortex-M Microcontroller Units (MCUs)<n>This work extends L-SGD to RISC-V-based MCUs, an open architecture that still lacks robust support for on-device training.<n>We introduce an 8-bit quantized version of L-SGD for RISC-V, which achieves nearly 4x reduction in memory usage and a 2.2x speedup in training time, with negligible accuracy degradation.
  arXiv  Detail & Related papers  (2025-09-10T09:33:54Z)
• PRISM: Distributed Inference for Foundation Models at Edge [73.54372283220444]
  PRISM is a communication-efficient and compute-aware strategy for distributed Transformer inference on edge devices.<n>We evaluate PRISM on ViT, BERT, and GPT-2 across diverse datasets.
  arXiv  Detail & Related papers  (2025-07-16T11:25:03Z)
• Tensor Program Optimization for the RISC-V Vector Extension Using Probabilistic Programs [0.6242215470795112]
  We present a workflow based on the TVM compiler to efficiently map AI workloads onto RISC-V vector units.<n>Our proposal shows a mean improvement of 46% in execution latency when compared against the autovectorization feature of GCC.<n>We open-sourced our proposal for the community to expand it to target other RISC-V extensions.
  arXiv  Detail & Related papers  (2025-07-02T08:15:33Z)
• V-Seek: Accelerating LLM Reasoning on Open-hardware Server-class RISC-V Platforms [13.87105456715319]
  This paper focuses on optimizing Large Language Models (LLMs) inference on the Sophon SG2042, the first commercially available many-core RISC-V CPU with vector processing capabilities.<n>On two recent state-of-the-art LLMs optimized for reasoning, DeepSeek R1 Distill Llama 8B and DeepSeek R1 Distill QWEN 14B, we achieve 4.32/2.29 token/s for token generation and 6.54/3.68 token/s for prompt processing, with a speed up of up 2.9x/3.0x compared to our baseline.
  arXiv  Detail & Related papers  (2025-03-21T09:00:19Z)
• Reconfigurable Distributed FPGA Cluster Design for Deep Learning Accelerators [59.11160990637615]
  We propose a distributed system based on lowpower embedded FPGAs designed for edge computing applications. The proposed system can simultaneously execute diverse Neural Network (NN) models, arrange the graph in a pipeline structure, and manually allocate greater resources to the most computationally intensive layers of the NN graph.
  arXiv  Detail & Related papers  (2023-05-24T16:08:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.