PoGO: A Scalable Proof of Useful Work via Quantized Gradient Descent and Merkle Proofs

• URL: http://arxiv.org/abs/2504.07540v2
• Date: Wed, 23 Apr 2025 12:59:42 GMT
• Title: PoGO: A Scalable Proof of Useful Work via Quantized Gradient Descent and Merkle Proofs
• Authors: José I. Orlicki,
• Abstract summary: We present a design called Proof of Gradient Optimization (PoGO) for blockchain consensus.<n>PoGO miners produce verifiable evidence of training large-scale machine-learning models.<n>We provide an empirical cost analysis showing that verification is significantly cheaper than training.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present a design called Proof of Gradient Optimization (PoGO) for blockchain consensus, where miners produce verifiable evidence of training large-scale machine-learning models. Building on previous work, we incorporate quantized gradients (4-bit precision) to reduce storage and computation requirements, while still preserving the ability of verifiers to check that real progress has been made on lowering the model's loss. Additionally, we employ Merkle proofs over the full 32-bit model to handle large parameter sets and to enable random leaf checks with minimal on-chain data. We illustrate these ideas using GPT-3 (175B parameters) as a reference example and also refer to smaller but high-performance models (e.g., Gemma~3 with 27B parameters). We provide an empirical cost analysis showing that verification is significantly cheaper than training, thanks in part to quantization and sampling. We also discuss the necessity of longer block times (potentially hours) when incorporating meaningful training steps, the trade-offs when using specialized GPU hardware, and how binary diffs may incrementally optimize updates. Finally, we note that fine-tuning can be handled in a similar manner, merely changing the dataset and the manner of sampling but preserving the overall verification flow. Our protocol allows verifiers to issue either positive or negative attestations; these are aggregated at finalization to either confirm the update or slash the miner.

Related papers

• Benchmarking Few-shot Transferability of Pre-trained Models with Improved Evaluation Protocols [123.73663884421272]
  Few-shot transfer has been revolutionized by stronger pre-trained models and improved adaptation algorithms.<n>We establish FEWTRANS, a comprehensive benchmark containing 10 diverse datasets.<n>By releasing FEWTRANS, we aim to provide a rigorous "ruler" to streamline reproducible advances in few-shot transfer learning research.
  arXiv  Detail & Related papers  (2026-02-28T05:41:57Z)
• Decomposing Reasoning Efficiency in Large Language Models [2.4149105714758545]
  We decompose token efficiency into interpretable factors: completion under a fixed token budget, conditional correctness given completion, and verbosity.<n>When reasoning traces are available, we add deterministic trace-quality measures to separate looping from verbose-but-engaged reasoning.<n>Our decomposition reveals distinct bottleneck profiles that suggest different efficiency interventions.
  arXiv  Detail & Related papers  (2026-02-10T14:09:18Z)
• Trust but Verify! A Survey on Verification Design for Test-time Scaling [8.428618801719198]
  Test-time scaling (TTS) has emerged as a new frontier for scaling the performance of Large Language Models.<n>Verifiers serve as reward models that help score the candidate outputs from the decoding process.<n>Verifiers could be prompt-based, fine-tuned as a discriminative or generative model.
  arXiv  Detail & Related papers  (2025-08-20T22:27:21Z)
• Unlearning at Scale: Implementing the Right to be Forgotten in Large Language Models [0.0]
  Our approach treats as a minimal program and logs permicrobatch record.<n>Under pinned stack and deterministic kernels, replaying the training tail yields the same parameters as training retain set.
  arXiv  Detail & Related papers  (2025-08-17T03:29:22Z)
• InfiJanice: Joint Analysis and In-situ Correction Engine for Quantization-Induced Math Degradation in Large Language Models [39.257022875813284]
  Large Language Models (LLMs) have demonstrated impressive performance on complex reasoning benchmarks such as GSM8K, MATH, and AIME.<n>Model quantization has emerged as a promising approach to reduce memory footprint and inference latency.<n>We show that quantization can degrade mathematical reasoning accuracy by up to 69.81%.
  arXiv  Detail & Related papers  (2025-05-16T12:11:40Z)
• The Geometry of Self-Verification in a Task-Specific Reasoning Model [45.669264589017665]
  We train a model using DeepSeek R1's recipe on the CountDown task. We do a top-down and bottom-up analysis to reverse-engineer how the model verifies its outputs.
  arXiv  Detail & Related papers  (2025-04-19T18:40:51Z)
• TOPLOC: A Locality Sensitive Hashing Scheme for Trustless Verifiable Inference [0.0]
  Large language models (LLMs) have proven to be very capable, but access to the best models currently rely on inference providers which introduces trust challenges. We propose TOPLOC, a novel method for verifiable inference that addresses this problem.
  arXiv  Detail & Related papers  (2025-01-27T12:46:45Z)
• TCMM: Token Constraint and Multi-Scale Memory Bank of Contrastive Learning for Unsupervised Person Re-identification [2.3183978396999967]
  This paper introduces the ViT Token Constraint to mitigate the damage caused by patch noises to the ViT architecture.<n>The proposed Multi-scale Memory enhances the exploration of outlier samples and maintains feature consistency.
  arXiv  Detail & Related papers  (2025-01-15T07:14:02Z)
• AutoPSV: Automated Process-Supervised Verifier [10.283965168399158]
  textbfAutomated textbfProcess-textbfSupervised textbfVerifier (textbftextscAutoPSV) textscAutoPSV begins by training a verification model on the correctness of final answers. We experimentally validate that the step-level confidence changes learned by the verification model trained on the final answer correctness can effectively identify errors in the reasoning steps.
  arXiv  Detail & Related papers  (2024-05-27T03:44:24Z)
• QCore: Data-Efficient, On-Device Continual Calibration for Quantized Models -- Extended Version [34.280197473547226]
  Machine learning models can be deployed on edge devices with limited storage and computational capabilities. We propose QCore to enable continual calibration on the edge.
  arXiv  Detail & Related papers  (2024-04-22T08:57:46Z)
• Training Chain-of-Thought via Latent-Variable Inference [30.21067593018967]
  Large language models (LLMs) solve problems more accurately and interpretably when instructed to work out the answer step by step using a chain-of-thought'' prompt. Naively combining CoT with supervised tuning requires supervision not just of the correct answers, but also of detailed rationales that lead to those answers. We propose a fine-tuning strategy that tries to maximize the emphmarginal log-likelihood of generating a correct answer using CoT prompting.
  arXiv  Detail & Related papers  (2023-11-28T17:47:32Z)
• One-bit Supervision for Image Classification: Problem, Solution, and Beyond [114.95815360508395]
  This paper presents one-bit supervision, a novel setting of learning with fewer labels, for image classification. We propose a multi-stage training paradigm and incorporate negative label suppression into an off-the-shelf semi-supervised learning algorithm. In multiple benchmarks, the learning efficiency of the proposed approach surpasses that using full-bit, semi-supervised supervision.
  arXiv  Detail & Related papers  (2023-11-26T07:39:00Z)
• Gaussian Process Probes (GPP) for Uncertainty-Aware Probing [61.91898698128994]
  We introduce a unified and simple framework for probing and measuring uncertainty about concepts represented by models. Our experiments show it can (1) probe a model's representations of concepts even with a very small number of examples, (2) accurately measure both epistemic uncertainty (how confident the probe is) and aleatory uncertainty (how fuzzy the concepts are to the model), and (3) detect out of distribution data using those uncertainty measures as well as classic methods do.
  arXiv  Detail & Related papers  (2023-05-29T17:00:16Z)
• Winner-Take-All Column Row Sampling for Memory Efficient Adaptation of Language Model [89.8764435351222]
  We propose a new family of unbiased estimators called WTA-CRS, for matrix production with reduced variance. Our work provides both theoretical and experimental evidence that, in the context of tuning transformers, our proposed estimators exhibit lower variance compared to existing ones.
  arXiv  Detail & Related papers  (2023-05-24T15:52:08Z)
• From Spectral Graph Convolutions to Large Scale Graph Convolutional Networks [0.0]
  Graph Convolutional Networks (GCNs) have been shown to be a powerful concept that has been successfully applied to a large variety of tasks. We study the theory that paved the way to the definition of GCN, including related parts of classical graph theory.
  arXiv  Detail & Related papers  (2022-07-12T16:57:08Z)
• Intrinsic Certified Robustness of Bagging against Data Poisoning Attacks [75.46678178805382]
  In a emphdata poisoning attack, an attacker modifies, deletes, and/or inserts some training examples to corrupt the learnt machine learning model. We prove the intrinsic certified robustness of bagging against data poisoning attacks. Our method achieves a certified accuracy of $91.1%$ on MNIST when arbitrarily modifying, deleting, and/or inserting 100 training examples.
  arXiv  Detail & Related papers  (2020-08-11T03:12:42Z)
• AQD: Towards Accurate Fully-Quantized Object Detection [94.06347866374927]
  We propose an Accurate Quantized object Detection solution, termed AQD, to get rid of floating-point computation. Our AQD achieves comparable or even better performance compared with the full-precision counterpart under extremely low-bit schemes.
  arXiv  Detail & Related papers  (2020-07-14T09:07:29Z)
• Evaluating Prediction-Time Batch Normalization for Robustness under Covariate Shift [81.74795324629712]
  We call prediction-time batch normalization, which significantly improves model accuracy and calibration under covariate shift. We show that prediction-time batch normalization provides complementary benefits to existing state-of-the-art approaches for improving robustness. The method has mixed results when used alongside pre-training, and does not seem to perform as well under more natural types of dataset shift.
  arXiv  Detail & Related papers  (2020-06-19T05:08:43Z)
• The Right Tool for the Job: Matching Model and Instance Complexities [62.95183777679024]
  As NLP models become larger, executing a trained model requires significant computational resources incurring monetary and environmental costs. We propose a modification to contextual representation fine-tuning which, during inference, allows for an early (and fast) "exit" We test our proposed modification on five different datasets in two tasks: three text classification datasets and two natural language inference benchmarks.
  arXiv  Detail & Related papers  (2020-04-16T04:28:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.