Nanbeige4.2-3B on Apple Silicon: Fixing Deployment Bugs and Decreasing Looped Transformer Memory Overhead

• URL: http://arxiv.org/abs/2608.13987v1
• Date: Fri, 14 Aug 2026 06:05:26 GMT
• Title: Nanbeige4.2-3B on Apple Silicon: Fixing Deployment Bugs and Decreasing Looped Transformer Memory Overhead
• Abstract summary: Nanbeige4.2-3B is built around a Looped Transformer (LT) that reuses one stack layer for a second forward pass.<n>We show that fixing these bugs is still not sufficient for agentic tasks.<n>We introduce a chunked-prefill strategy which alleviates the incurred memory penalty.
• Score: 0.5076419064097732
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Nanbeige4.2-3B is a 3B-parameter agentic model built around a Looped Transformer (LT) that reuses one stack of layers for a second forward pass, adding effective depth without additional parameters. Evaluated on Apple Silicon (MPS), we identify five independent bugs which prevent the released checkpoint from running via Hugging Face transformers out of the box (including a silently-zeroed RoPE buffer and calls to removed transformers cache APIs). Furthermore, we show that fixing these bugs is still not sufficient for agentic tasks, due to the LT's layer-reuse strategy (which effectively doubles peak attention memory) used to achieve parameter efficiency. We thus introduce a chunked-prefill strategy which alleviates the incurred memory-capacity penalty, extending allowable context width by $2.7 \times$ on 32~GiB shared memory. However, even with the reduced memory overhead, we show that patches are required to render Nanbeige4.2-3B usable; resolving both system prompt and MPS-native memory bugs finally allows reliable evaluation on standard MCP and tool-calling benchmarks. On a subset of MCPMark, the debugged model completes up to 30\% of real agentic tasks (up from the original's 0\%), while, on BFCL, it is near-perfect at single tool calls (yet fails the majority of multi-tool tests). We release the patched checkpoint, system prompt optimizer, and evaluation harnesses at https://github.com/johnhalloran321/Nanbeige4.2-3B-mps-fix.

Related papers

• Dual-Grained Agent Memory and Shapley Context Attribution for Multimodal Agentic Learner [55.80248281761638]
  multimodal large language models (MLLMs) deliver impressive perception yet still falter on scientific and mathematical reasoning.<n>We propose DG-Mem, a dual-grained agentic memory framework that augments a frozen MLLM with a non- externally stored memory built once from training-time rollouts and consulted read-only at test time.
  arXiv  Detail & Related papers  (2026-08-24T13:56:01Z)
• MemGuard: Persisting Verifier Signals for LLM-Agent Memory Governance [21.021172368088504]
  LLM agents are moving from single-prompt use to long task streams in which reusable memory becomes a core capability.<n>MemGuard treats verifier output not as a one-shot filter, but as persistent lifecycle metadata.<n>MemGuard achieves the best success metric and lowest average steps in all 16 backbone-benchmark settings.
  arXiv  Detail & Related papers  (2026-08-22T09:25:23Z)
• The Recall Trap: A Recall-Maximizing Retriever Configuration Reduces Issue Resolution in Fixed-Budget Code Context [45.88028371034407]
  We report a controlled case study in code repair, not a new phenomenon.<n>We toggle one flag (one-chunk-perfile deduplication) on an otherwise identical stack.<n>We map where it holds: it reverses on a lexical BM25 (3.2pp, significant cross-paradigm interaction)<n>We show that at a tight fixed budget, do not hard-deduplicate by file, and A/B packing policies against the task, not the metric flag was tuned.
  arXiv  Detail & Related papers  (2026-08-14T19:22:50Z)
• Benchmarking LLM-Guided Control-Plane Policies for Backend Fault Isolation in HAProxy [0.0]
  We ask whether a Large Language Model can replace the static routing policy itself.<n>We find a capability threshold near 3B active parameters.<n>Above it, every model, regardless of architecture, saturates near an 88% reduction in client-perceived 5xx errors.
  arXiv  Detail & Related papers  (2026-08-11T06:15:20Z)
• Agent Memory Distillation: Empowering Small LLM Agents with Hierarchical Teacher Memory [49.6908961629532]
  Agent Memory Distillation (AMD) is a training-free framework that transfers structured knowledge from a large teacher agent to a small student agent through hierarchical memory.<n>AMD constructs three complementary memory types from successful teacher trajectories: memory encodes task-level strategies, Subtask memory provides concrete behavioral examples, and Function memory captures per-function calling conventions.<n>We evaluate AMD on three tool-use benchmarks using four student models (4B-8B parameters) with GPT-5-mini as the teacher, achieving average accuracy gains of 27.2%p, 11.2%p, and 3.4%p on AppWorld, BFCL V
  arXiv  Detail & Related papers  (2026-08-07T12:43:00Z)
• AgentDebugX: An Open-Source Toolkit for Failure Observability, Attribution, and Recovery in LLM Agents [83.37969790806069]
  LLM agent failures are difficult to debug because the step where an error surfaces is often not the one that caused it.<n>We present Agent DebuggerX, an open-source framework that organizes debug as a closed loop of Detect, Attribute, Recover, and Rerun.<n>At its core, Deep Debugger performs multi-turn root-cause diagnosis through global trajectory understanding.
  arXiv  Detail & Related papers  (2026-07-21T06:21:13Z)
• Asymmetric Virtual Memory Paging for Hybrid Mamba-Transformer Inference [0.0]
  Hybrid language models like Jamba mix attention layers with State Space Models (SSMs)<n>Unified pools pad SSM states to attention page sizes, wasting up to 7.3x capacity.<n>Asymmetric Virtual Memory Paging (AVMP) separates the two cache types into physically distinct pools behind a unified virtual address space.
  arXiv  Detail & Related papers  (2026-05-21T12:37:34Z)
• MemGym: a Long-Horizon Memory Environment for LLM Agents [69.79226770543049]
  We present MemGym, a benchmark for agentic memory that unifies existing agent gyms and in-house memory-grounded pipelines.<n>MemGym reports memory-isolated scores that decouple memory performance from reasoning, retrieval, and tool-use ability.<n>Our synthetic pipelines for MEMGYM-CODEQA and MEMGYM-DR are length-controllable, ablation-verified at every stage, and tightly aligned with downstream scenarios.
  arXiv  Detail & Related papers  (2026-05-20T07:25:33Z)
• When Simulation Lies: A Sim-to-Real Benchmark and Domain-Randomized RL Recipe for Tool-Use Agents [32.54888729689753]
  We study failures as a sim-to-real gap in the tool-use partially observable Markov decision process (POMDP)<n>We propose ToolRL-DR, a domain-randomization reinforcement learning (RL) recipe that trains a tool-use agent on perturbation-augmented trajectories.<n>On a 3B backbone, ToolRL-DR-Full retains roughly three-quarters of clean accuracy and reaches an aggregate accuracy comparable to open-source 14B function-calling baselines.
  arXiv  Detail & Related papers  (2026-05-12T10:40:28Z)
• NextMem: Towards Latent Factual Memory for LLM-based Agents [58.35585202907478]
  NextMem is a latent factual memory framework that utilizes an autoregressive autoencoder to efficiently construct latent memory.<n>Extensive experiments demonstrate that NextMem achieves superior performance.
  arXiv  Detail & Related papers  (2026-02-26T14:35:27Z)
• Memory-Efficient Structured Backpropagation for On-Device LLM Fine-Tuning [10.913120072779193]
  On-device fine-tuning enables privacy-preserving personalization of large language models.<n>Mobile devices impose severe memory constraints, typically 6--12GB shared across all workloads.<n>We propose Memory-efficient Structured Backpropagation (MeSP)<n>MeSP reduces peak memory from 361MB to 136MB for Qwen2.5-0.5B, enabling fine-tuning scenarios previously infeasible.
  arXiv  Detail & Related papers  (2026-02-13T16:24:33Z)
• InfMem: Learning System-2 Memory Control for Long-Context Agent [43.413381779578266]
  InfMem is a control-centric agent that instantiates System-2-style control via a PreThink-Retrieve-Write protocol.<n>On ultra-long QA benchmarks from 32k to 1M tokens, InfMem consistently outperforms MemAgent across backbones.
  arXiv  Detail & Related papers  (2026-02-02T19:15:08Z)
• A Simple Linear Patch Revives Layer-Pruned Large Language Models [58.056251480151104]
  Layer pruning has emerged as a widely used technique for compressing large language models (LLMs)<n>textscLinearPatch fuses two operations into one matrix multiply at the pruning interface.<n>The patch can be further refined with 5K unlabeled samples via memory-efficient offline distillation, pushing the retention to 95.16% within only 30 minutes on a single GPU.
  arXiv  Detail & Related papers  (2025-05-30T15:06:08Z)
• A Fast Post-Training Pruning Framework for Transformers [74.59556951906468]
  Pruning is an effective way to reduce the huge inference cost of large Transformer models. Prior work on model pruning requires retraining the model. We propose a fast post-training pruning framework for Transformers that does not require any retraining.
  arXiv  Detail & Related papers  (2022-03-29T07:41:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.