Bimanual Manipulation Within an 8 GB Budget: Zero-Copy Sensing and Quantized ACT on an Entry-Level Jetson

• URL: http://arxiv.org/abs/2608.03938v1
• Date: Tue, 04 Aug 2026 17:09:41 GMT
• Title: Bimanual Manipulation Within an 8 GB Budget: Zero-Copy Sensing and Quantized ACT on an Entry-Level Jetson
• Abstract summary: We present a bimanual SO-101 system running entirely on an NVIDIA Jetson Orin Nano Super (8 GB)<n>We build a GStreamer capture pipeline backed by NVMM buffers that removes redundant host-device copies from three-camera sensing.<n>We train ACT and Diffusion Policy on identical demonstrations, each at its own reference budget.
• Score: 35.18016233072556
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Bimanual manipulation policies trained with imitation learning are typically evaluated on workstation or datacenter-class GPUs, leaving the cost of deploying them on embedded hardware largely uncharacterized. We present a bimanual SO-101 system running entirely on an NVIDIA Jetson Orin Nano Super (8 GB), the entry-level tier of NVIDIA's embedded line, using a desktop GPU (RTX 3070) only for offline training, evaluated on pick-and-place of a deformable beanbag. First, we build a GStreamer capture pipeline backed by NVMM buffers that removes redundant host-device copies from three-camera sensing. Contrary to expectation, the conventional path fit the memory budget and dropped no frames; what zero-copy sensing recovers is CPU headroom (peak single-core utilization 98.0% to 77.0%) and worst-case latency (117.31 ms to 101.52 ms). Second, we train ACT and Diffusion Policy on identical demonstrations, each at its own reference budget (100k gradient steps for ACT, 200k for Diffusion Policy). ACT converges to a task-competent policy (19/20 trials) while Diffusion Policy does not converge to a usable one (0/10) even at twice the step count, which we attribute to differing convergence costs rather than an accuracy ceiling. Third, we convert ACT to TensorRT. FP16 reduces mean inference latency from 114.02 ms to 17.93 ms (6.4x) and INT8 to 12.65 ms (9.0x), with task success preserved at all three precisions (19/20, 18/20, 19/20). We report two findings not previously documented for ACT: TensorRT's general-purpose INT8 calibration quantizes the ResNet18 backbone but accepts zero of 145 transformer layers, explaining INT8's negligible size reduction over FP16 (0.9%) despite a further 28% latency gain; and the need for quantization is conditional on ACT's action-chunking configuration, feasible in full precision at n_action_steps = 100 but not at the per-step re-prediction temporal ensembling requires.

Related papers

• The Integer Alibi: Localizing Cross-Kernel Divergence in INT8-Quantized LLM Inference [0.10152838128195468]
  Two GPU kernels implementing the same scaled INT8 GEMM interface are usually treated as interchangeable.<n>We test that assumption: holding the checkpoint, prompts, hardware, inference engine, decoding, and quantization configuration fixed, we swap only the INT8 linear kernel inside vLLM.<n>At 1.7B each arm reproduces itself bit-for-bit across cold restarts, yet the arms agree on no sequence in any end-to-end comparison we ran (0/8, 0/16, and 0/64).
  arXiv  Detail & Related papers  (2026-08-13T20:34:36Z)
• Low Cost Two-Stage Fabric Defect Detection at the Edge [0.0]
  Fabric inspection in the garment industries remains largely manual,3.45 and commercial vision systems are priced beyond most small and medium mills.<n>We build such a cascade for four knitfabric defect classes and deploy it end-to-end on an NVIDIA Jetson Nano with FP16.<n>We report this as a caution for cascade speedups measured without controlling the data path, and position the system as AI-assisted triage rather than autonomous acceptance.
  arXiv  Detail & Related papers  (2026-08-12T19:49:00Z)
• Lightweight Transformer Models for On-Device Fault Detection: A Benchmark Study on Resource-Constrained Deployment [0.0]
  On-device fault detection enables real-time diagnostics without cloud dependency, but deploying machine learning models on resource-constrained hardware demands careful tradeoffs between accuracy, latency, and model size.<n>We present a benchmark comparing traditional ML methods against lightweight transformer architectures for binary fault detection.<n>Our results reveal that on well-separated sensor data, lightweight transformers match traditional ML at 87.8% F1 but at 100x the model size and 9000x the latency.
  arXiv  Detail & Related papers  (2026-06-23T05:54:33Z)
• Holding the FP8 Quality Ceiling at 8-Bit Weights and Activations: INT8 and GGUF Post-Training Quantization of Ideogram 4.0 for Consumer GPUs [0.08599681538174887]
  We study post-training quantization of Ideogram 4.0, a 9.3B flow-based diffusion transformer (DiT)<n>We evaluate every variant under schema-valid prompts produced by an expander built to Ideogram's published caption specification.<n>We release the INT8 W8A8 and GG Q4_K quantized weights on Hugging under a gated, non-commercial license.
  arXiv  Detail & Related papers  (2026-06-10T16:19:49Z)
• Visual Implicit Autoregressive Modeling [42.343328817507505]
  We introduce Visual Implicit Autoregressive Modeling (VIAR), a next-scale autoregressive generator that embeds an implicit equilibrium layer between shallow pre/post blocks.<n>VIAR attains FID 2.16, and sFID 8.07 on the ImageNet 256x256 benchmark.<n>By controlling the per-scale knob, VIAR can reduce peak memory from 19.24 GB to 8.53 GB and doubles throughput.
  arXiv  Detail & Related papers  (2026-05-02T03:23:17Z)
• ProbeLogits: Kernel-Level LLM Inference Primitives for AI-Native Operating Systems [0.0]
  An OS kernel that runs LLM inference internally can read logit distributions before any text is generated.<n>This paper presents ProbeLogits, a kernel-level operation that performs a single forward pass and reads specific token logits to classify agent actions as safe or dangerous.
  arXiv  Detail & Related papers  (2026-04-13T18:32:02Z)
• GPU-Accelerated Optimization of Transformer-Based Neural Networks for Real-Time Inference [0.0]
  This paper presents the design and evaluation of a GPU-accelerated inference pipeline for transformer models using NVIDIART with mixed-precision optimization.<n>The system achieves up to 64.4x speedup over CPU baselines, sub-10 ms latency for single-sample inference, and a 63 percent reduction in memory usage.
  arXiv  Detail & Related papers  (2026-03-30T17:27:33Z)
• Collapse or Preserve: Data-Dependent Temporal Aggregation for Spiking Neural Network Acceleration [1.1597621848542221]
  Spike sparsity is widely believed to enable efficient spiking neural network (SNN) inference on GPU hardware.<n>We demonstrate this is an illusion: SIMD architectures cannot exploit the fine-grained, unstructured sparsity of i.i.d. binary spikes.<n>We propose Temporal Aggregated Convolution (TAC), which exploits convolution linearity to pre-aggregate $K$ spike frames before a single convolution call.
  arXiv  Detail & Related papers  (2026-03-14T07:30:22Z)
• InfMem: Learning System-2 Memory Control for Long-Context Agent [43.413381779578266]
  InfMem is a control-centric agent that instantiates System-2-style control via a PreThink-Retrieve-Write protocol.<n>On ultra-long QA benchmarks from 32k to 1M tokens, InfMem consistently outperforms MemAgent across backbones.
  arXiv  Detail & Related papers  (2026-02-02T19:15:08Z)
• Speeding Up MACE: Low-Precision Tricks for Equivarient Force Fields [51.95157731126864]
  Machine-learning force fields can deliver accurate molecular dynamics (MD) at high computational cost.<n>This thesis aims to make MACE cheaper and faster by identifying computational bottlenecks and evaluating low-precision execution policies.
  arXiv  Detail & Related papers  (2025-10-23T14:02:34Z)
• Tracking Meets LoRA: Faster Training, Larger Model, Stronger Performance [87.19164603145056]
  We propose LoRAT, a method that unveils the power of large ViT model for tracking within laboratory-level resources. The essence of our work lies in adapting LoRA, a technique that fine-tunes a small subset of model parameters without adding inference latency. We design an anchor-free head solely based on to adapt PETR, enabling better performance with less computational overhead.
  arXiv  Detail & Related papers  (2024-03-08T11:41:48Z)
• Data-Free Dynamic Compression of CNNs for Tractable Efficiency [46.498278084317704]
  structured pruning approaches have shown promise in lowering floating-point operations without substantial drops in accuracy.<n>We propose HASTE (Hashing for Tractable Efficiency), a data-free, plug-and-play convolution module that instantly reduces a network's test-time inference cost without training or fine-tuning.<n>We demonstrate our approach on the popular vision benchmarks CIFAR-10 and ImageNet, where we achieve a 46.72% reduction in FLOPs with only a 1.25% loss in accuracy.
  arXiv  Detail & Related papers  (2023-09-29T13:09:40Z)
• I-BERT: Integer-only BERT Quantization [78.43819756382103]
  We propose I-BERT, a novel quantization scheme for Transformer based models. I-BERT performs an end-to-end integer-only BERT inference without any floating point calculation. We show that for both cases, I-BERT achieves similar (and slightly higher) accuracy as compared to the full-precision baseline.
  arXiv  Detail & Related papers  (2021-01-05T02:42:58Z)
• Efficient Integer-Arithmetic-Only Convolutional Neural Networks [87.01739569518513]
  We replace conventional ReLU with Bounded ReLU and find that the decline is due to activation quantization. Our integer networks achieve equivalent performance as the corresponding FPN networks, but have only 1/4 memory cost and run 2x faster on modern GPU.
  arXiv  Detail & Related papers  (2020-06-21T08:23:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.