Daily Paper Briefing

Abstract Overview

This paper presents NoPA, an online 3D scene graph generation method that replaces single-Gaussian object representations with fixed-size non-parametric particle sets derived from RGB-D observations. The method uses a two-stage association procedure: a Hellinger-distance pre-filter for clear cases and a Maximum Mean Discrepancy (MMD) test for ambiguous merges, followed by KDE-based resampling to keep memory constant per object. It also propagates relationships within high-affinity clusters to recover missing edges caused by noisy single-view predictions. The experiments target real-time online 3D scene graph generation on 3DSSG and ReplicaSSG, comparing accuracy, latency, and VRAM usage against prior online methods.

Novelty

The main novelty is the use of a fixed-size non-parametric particle representation for each object in online 3D scene graph generation, instead of the single-Gaussian approximation used by prior real-time methods. The paper also introduces an MMD-based distribution-level merging criterion and an affinity-cluster-based relation propagation mechanism tailored to this representation.

Results

On 3DSSG, NoPA with 256 particles reports 53.2 relationship recall, 69.0 object recall, and 61.4 predicate recall, exceeding the reproduced FROSS baseline (25.7, 60.6, 30.7) while maintaining similar real-time latency (27 ms vs. 22 ms) and nearly identical VRAM usage (1206 MB vs. 1204 MB). On ReplicaSSG, NoPA with 256 particles achieves 36.9 relationship recall, 28.6 object recall, and 39.5 predicate recall, improving over reproduced FROSS results at comparable latency and memory overhead. Ablations indicate that the largest gains come from combining the non-parametric representation with the tailored MMD-based merging and relation propagation components.

Key Points

1. NoPA models each object as a fixed-size 3D particle set with a kernel density interpretation, preserving more geometric support than a single Gaussian while keeping bounded memory.
2. The online fusion step uses a cheap Hellinger pre-filter and applies MMD only to ambiguous pairs, aiming to improve merge robustness without giving up real-time operation.
3. Quantitative and ablation results on 3DSSG and ReplicaSSG show that the full system improves object, predicate, and relationship recall over prior online baselines, with modest latency and memory overhead.

References

• arXiv: https://arxiv.org/abs/2607.00529v1
• Fugu-MT: https://fugumt.com/fugumt/paper_check/2607.00529v1
• Hugging Face Papers: https://huggingface.co/papers/2607.00529