Lift, Associate, and Fuse: A Decision-Centric Framework for 2D-to-3D Foundation Model Transfer

• URL: http://arxiv.org/abs/2608.20659v1
• Date: Fri, 21 Aug 2026 01:33:23 GMT
• Title: Lift, Associate, and Fuse: A Decision-Centric Framework for 2D-to-3D Foundation Model Transfer
• Abstract summary: Methods that transfer predictions from two-dimensional foundation models into three-dimensional segmentation are commonly grouped by task or representation.<n>We introduce textbfLift, Associate, and Fuse (LAF), a decision-centric framework that represents a transfer system as five operators.<n>LAF defines an explicit contract for the persistent carrier and identifies the first stage at which discarded evidence becomes unrecoverable.
• Score: 18.616107275409764
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Methods that transfer predictions from two-dimensional foundation models into three-dimensional segmentation are commonly grouped by task or representation. Those groupings obscure the decisions that determine whether a system remains coherent across views: where image evidence is grounded, when observations become one identity, how semantic and granularity conflicts are handled, which information is fused, and what state survives for later queries. We introduce \textbf{Lift, Associate, and Fuse (LAF)}, a decision-centric framework that represents a transfer system as five operators: \textbf{Generate, Associate, Reconcile, Fuse, and Persist/Query}. LAF defines an explicit contract for the persistent carrier---its spatial support, semantic state, identity state, uncertainty, provenance, and supported operations---and identifies the first stage at which discarded evidence becomes unrecoverable. We operationalize the framework as a structured audit protocol and apply it to 161 systems available through 7 August 2026, spanning point-, field-, Gaussian-, object-, graph-, and memory-based carriers. Representation, temporal, relational, and feed-forward stress tests required no additional analytical stage after the final confirmation pass. The resulting decision traces expose four recurring properties: association does not establish identity; carrier design fixes both the query interface and correction boundary; rendered-view, native-3D, and proposal-level evaluations are not interchangeable; and qualifiers such as \emph{training-free}, \emph{real-time}, \emph{open-vocabulary}, and \emph{generalizable} are meaningful only when attached to a stage and a complete cost ledger. LAF therefore supplies a representation-neutral method for comparing existing systems, diagnosing irreversible failures, and specifying revisable 3D perception for future agents.

Related papers

• MNIST-PRO: MNIST is Back as a Partially Observable World for AI Agents [74.83535434786145]
  MNIST-PRO is a benchmark that isolates agentic perception by converting MNIST digit recognition into a sequential, glimpse-based search task with lookback constraints.<n>We evaluate ten multimodal models across four memory representations, including raw visual history, textual states, structured metric grid maps, and a consolidated visual canvas.
  arXiv  Detail & Related papers  (2026-08-31T16:05:39Z)
• Formal Verification of Agentic Systems over Operational Data [59.98246281888422]
  Agentic systems driven by large language models (LLMs) are increasingly deployed in real-world where they act on persistent operational data.<n>Existing approaches do not provide such system-level guarantees.<n>We study here the verification of agentic systems comprising a single LLM and a tool orchestration harness over relational operational data.
  arXiv  Detail & Related papers  (2026-08-04T13:01:30Z)
• CaIRec: Calibrated Modality Imputation for Incomplete Multimodal Recommendation [49.24608818070328]
  Real-world recommender systems often face incomplete modality observations, where items lack images, text, or other content features.<n>Existing modality imputation methods estimate missing representations from available item content, but two challenges remain.<n>First, they optimize the recovered representation itself without explicitly considering its relations with other modalities of the same item.<n>Second, even structurally coherent recovered information may remain ineffective for personalized ranking.<n>We propose Calibrated Imputation for Incomplete Multimodal Recommendation (CaIRec) to address these challenges.
  arXiv  Detail & Related papers  (2026-07-29T10:10:20Z)
• Reasoning without Gold Standards: A Proxy-Judge Theory of Autoformalization [25.540859911959885]
  Autoformalization (AF) asks a model to translate informal mathematical or logical reasoning into a formally checkable object.<n>We introduce a reference-free proxy-judge framework for AF that replaces gold-standard matching with a vector of per-axis property checks.<n>Across seven formalization backbones on miniF2F, ProofNet, e-SNLI, and ProntoQA, refinement consistently lifts Pass Rate over the single-shot ICL baseline.
  arXiv  Detail & Related papers  (2026-06-08T12:57:09Z)
• Beyond tokens: a unified framework for latent communication in LLM-based multi-agent systems [0.65268245109828]
  Multi-agent systems built on large language models (LLMs) have become a prevailing paradigm for tackling complex reasoning, planning, and tool-use tasks.<n>The dominant communication protocol in such systems is natural language: agents exchange messages token-by-token, verbalising their internal reasoning so that peers can read, verify, and respond.<n>A growing body of work therefore explores an alternative protocol -- latent communication, in which agents exchange continuous representations directly, bypassing the bottleneck of text generation.<n>This paper presents a unified framework for organising the rapidly expanding literature on latent communication.
  arXiv  Detail & Related papers  (2026-06-04T05:10:20Z)
• Interpretation, Learning, and Empathy as One Constraint: A Residual-Adequacy Architecture with Accountable Abstention [0.0]
  We develop a small cognitive architecture in which limits arise from a single quantity.<n>An Interpretation-Decision Unit (IDU) interprets a content vector through a family of regimes and decides which actions it licenses.<n>We prove the unit is total and deterministic: for any content and fixed configuration it halts in finitely many bounded-cost steps with a unique terminal witness.
  arXiv  Detail & Related papers  (2026-05-24T10:57:28Z)
• SAAG: Structured Agent Assessment and Grounding [13.62148061055744]
  We propose a cascaded diagnostic framework that decomposes agent-calling evaluation into three sequential stages.<n>We evaluate this framework on a benchmark derived from Glaive's function-calling dataset across registry sizes of 5, 10, and 15 agents.
  arXiv  Detail & Related papers  (2026-04-30T19:33:58Z)
• Semantic Entanglement in Vector-Based Retrieval: A Formal Framework and Context-Conditioned Disentanglement Pipeline for Agentic RAG Systems [0.0]
  We formalize semantic entanglement as a model-relative measure of cross-topic overlap in embedding space.<n>We introduce the Semantic Disentanglement Pipeline (SDP), a four-stage preprocessing framework that restructures documents prior to embedding.<n>We evaluate SDP on a real-world enterprise healthcare knowledge base comprising over 2,000 documents across approximately 25 sub-domains.
  arXiv  Detail & Related papers  (2026-04-20T00:24:34Z)
• Taming SAM3 in the Wild: A Concept Bank for Open-Vocabulary Segmentation [57.427604620940734]
  We present textscConceptBank, a framework to restore alignment between visual evidence and prompts on the fly.<n>Our approach anchors target-domain evidence via class-wise visual prototypes, (textitii) mines representative supports to suppress outliers under data drift, and (textitiii) fuses candidate concepts to rectify concept drift.
  arXiv  Detail & Related papers  (2026-02-06T02:59:11Z)
• OMG-Agent: Toward Robust Missing Modality Generation with Decoupled Coarse-to-Fine Agentic Workflows [9.617220633655716]
  We present textbfunderlineOmni-textbfunderlineModality textbfunderlineGeneration Agent (textbfOMG-Agent)
  arXiv  Detail & Related papers  (2026-02-04T02:25:40Z)
• Reasoning-Driven Amodal Completion: Collaborative Agents and Perceptual Evaluation [17.405818788700234]
  We present a Collaborative Multi-Agent Reasoning Framework that explicitly decouples Semantic Planning from Visual Synthesis.<n>Our method generates a structured, explicit plan before pixel generation, enabling visually and semantically coherent single-pass synthesis.<n>Addressing the limitations of traditional metrics in assessing inferred invisible content, we introduce the MAC-Score, a novel human-aligned evaluation metric.
  arXiv  Detail & Related papers  (2025-12-24T04:39:45Z)
• TransFIRA: Transfer Learning for Face Image Recognizability Assessment [73.61309363885552]
  TransFIRA is a lightweight and annotation-free framework that grounds recognizability directly in embedding space.<n>New extensions beyond faces include encoder-grounded explainability that reveals how degradations and subject-specific factors affect recognizability.<n> Experiments confirm state-of-the-art results on faces, strong robustness on body recognition, and under cross-dataset shifts.
  arXiv  Detail & Related papers  (2025-10-07T18:16:21Z)
• Hierarchical Context Alignment with Disentangled Geometric and Temporal Modeling for Semantic Occupancy Prediction [61.484280369655536]
  Camera-based 3D Semantic Occupancy Prediction (SOP) is crucial for understanding complex 3D scenes from limited 2D image observations.<n>Existing SOP methods typically aggregate contextual features to assist the occupancy representation learning.<n>We introduce a new Hierarchical context alignment paradigm for a more accurate SOP (Hi-SOP)
  arXiv  Detail & Related papers  (2024-12-11T09:53:10Z)
• Rethinking the Two-Stage Framework for Grounded Situation Recognition [61.93345308377144]
  Grounded Situation Recognition is an essential step towards "human-like" event understanding. Existing GSR methods resort to a two-stage framework: predicting the verb in the first stage and detecting the semantic roles in the second stage. We propose a novel SituFormer for GSR which consists of a Coarse-to-Fine Verb Model (CFVM) and a Transformer-based Noun Model (TNM)
  arXiv  Detail & Related papers  (2021-12-10T08:10:56Z)
• Inter-class Discrepancy Alignment for Face Recognition [55.578063356210144]
  We propose a unified framework calledInter-class DiscrepancyAlignment(IDA) IDA-DAO is used to align the similarity scores considering the discrepancy between the images and its neighbors. IDA-SSE can provide convincing inter-class neighbors by introducing virtual candidate images generated with GAN.
  arXiv  Detail & Related papers  (2021-03-02T08:20:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.