RelateAnything: Real-Time Open-Vocabulary Relation Prediction From Any Inputs

• URL: http://arxiv.org/abs/2609.12552v1
• Date: Fri, 11 Sep 2026 07:57:41 GMT
• Title: RelateAnything: Real-Time Open-Vocabulary Relation Prediction From Any Inputs
• Abstract summary: RelateAnything takes an image and regions from any source and returning scored relations over a predicate vocabulary supplied at inference as strings.<n>Training over 19,103 predicates requires positive-unlabeled supervision and a text encoder that separates antonyms.<n>On three cross-dataset benchmarks and a fourth zero-shot, RelateAnything has 2.3-3.5x the mean recall of the strongest open-vocabulary method of comparable scale.
• Score: 0.40611352512781873
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Open-vocabulary detection accepts any class list at inference, and promptable segmentation returns regions without class names: the taxonomy has left the model and become an input. Relation prediction has not. Scene-graph models are still trained and evaluated on the 50 or 56 predicates of one annotation style, their relation head conditioned on object labels and so tied to one detector. Three obstacles explain this, none primarily modelling: no relation corpus is both free-text and verified, a label-conditioned architecture cannot accept a vocabulary it was not trained on, and the standard metric rewards agreement with the training corpus, so a larger vocabulary scores as a regression. We present RelateAnything, a 53M-parameter model taking an image and regions from any source and returning scored relations over a predicate vocabulary supplied at inference as strings. Object labels are never an input, so the region source can change without retraining, and the vocabulary is a bank of text embeddings, not a learned classifier. It runs at 20 ms/frame. Training over 19,103 predicates requires positive-unlabeled supervision and a text encoder that separates antonyms, which contrastive encoders embed at cosine 0.95. To supply the supervision we build RA-4M, 474k images and 4.3M relations over 10,102 free-text predicates, generated against numbered box markers and geometrically verified. To measure it we build OV-SGG-Bench, six axes scored across datasets that the priors standard recall rewards cannot satisfy. On three cross-dataset benchmarks and a fourth zero-shot, RelateAnything has 2.3-3.5x the mean recall of the strongest open-vocabulary method of comparable scale, margins that survive a real detector, and leads a 3B-VLM scene-graph model on both metrics at under 2% of its parameters. In-domain measurement overstates transfer gains ~5x. Model, corpus and benchmark are public.

Related papers

• Reification as a Transferable Vocabulary: Zero-Shot Link Prediction with Vanilla GNNs [0.033842793760651545]
  Knowledge graph foundation models such as ULTRA achieve zero-shot link prediction on unseen graphs through dedicated architectures that hard-code a transfer mechanism.<n>In this work we move that mechanism out of the architecture and into the representation, by emphreifying the input graph.<n>Five textbook GNNs each trained on a single knowledge graph of 4,245 triples for 30 minutes on one NVIDIA A100, transfer zero-shot to 40 inductive link-prediction benchmarks.
  arXiv  Detail & Related papers  (2026-09-10T10:26:09Z)
• Kathleen Writes: Autoregressive Generation and Data Scaling Without Attention [0.0]
  We show that a byte-level, attention-free architecture can match strong baselines on classification at 450-700K parameters, without pretraining.<n>We introduce FORM DISTANCE, a non-parametric, gaming-resistant instrument for "reads like text"<n>We also report four architectural additions that did not help, and a computed lexicon reaching 94% of a learned table's top-1 accuracy at one fifth of the parameters.
  arXiv  Detail & Related papers  (2026-08-05T10:44:26Z)
• Benchmarking Speech-to-Speech Translation Models [55.00303727199927]
  Speech-to-speech translation (S2ST) has advanced rapidly, but offline evaluation lacks a unified protocol.<n>We introduce a benchmarking framework integrating 46 metrics across eight dimensions.<n>We deploy it on 1,248 model-language configurations from FLEURS and CVSS.
  arXiv  Detail & Related papers  (2026-06-02T07:01:33Z)
• RelWitness: Open-Vocabulary 3D Scene Graph Generation with Visual-Geometric Relation Witnesses [1.3468350096927395]
  We propose a framework for open-vocabulary 3D scene graph generation from posed RGB-D sequences under incomplete relation supervision.<n>The key concept is a relation witness: a concrete visual-geometric cue that makes a relation observable in the captured scene.<n> RelWitness constructs relation witness records from RGB views, depth maps, reconstructed 3D geometry, role-sensitive text, object-prior null views, and multi-view consistency.
  arXiv  Detail & Related papers  (2026-05-20T07:18:56Z)
• DataDignity: Training Data Attribution for Large Language Models [8.195274857647782]
  We introduce FakeWiki, a benchmark of 3,537 fabricated Wikipedia-style articles.<n>FakeWiki includes QA probes, source-preserving paraphrases, retro-generated variants, hard anti- documents that remain topically similar while removing answer-critical facts.<n>We evaluate seven retrieval baselines, a training-free activation-steering retrieval-fusion method, SteerFuse, and a supervised contrastive provenance ranker, ScoringModel.
  arXiv  Detail & Related papers  (2026-05-07T05:27:45Z)
• Erasing Without Remembering: Implicit Knowledge Forgetting in Large Language Models [81.62767292169225]
  We investigate knowledge forgetting in large language models with a focus on its generalisation.<n>We propose PerMU, a novel probability perturbation-based unlearning paradigm.<n>Experiments are conducted on a diverse range of datasets, including TOFU, Harry Potter, ZsRE, WMDP, and MUSE.
  arXiv  Detail & Related papers  (2025-02-27T11:03:33Z)
• Paloma: A Benchmark for Evaluating Language Model Fit [112.481957296585]
  Evaluations of language models (LMs) commonly report perplexity on monolithic data held out from training.<n>We introduce Perplexity Analysis for Language Model Assessment (Paloma), a benchmark to measure LM fit to 546 English and code domains.
  arXiv  Detail & Related papers  (2023-12-16T19:12:45Z)
• Telling Left from Right: Identifying Geometry-Aware Semantic Correspondence [80.6840060272386]
  This paper identifies the importance of being geometry-aware for semantic correspondence. We show that incorporating this information can markedly enhance semantic correspondence performance. Our method achieves a PCK@0.10 score of 65.4 (zero-shot) and 85.6 (supervised) on the challenging SPair-71k dataset.
  arXiv  Detail & Related papers  (2023-11-28T18:45:13Z)
• Retrieval-based Disentangled Representation Learning with Natural Language Supervision [61.75109410513864]
  We present Vocabulary Disentangled Retrieval (VDR), a retrieval-based framework that harnesses natural language as proxies of the underlying data variation to drive disentangled representation learning. Our approach employ a bi-encoder model to represent both data and natural language in a vocabulary space, enabling the model to distinguish intrinsic dimensions that capture characteristics within data through its natural language counterpart, thus disentanglement.
  arXiv  Detail & Related papers  (2022-12-15T10:20:42Z)
• Quark: Controllable Text Generation with Reinforced Unlearning [68.07749519374089]
  Large-scale language models often learn behaviors that are misaligned with user expectations. We introduce Quantized Reward Konditioning (Quark), an algorithm for optimizing a reward function that quantifies an (un)wanted property. For unlearning toxicity, negative sentiment, and repetition, our experiments show that Quark outperforms both strong baselines and state-of-the-art reinforcement learning methods.
  arXiv  Detail & Related papers  (2022-05-26T21:11:51Z)
• Fast and Robust Unsupervised Contextual Biasing for Speech Recognition [16.557586847398778]
  We propose an alternative approach that does not entail explicit contextual language model. We derive the bias score for every word in the system vocabulary from the training corpus. We show significant improvement in recognition accuracy when the relevant context is available.
  arXiv  Detail & Related papers  (2020-05-04T17:29:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.