Predicate Hierarchies Improve Few-Shot State Classification

• URL: http://arxiv.org/abs/2502.12481v1
• Date: Tue, 18 Feb 2025 03:08:37 GMT
• Title: Predicate Hierarchies Improve Few-Shot State Classification
• Authors: Emily Jin, Joy Hsu, Jiajun Wu,
• Abstract summary: PHIER is an object-centric scene encoder, self-supervised losses that infer semantic relations between predicates, and a hyperbolic distance metric that captures hierarchical structure. We evaluate PHIER in the CALVIN and BEHAVIOR robotic environments and show that PHIER significantly outperforms existing methods in few-shot, out-of-distribution state classification.
• Score: 12.683069420087962
• License:
• Abstract: State classification of objects and their relations is core to many long-horizon tasks, particularly in robot planning and manipulation. However, the combinatorial explosion of possible object-predicate combinations, coupled with the need to adapt to novel real-world environments, makes it a desideratum for state classification models to generalize to novel queries with few examples. To this end, we propose PHIER, which leverages predicate hierarchies to generalize effectively in few-shot scenarios. PHIER uses an object-centric scene encoder, self-supervised losses that infer semantic relations between predicates, and a hyperbolic distance metric that captures hierarchical structure; it learns a structured latent space of image-predicate pairs that guides reasoning over state classification queries. We evaluate PHIER in the CALVIN and BEHAVIOR robotic environments and show that PHIER significantly outperforms existing methods in few-shot, out-of-distribution state classification, and demonstrates strong zero- and few-shot generalization from simulated to real-world tasks. Our results demonstrate that leveraging predicate hierarchies improves performance on state classification tasks with limited data.

Related papers

• From Logits to Hierarchies: Hierarchical Clustering made Simple [16.132657141993548]
  We show that a lightweight procedure implemented on top of pre-trained non-hierarchical clustering models outperforms models designed specifically for hierarchical clustering. Our proposed approach is computationally efficient and applicable to any pre-trained clustering model that outputs logits, without requiring any fine-tuning.
  arXiv  Detail & Related papers  (2024-10-10T12:27:45Z)
• Coherent Entity Disambiguation via Modeling Topic and Categorical Dependency [87.16283281290053]
  Previous entity disambiguation (ED) methods adopt a discriminative paradigm, where prediction is made based on matching scores between mention context and candidate entities. We propose CoherentED, an ED system equipped with novel designs aimed at enhancing the coherence of entity predictions. We achieve new state-of-the-art results on popular ED benchmarks, with an average improvement of 1.3 F1 points.
  arXiv  Detail & Related papers  (2023-11-06T16:40:13Z)
• Prototype-based Embedding Network for Scene Graph Generation [105.97836135784794]
  Current Scene Graph Generation (SGG) methods explore contextual information to predict relationships among entity pairs. Due to the diverse visual appearance of numerous possible subject-object combinations, there is a large intra-class variation within each predicate category. Prototype-based Embedding Network (PE-Net) models entities/predicates with prototype-aligned compact and distinctive representations. PL is introduced to help PE-Net efficiently learn such entitypredicate matching, and Prototype Regularization (PR) is devised to relieve the ambiguous entity-predicate matching.
  arXiv  Detail & Related papers  (2023-03-13T13:30:59Z)
• Hierarchical Relationships: A New Perspective to Enhance Scene Graph Generation [8.28849026314542]
  This paper presents a finding that leveraging the hierarchical structures among labels for relationships and objects can substantially improve the performance of scene graph generation systems. We introduce a Bayesian prediction head to jointly predict the super-category of relationships between a pair of object instances. Experiments on the Visual Genome dataset show its strong performance, particularly in predicate classifications and zero-shot settings.
  arXiv  Detail & Related papers  (2023-03-13T04:16:42Z)
• SLOTH: Structured Learning and Task-based Optimization for Time Series Forecasting on Hierarchies [16.12477042879166]
  The hierarchical time series (HTS) forecasting includes two sub-tasks, i.e., forecasting and reconciliation. In this paper, we propose two novel tree-based feature integration mechanisms, i.e., top-down convolution and bottom-up attention. Unlike most previous reconciliation methods which either rely on strong assumptions or focus on coherent constraints only, we utilize deep neural optimization networks.
  arXiv  Detail & Related papers  (2023-02-11T10:50:33Z)
• HIER: Metric Learning Beyond Class Labels via Hierarchical Regularization [17.575016642108253]
  We propose a new regularization method, dubbed HIER, to discover the latent semantic hierarchy of training data. HIER achieves this goal with no annotation for the semantic hierarchy but by learning hierarchical proxies in hyperbolic spaces.
  arXiv  Detail & Related papers  (2022-12-29T11:05:47Z)
• Weakly-supervised Action Localization via Hierarchical Mining [76.00021423700497]
  Weakly-supervised action localization aims to localize and classify action instances in the given videos temporally with only video-level categorical labels. We propose a hierarchical mining strategy under video-level and snippet-level manners, i.e., hierarchical supervision and hierarchical consistency mining. We show that HiM-Net outperforms existing methods on THUMOS14 and ActivityNet1.3 datasets with large margins by hierarchically mining the supervision and consistency.
  arXiv  Detail & Related papers  (2022-06-22T12:19:09Z)
• Modeling Heterogeneous Hierarchies with Relation-specific Hyperbolic Cones [64.75766944882389]
  We present ConE (Cone Embedding), a KG embedding model that is able to simultaneously model multiple hierarchical as well as non-hierarchical relations in a knowledge graph. In particular, ConE uses cone containment constraints in different subspaces of the hyperbolic embedding space to capture multiple heterogeneous hierarchies. Our approach yields new state-of-the-art Hits@1 of 45.3% on WN18RR and 16.1% on DDB14 (0.231 MRR)
  arXiv  Detail & Related papers  (2021-10-28T07:16:08Z)
• HRKD: Hierarchical Relational Knowledge Distillation for Cross-domain Language Model Compression [53.90578309960526]
  Large pre-trained language models (PLMs) have shown overwhelming performances compared with traditional neural network methods. We propose a hierarchical relational knowledge distillation (HRKD) method to capture both hierarchical and domain relational information.
  arXiv  Detail & Related papers  (2021-10-16T11:23:02Z)
• Automated Concatenation of Embeddings for Structured Prediction [75.44925576268052]
  We propose Automated Concatenation of Embeddings (ACE) to automate the process of finding better concatenations of embeddings for structured prediction tasks. We follow strategies in reinforcement learning to optimize the parameters of the controller and compute the reward based on the accuracy of a task model.
  arXiv  Detail & Related papers  (2020-10-10T14:03:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.