Clone-Resistant Weights in Metric Spaces: A Framework for Handling Redundancy Bias

• URL: http://arxiv.org/abs/2502.03576v1
• Date: Wed, 05 Feb 2025 19:50:51 GMT
• Title: Clone-Resistant Weights in Metric Spaces: A Framework for Handling Redundancy Bias
• Authors: Damien Berriaud, Roger Wattenhofer,
• Abstract summary: We are given a set of elements in a metric space. The distribution of the elements is arbitrary, possibly adversarial. Can we weigh the elements in a way that is resistant to such (adversarial) manipulations?
• Score: 23.27199615640474
• License:
• Abstract: We are given a set of elements in a metric space. The distribution of the elements is arbitrary, possibly adversarial. Can we weigh the elements in a way that is resistant to such (adversarial) manipulations? This problem arises in various contexts. For instance, the elements could represent data points, requiring robust domain adaptation. Alternatively, they might represent tasks to be aggregated into a benchmark; or questions about personal political opinions in voting advice applications. This article introduces a theoretical framework for dealing with such problems. We propose clone-proof representation functions as a solution concept. These functions distribute importance across elements of a set such that similar objects (``clones'') share (some of) their weights, thus avoiding a potential bias introduced by their multiplicity. Our framework extends the maximum uncertainty principle to accommodate general metric spaces and includes a set of axioms - symmetry, continuity, and clone-proofness - that guide the construction of representation functions. Finally, we address the existence of representation functions satisfying our axioms in the significant case of Euclidean spaces and propose a general method for their construction.

Related papers

• Symbolic Disentangled Representations for Images [83.88591755871734]
  We propose ArSyD (Architecture for Disentanglement), which represents each generative factor as a vector of the same dimension as the resulting representation. We study ArSyD on the dSprites and CLEVR datasets and provide a comprehensive analysis of the learned symbolic disentangled representations.
  arXiv  Detail & Related papers  (2024-12-25T09:20:13Z)
• Knowledge Graph Embedding by Normalizing Flows [26.732799849307316]
  Key to knowledge graph embedding (KGE) is to choose a proper representation space, e.g., point-wise Euclidean space and complex vector space. We propose a unified perspective of embedding and introduce uncertainty into KGE from the view of group theory.
  arXiv  Detail & Related papers  (2024-09-30T06:04:34Z)
• Equivariant Frames and the Impossibility of Continuous Canonicalization [10.02508080274145]
  We show that unweighted frame-averaging can turn a smooth, non-symmetric function into a discontinuous, symmetric function. We construct efficient and continuous weighted frames for the actions of $SO(2)$, $SO(3)$, and $S_n$ on point clouds.
  arXiv  Detail & Related papers  (2024-02-25T12:40:42Z)
• Object-centric architectures enable efficient causal representation learning [51.6196391784561]
  We show that when the observations are of multiple objects, the generative function is no longer injective and disentanglement fails in practice. We develop an object-centric architecture that leverages weak supervision from sparse perturbations to disentangle each object's properties. This approach is more data-efficient in the sense that it requires significantly fewer perturbations than a comparable approach that encodes to a Euclidean space.
  arXiv  Detail & Related papers  (2023-10-29T16:01:03Z)
• Algebras of actions in an agent's representations of the world [51.06229789727133]
  We use our framework to reproduce the symmetry-based representations from the symmetry-based disentangled representation learning formalism. We then study the algebras of the transformations of worlds with features that occur in simple reinforcement learning scenarios. Using computational methods, that we developed, we extract the algebras of the transformations of these worlds and classify them according to their properties.
  arXiv  Detail & Related papers  (2023-10-02T18:24:51Z)
• Provably Efficient Causal Model-Based Reinforcement Learning for Systematic Generalization [30.456180468318305]
  In the sequential decision making setting, an agent aims to achieve systematic generalization over a large, possibly infinite, set of environments. In this paper, we provide a tractable formulation of systematic generalization by employing a causal viewpoint. Under specific structural assumptions, we provide a simple learning algorithm that guarantees any desired planning error up to an unavoidable sub-optimality term.
  arXiv  Detail & Related papers  (2022-02-14T08:34:51Z)
• SPANet: Generalized Permutationless Set Assignment for Particle Physics using Symmetry Preserving Attention [62.43586180025247]
  Collisions at the Large Hadron Collider produce variable-size sets of observed particles. Physical symmetries of decay products complicate assignment of observed particles to decay products. We introduce a novel method for constructing symmetry-preserving attention networks.
  arXiv  Detail & Related papers  (2021-06-07T18:18:20Z)
• A Class of Dimension-free Metrics for the Convergence of Empirical Measures [6.253771639590562]
  We show that under the proposed metrics, the convergence of empirical measures in high dimensions is free of the curse of dimensionality (CoD) Examples of selected test function spaces include the kernel reproducing Hilbert spaces, Barron space, and flow-induced function spaces. We show that the proposed class of metrics is a powerful tool to analyze the convergence of empirical measures in high dimensions without CoD.
  arXiv  Detail & Related papers  (2021-04-24T23:27:40Z)
• Efficient State Abstraction using Object-centered Predicates for Manipulation Planning [86.24148040040885]
  We propose an object-centered representation that permits characterizing a much wider set of possible changes in configuration spaces. Based on this representation, we define universal planning operators for picking and placing actions that permits generating plans with geometric and force consistency.
  arXiv  Detail & Related papers  (2020-07-16T10:52:53Z)
• Infinite Feature Selection: A Graph-based Feature Filtering Approach [78.63188057505012]
  We propose a filtering feature selection framework that considers subsets of features as paths in a graph. Going to infinite allows to constrain the computational complexity of the selection process. We show that Inf-FS behaves better in almost any situation, that is, when the number of features to keep are fixed a priori.
  arXiv  Detail & Related papers  (2020-06-15T07:20:40Z)
• Matrix integrals over unitary groups: An application of Schur-Weyl duality [4.927579219242575]
  The integral formulae pertaining to the unitary group $mathsfU(d)$ have been comprehensively reviewed. Schur-Weyl duality serves as a bridge, establishing a profound connection between the representation theory of finite groups and that of classical Lie groups.
  arXiv  Detail & Related papers  (2014-08-17T00:50:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.