ConstraintMatch for Semi-constrained Clustering
- URL: http://arxiv.org/abs/2311.15395v1
- Date: Sun, 26 Nov 2023 19:31:52 GMT
- Title: ConstraintMatch for Semi-constrained Clustering
- Authors: Jann Goschenhofer, Bernd Bischl, Zsolt Kira
- Abstract summary: Constrained clustering allows the training of classification models using pairwise constraints only, which are weak and relatively easy to mine.
We propose a semi-supervised context whereby a large amount of textitunconstrained data is available alongside a smaller set of constraints, and propose textitConstraintMatch to leverage such unconstrained data.
- Score: 32.92933231199262
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Constrained clustering allows the training of classification models using
pairwise constraints only, which are weak and relatively easy to mine, while
still yielding full-supervision-level model performance. While they perform
well even in the absence of the true underlying class labels, constrained
clustering models still require large amounts of binary constraint annotations
for training. In this paper, we propose a semi-supervised context whereby a
large amount of \textit{unconstrained} data is available alongside a smaller
set of constraints, and propose \textit{ConstraintMatch} to leverage such
unconstrained data. While a great deal of progress has been made in
semi-supervised learning using full labels, there are a number of challenges
that prevent a naive application of the resulting methods in the
constraint-based label setting. Therefore, we reason about and analyze these
challenges, specifically 1) proposing a \textit{pseudo-constraining} mechanism
to overcome the confirmation bias, a major weakness of pseudo-labeling, 2)
developing new methods for pseudo-labeling towards the selection of
\textit{informative} unconstrained samples, 3) showing that this also allows
the use of pairwise loss functions for the initial and auxiliary losses which
facilitates semi-constrained model training. In extensive experiments, we
demonstrate the effectiveness of ConstraintMatch over relevant baselines in
both the regular clustering and overclustering scenarios on five challenging
benchmarks and provide analyses of its several components.
Related papers
- Stable Cluster Discrimination for Deep Clustering [7.175082696240088]
Deep clustering can optimize representations of instances (i.e., representation learning) and explore the inherent data distribution.
The coupled objective implies a trivial solution that all instances collapse to the uniform features.
In this work, we first show that the prevalent discrimination task in supervised learning is unstable for one-stage clustering.
A novel stable cluster discrimination (SeCu) task is proposed and a new hardness-aware clustering criterion can be obtained accordingly.
arXiv Detail & Related papers (2023-11-24T06:43:26Z) - Adaptive End-to-End Metric Learning for Zero-Shot Cross-Domain Slot
Filling [2.6056468338837457]
Slot filling poses a critical challenge to handle a novel domain whose samples are never seen during training.
Most prior works deal with this problem in a two-pass pipeline manner based on metric learning.
We propose a new adaptive end-to-end metric learning scheme for the challenging zero-shot slot filling.
arXiv Detail & Related papers (2023-10-23T19:01:16Z) - On Regularization and Inference with Label Constraints [62.60903248392479]
We compare two strategies for encoding label constraints in a machine learning pipeline, regularization with constraints and constrained inference.
For regularization, we show that it narrows the generalization gap by precluding models that are inconsistent with the constraints.
For constrained inference, we show that it reduces the population risk by correcting a model's violation, and hence turns the violation into an advantage.
arXiv Detail & Related papers (2023-07-08T03:39:22Z) - Semi-Supervised Constrained Clustering: An In-Depth Overview, Ranked
Taxonomy and Future Research Directions [2.5957372084704238]
The research area of constrained clustering has grown significantly over the years.
No unifying overview is available to easily understand the wide variety of available methods, constraints and benchmarks.
This study presents in-detail the background of constrained clustering and provides a novel ranked taxonomy of the types of constraints that can be used in constrained clustering.
arXiv Detail & Related papers (2023-02-28T17:46:31Z) - Optimal Decision Trees For Interpretable Clustering with Constraints
(Extended Version) [7.799182201815762]
Constrained clustering is a semi-supervised task that employs a limited amount of labelled data, formulated as constraints.
We present a novel SAT-based framework for interpretable clustering that supports clustering constraints.
We also present new insight into the trade-off between interpretability and satisfaction of such user-provided constraints.
arXiv Detail & Related papers (2023-01-30T05:34:49Z) - SoftMatch: Addressing the Quantity-Quality Trade-off in Semi-supervised
Learning [101.86916775218403]
This paper revisits the popular pseudo-labeling methods via a unified sample weighting formulation.
We propose SoftMatch to overcome the trade-off by maintaining both high quantity and high quality of pseudo-labels during training.
In experiments, SoftMatch shows substantial improvements across a wide variety of benchmarks, including image, text, and imbalanced classification.
arXiv Detail & Related papers (2023-01-26T03:53:25Z) - An Embarrassingly Simple Approach to Semi-Supervised Few-Shot Learning [58.59343434538218]
We propose a simple but quite effective approach to predict accurate negative pseudo-labels of unlabeled data from an indirect learning perspective.
Our approach can be implemented in just few lines of code by only using off-the-shelf operations.
arXiv Detail & Related papers (2022-09-28T02:11:34Z) - Rethinking Clustering-Based Pseudo-Labeling for Unsupervised
Meta-Learning [146.11600461034746]
Method for unsupervised meta-learning, CACTUs, is a clustering-based approach with pseudo-labeling.
This approach is model-agnostic and can be combined with supervised algorithms to learn from unlabeled data.
We prove that the core reason for this is lack of a clustering-friendly property in the embedding space.
arXiv Detail & Related papers (2022-09-27T19:04:36Z) - Unsupervised Learning of Debiased Representations with Pseudo-Attributes [85.5691102676175]
We propose a simple but effective debiasing technique in an unsupervised manner.
We perform clustering on the feature embedding space and identify pseudoattributes by taking advantage of the clustering results.
We then employ a novel cluster-based reweighting scheme for learning debiased representation.
arXiv Detail & Related papers (2021-08-06T05:20:46Z) - A Framework for Deep Constrained Clustering [19.07636653413663]
Constrained clustering formulations exist for popular algorithms such as k-means, mixture models, and spectral clustering but have several limitations.
Here we explore a deep learning framework for constrained clustering and in particular explore how it can extend the field of constrained clustering.
We show that our framework can not only handle standard together/apart constraints (without the well documented negative effects reported earlier) generated from labeled side information.
We propose an efficient training paradigm that is generally applicable to these four types of constraints.
arXiv Detail & Related papers (2021-01-07T22:49:06Z) - An Integer Linear Programming Framework for Mining Constraints from Data [81.60135973848125]
We present a general framework for mining constraints from data.
In particular, we consider the inference in structured output prediction as an integer linear programming (ILP) problem.
We show that our approach can learn to solve 9x9 Sudoku puzzles and minimal spanning tree problems from examples without providing the underlying rules.
arXiv Detail & Related papers (2020-06-18T20:09:53Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.