Comparison of Outlier Detection Algorithms on String Data

• URL: http://arxiv.org/abs/2603.11049v1
• Date: Tue, 13 Jan 2026 13:36:39 GMT
• Title: Comparison of Outlier Detection Algorithms on String Data
• Abstract summary: A robust string data outlier detection algorithm could assist with data cleaning or anomaly detection in system log files.<n>In this thesis, we compare two string outlier detection algorithms. Firstly, we introduce a variant of the well-known local outlier factor algorithm, which we tailor to detect outliers on string data.<n> Secondly, we introduce a new kind of outlier detection algorithm based on the hierarchical left regular expression learner, which infers a regular expression for the expected data.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Outlier detection is a well-researched and crucial problem in machine learning. However, there is little research on string data outlier detection, as most literature focuses on outlier detection of numerical data. A robust string data outlier detection algorithm could assist with data cleaning or anomaly detection in system log files. In this thesis, we compare two string outlier detection algorithms. Firstly, we introduce a variant of the well-known local outlier factor algorithm, which we tailor to detect outliers on string data using the Levenshtein measure to calculate the density of the dataset. We present a differently weighted Levenshtein measure, which considers hierarchical character classes and can be used to tune the algorithm to a specific string dataset. Secondly, we introduce a new kind of outlier detection algorithm based on the hierarchical left regular expression learner, which infers a regular expression for the expected data. Using various datasets and parameters, we experimentally show that both algorithms can conceptually find outliers in string data. We show that the regular expression-based algorithm is especially good at finding outliers if the expected values have a distinct structure that is sufficiently different from the structure of the outliers. In contrast, the local outlier factor algorithms are best at finding outliers if their edit distance to the expected data is sufficiently distinct from the edit distance between the expected data.

Related papers

• Outlier detection in mixed-attribute data: a semi-supervised approach with fuzzy approximations and relative entropy [44.721694491724406]
  Outlier detection is a critical task in data mining, aimed at identifying objects that significantly deviate from the norm.<n>This paper introduces a semi-supervised outlier detection method, namely fuzzy rough sets-based outlier detection (FROD)<n> Experimental results on 16 public datasets show that FROD is comparable with or better than leading detection algorithms.
  arXiv  Detail & Related papers  (2025-12-22T02:41:43Z)
• Consistency-guided semi-supervised outlier detection in heterogeneous data using fuzzy rough sets [45.9876416284051]
  Outlier detection aims to find samples that behave differently from the majority of the data.<n>Semi-supervised detection methods can utilize the supervision of partial labels, thus reducing false positive rates.<n>We propose a consistency-guided outlier detection algorithm (COD) for heterogeneous data with the fuzzy rough set theory in a semi-supervised manner.
  arXiv  Detail & Related papers  (2025-12-22T02:41:08Z)
• Feature Explosion: a generic optimization strategy for outlier detection algorithms [8.206124331448931]
  Outlier detection tasks aim at discovering potential issues or opportunities.<n>Thousands of outlier detection algorithms have been proposed.<n>The root cause of this redundancy lies in the current highly customized (i.e., non-generic) optimization strategies.
  arXiv  Detail & Related papers  (2025-02-08T08:58:12Z)
• Fuzzy Granule Density-Based Outlier Detection with Multi-Scale Granular Balls [65.44462297594308]
  Outlier detection refers to the identification of anomalous samples that deviate significantly from the distribution of normal data.<n>Most unsupervised outlier detection methods are carefully designed to detect specified outliers.<n>We propose a fuzzy rough sets-based multi-scale outlier detection method to identify various types of outliers.
  arXiv  Detail & Related papers  (2025-01-06T12:35:51Z)
• Unsupervised anomaly detection algorithms on real-world data: how many do we need? [1.4610038284393165]
  This study is the largest comparison of unsupervised anomaly detection algorithms to date. On the local datasets the $k$NN ($k$-nearest neighbor) algorithm comes out on top. On the global datasets the EIF (extended isolation forest) algorithm performs the best.
  arXiv  Detail & Related papers  (2023-05-01T09:27:42Z)
• A Gold Standard Dataset for the Reviewer Assignment Problem [70.45113777449373]
  "Similarity score" is a numerical estimate of the expertise of a reviewer in reviewing a paper.<n>Key challenge in comparing existing algorithms and developing better algorithms is the lack of publicly available gold-standard data.<n>We collect a novel dataset of similarity scores that we release to the research community.
  arXiv  Detail & Related papers  (2023-03-23T16:15:03Z)
• From One to Many: A Deep Learning Coincident Gravitational-Wave Search [58.720142291102135]
  We construct a two-detector search for gravitational waves from binary black hole mergers using neural networks trained on non-spinning binary black hole data from a single detector. We find that none of these simple two-detector networks are capable of improving the sensitivity over applying networks individually to the data from the detectors.
  arXiv  Detail & Related papers  (2021-08-24T13:25:02Z)
• Unsupervised Outlier Detection using Memory and Contrastive Learning [53.77693158251706]
  We think outlier detection can be done in the feature space by measuring the feature distance between outliers and inliers. We propose a framework, MCOD, using a memory module and a contrastive learning module. Our proposed MCOD achieves a considerable performance and outperforms nine state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-07-27T07:35:42Z)
• Achieving differential privacy for $k$-nearest neighbors based outlier detection by data partitioning [0.3437656066916039]
  We develop a differentially private ($epsilon$-DP) approach for $k$-NN based outlier detection. We propose a method for $k$-NN based outlier detection by separating the procedure into a fitting step on reference inlier data and then apply the outlier classifier to new data. Our approach yields nearly optimal performance on real-world data with varying dimensions when compared to the non-private versions of $k$-NN.
  arXiv  Detail & Related papers  (2021-04-16T07:35:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.