A Comparison of Document Similarity Algorithms

• URL: http://arxiv.org/abs/2304.01330v1
• Date: Mon, 3 Apr 2023 19:50:55 GMT
• Title: A Comparison of Document Similarity Algorithms
• Abstract summary: It addresses the most effective document similarity algorithm by categorizing them into 3 types of document similarity algorithms. The most effective algorithms in each category are also compared in our work using a series of benchmark datasets and evaluations.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Document similarity is an important part of Natural Language Processing and is most commonly used for plagiarism-detection and text summarization. Thus, finding the overall most effective document similarity algorithm could have a major positive impact on the field of Natural Language Processing. This report sets out to examine the numerous document similarity algorithms, and determine which ones are the most useful. It addresses the most effective document similarity algorithm by categorizing them into 3 types of document similarity algorithms: statistical algorithms, neural networks, and corpus/knowledge-based algorithms. The most effective algorithms in each category are also compared in our work using a series of benchmark datasets and evaluations that test every possible area that each algorithm could be used in.

Related papers

• Exploring the Effectiveness of Abstract Syntax Tree Patterns for Algorithm Recognition [0.41998444721319217]
  In this paper, we investigate how well our approach based on the abstract syntax tree of a program performs for automatic algorithm recognition.<n>A prototype consisting of: A domain-specific language designed to capture the key features of an algorithm and used to express a search pattern on the abstract syntax tree.<n>We evaluate our prototype on a subset of the BigCloneEval benchmark containing algorithms like Fibonacci, Bubble Sort, and Binary Search.
  arXiv  Detail & Related papers  (2026-05-07T12:16:16Z)
• Relation-aware Ensemble Learning for Knowledge Graph Embedding [68.94900786314666]
  We propose to learn an ensemble by leveraging existing methods in a relation-aware manner. exploring these semantics using relation-aware ensemble leads to a much larger search space than general ensemble methods. We propose a divide-search-combine algorithm RelEns-DSC that searches the relation-wise ensemble weights independently.
  arXiv  Detail & Related papers  (2023-10-13T07:40:12Z)
• Computational thematics: Comparing algorithms for clustering the genres of literary fiction [0.0]
  This paper compares a variety of algorithms for unsupervised learning of thematic similarities between texts. Every combination of algorithms is given a task to cluster a corpus of books belonging to four pre-tagged genres of fiction. This clustering is validated against the "ground truth" genre labels.
  arXiv  Detail & Related papers  (2023-05-18T18:32:03Z)
• A Gold Standard Dataset for the Reviewer Assignment Problem [117.59690218507565]
  "Similarity score" is a numerical estimate of the expertise of a reviewer in reviewing a paper. Our dataset consists of 477 self-reported expertise scores provided by 58 researchers. For the task of ordering two papers in terms of their relevance for a reviewer, the error rates range from 12%-30% in easy cases to 36%-43% in hard cases.
  arXiv  Detail & Related papers  (2023-03-23T16:15:03Z)
• Document Provenance and Authentication through Authorship Classification [5.2545206693029884]
  We propose an ensemble-based text-processing framework for the classification of single and multi-authored documents. The proposed framework incorporates several state-of-the-art text classification algorithms. The framework is evaluated on a large-scale benchmark dataset.
  arXiv  Detail & Related papers  (2023-03-02T12:26:03Z)
• Comparative Study of Long Document Classification [0.0]
  We revisit long document classification using standard machine learning approaches. We benchmark approaches ranging from simple Naive Bayes to complex BERT on six standard text classification datasets.
  arXiv  Detail & Related papers  (2021-11-01T04:51:51Z)
• Algorithm Selection on a Meta Level [58.720142291102135]
  We introduce the problem of meta algorithm selection, which essentially asks for the best way to combine a given set of algorithm selectors. We present a general methodological framework for meta algorithm selection as well as several concrete learning methods as instantiations of this framework.
  arXiv  Detail & Related papers  (2021-07-20T11:23:21Z)
• Estimating leverage scores via rank revealing methods and randomization [50.591267188664666]
  We study algorithms for estimating the statistical leverage scores of rectangular dense or sparse matrices of arbitrary rank. Our approach is based on combining rank revealing methods with compositions of dense and sparse randomized dimensionality reduction transforms.
  arXiv  Detail & Related papers  (2021-05-23T19:21:55Z)
• Towards Optimally Efficient Tree Search with Deep Learning [76.64632985696237]
  This paper investigates the classical integer least-squares problem which estimates signals integer from linear models. The problem is NP-hard and often arises in diverse applications such as signal processing, bioinformatics, communications and machine learning. We propose a general hyper-accelerated tree search (HATS) algorithm by employing a deep neural network to estimate the optimal estimation for the underlying simplified memory-bounded A* algorithm.
  arXiv  Detail & Related papers  (2021-01-07T08:00:02Z)
• Efficient Clustering from Distributions over Topics [0.0]
  We present an approach that relies on the results of a topic modeling algorithm over documents in a collection as a means to identify smaller subsets of documents where the similarity function can be computed. This approach has proved to obtain promising results when identifying similar documents in the domain of scientific publications.
  arXiv  Detail & Related papers  (2020-12-15T10:52:19Z)
• A Topological Method for Comparing Document Semantics [0.0]
  We propose a novel algorithm for comparing semantics similarity between two documents. Our experiments are conducted on a document dataset with human judges' results. Our algorithm can produce highly human-consistent results, and also beats most state-of-the-art methods though ties with NLTK.
  arXiv  Detail & Related papers  (2020-12-08T04:21:40Z)
• A Systematic Characterization of Sampling Algorithms for Open-ended Language Generation [71.31905141672529]
  We study the widely adopted ancestral sampling algorithms for auto-regressive language models. We identify three key properties that are shared among them: entropy reduction, order preservation, and slope preservation. We find that the set of sampling algorithms that satisfies these properties performs on par with the existing sampling algorithms.
  arXiv  Detail & Related papers  (2020-09-15T17:28:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.