Association via Entropy Reduction

• URL: http://arxiv.org/abs/2511.04901v1
• Date: Fri, 07 Nov 2025 01:03:48 GMT
• Title: Association via Entropy Reduction
• Abstract summary: We provide a different score, aver, and observe, on a dataset with ground truth marking for association, that aver does better at finding assciated pairs than tf-idf.<n>We observe that (1) aver has a natural threshold for declaring pairs as unassociated while tf-idf does not, (2) aver can distinguish between pairs of documents for which tf-idf gives a score of 1.0, (3) aver can be applied to larger collections of documents than pairs while tf-idf cannot, and (4) that aver is derived from entropy under a simple statistical model.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Prior to recent successes using neural networks, term frequency-inverse document frequency (tf-idf) was clearly regarded as the best choice for identifying documents related to a query. We provide a different score, aver, and observe, on a dataset with ground truth marking for association, that aver does do better at finding assciated pairs than tf-idf. This example involves finding associated vertices in a large graph and that may be an area where neural networks are not currently an obvious best choice. Beyond this one anecdote, we observe that (1) aver has a natural threshold for declaring pairs as unassociated while tf-idf does not, (2) aver can distinguish between pairs of documents for which tf-idf gives a score of 1.0, (3) aver can be applied to larger collections of documents than pairs while tf-idf cannot, and (4) that aver is derived from entropy under a simple statistical model while tf-idf is a construction designed to achieve a certain goal and hence aver may be more "natural." To be fair, we also observe that (1) writing down and computing the aver score for a pair is more complex than for tf-idf and (2) that the fact that the aver score is naturally scale-free makes it more complicated to interpret aver scores.

Related papers

• A Fisher's exact test justification of the TF-IDF term-weighting scheme [0.0]
  Term frequency-inverse document frequency, or TF-IDF, is arguably the most celebrated mathematical expression in the history of information retrieval.<n>This paper justifies the use of TF-IDF to the statistics community by demonstrating how the famed expression can be understood from a significance testing perspective.
  arXiv  Detail & Related papers  (2025-07-21T15:54:23Z)
• Tensor Convolutional Network for Higher-Order Interaction Prediction in Sparse Tensors [74.31355755781343]
  We propose TCN, an accurate and compatible tensor convolutional network that integrates seamlessly with TF methods for predicting top-k interactions.<n>We show that TCN integrated with a TF method outperforms competitors, including TF methods and a hyperedge prediction method.
  arXiv  Detail & Related papers  (2025-03-14T18:22:20Z)
• The Heterophilic Snowflake Hypothesis: Training and Empowering GNNs for Heterophilic Graphs [59.03660013787925]
  We introduce the Heterophily Snowflake Hypothesis and provide an effective solution to guide and facilitate research on heterophilic graphs. Our observations show that our framework acts as a versatile operator for diverse tasks. It can be integrated into various GNN frameworks, boosting performance in-depth and offering an explainable approach to choosing the optimal network depth.
  arXiv  Detail & Related papers  (2024-06-18T12:16:00Z)
• A Comparative Study on TF-IDF feature Weighting Method and its Analysis using Unstructured Dataset [0.5156484100374058]
  Term Frequency-Inverse Document Frequency (TF-IDF) and Natural Language Processing (NLP) are the most highly used information retrieval methods in text classification. We have investigated and analyzed the feature weighting method for text classification on unstructured data. The proposed model considered two features N-Grams and TF-IDF on IMDB movie reviews and Amazon Alexa reviews dataset for sentiment analysis.
  arXiv  Detail & Related papers  (2023-08-08T04:27:34Z)
• HIORE: Leveraging High-order Interactions for Unified Entity Relation Extraction [85.80317530027212]
  We propose HIORE, a new method for unified entity relation extraction. The key insight is to leverage the complex association among word pairs, which contains richer information than the first-order word-by-word interactions. Experiments show that HIORE achieves the state-of-the-art performance on relation extraction and an improvement of 1.11.8 F1 points over the prior best unified model.
  arXiv  Detail & Related papers  (2023-05-07T14:57:42Z)
• HL-Net: Heterophily Learning Network for Scene Graph Generation [90.2766568914452]
  We propose a novel Heterophily Learning Network (HL-Net) to explore the homophily and heterophily between objects/relationships in scene graphs. HL-Net comprises the following 1) an adaptive reweighting transformer module, which adaptively integrates the information from different layers to exploit both the heterophily and homophily in objects. We conducted extensive experiments on two public datasets: Visual Genome (VG) and Open Images (OI)
  arXiv  Detail & Related papers  (2022-05-03T06:00:29Z)
• Document-Level Relation Extraction with Sentences Importance Estimation and Focusing [52.069206266557266]
  Document-level relation extraction (DocRE) aims to determine the relation between two entities from a document of multiple sentences. We propose a Sentence Estimation and Focusing (SIEF) framework for DocRE, where we design a sentence importance score and a sentence focusing loss. Experimental results on two domains show that our SIEF not only improves overall performance, but also makes DocRE models more robust.
  arXiv  Detail & Related papers  (2022-04-27T03:20:07Z)
• Eider: Evidence-enhanced Document-level Relation Extraction [56.71004595444816]
  Document-level relation extraction (DocRE) aims at extracting semantic relations among entity pairs in a document. We propose a three-stage evidence-enhanced DocRE framework consisting of joint relation and evidence extraction, evidence-centered relation extraction (RE), and fusion of extraction results.
  arXiv  Detail & Related papers  (2021-06-16T09:43:16Z)
• Learning Informative Representations of Biomedical Relations with Latent Variable Models [2.4366811507669115]
  We propose a latent variable model with an arbitrarily flexible distribution to represent the relation between an entity pair. We demonstrate that our model achieves results competitive with strong baselines for both tasks while having fewer parameters and being significantly faster to train.
  arXiv  Detail & Related papers  (2020-11-20T08:56:31Z)
• Double Graph Based Reasoning for Document-level Relation Extraction [29.19714611415326]
  Document-level relation extraction aims to extract relations among entities within a document. We propose Graph Aggregation-and-Inference Network (GAIN) featuring double graphs. Experiments on the public dataset, DocRED, show GAIN achieves a significant performance improvement (2.85 on F1) over the previous state-of-the-art.
  arXiv  Detail & Related papers  (2020-09-29T03:41:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.