Sparse Mutual Information Graph Averaging for Improving Random Indexing Embeddings

• URL: http://arxiv.org/abs/2608.05724v1
• Date: Thu, 06 Aug 2026 08:07:18 GMT
• Title: Sparse Mutual Information Graph Averaging for Improving Random Indexing Embeddings
• Abstract summary: Sparse word embedding pipelines can avoid dense co-occurrence matrix materialization, dense factorization, and gradient training while still relying on sparse global corpus statistics.<n>This paper studies Random Indexing (RI) vectors refined by weighted averaging on a sparse Positive Pointwise Mutual Information (PPMI) graph.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Sparse word embedding pipelines can avoid dense co-occurrence matrix materialization, dense factorization, and gradient training while still relying on sparse global corpus statistics. This paper studies Random Indexing (RI) vectors refined by weighted averaging on a sparse Positive Pointwise Mutual Information (PPMI) graph. On a fairytales corpus, the covered semantic analogy set consists of 272 Google family- category questions. On this family subset, PPMI top-K graph averaging repairs a weak RI initialization, improving accuracy from 19.4+-0.7% to 30.7+-2.9% across five seeds. Under the single tested runs, the same neighborhood averaging reduces family- subset analogy accuracy for PPMI+SVD (singular value decom- position), Binary+SVD, CBOW, and Skip-gram. Thus the method is not competitive with neural baselines on text8 and gives near- zero strict similarity correlation on SimLex-999. While Bloom filter sketches underperform RI in the tested configuration, we find that PPMI graph averaging with top-K pruning is a useful non-gradient repair for weak RI embeddings. On the fairytales dataset, PPMI top-K=50 graph averaging improves RI with accuracy going from 19.4+-0.7% to 30.7+-2.9%, and performing best with a seed42 of 34.6%.

Related papers

• Scaling Laws for Classical Machine Learning on Tabular Data: A Benchmark Study [0.0]
  127 graduate students each ran a fixed protocol on 3 assigned datasets.<n>For 5 of 6 families, a single family-level exponent predicts each family's cross-dataset curves nearly as well as per-dataset exponents.<n>We release the aggregated curves, per-cell fits, and a practical data-requirement table for N* to reach target error 0.15.
  arXiv  Detail & Related papers  (2026-07-23T23:45:09Z)
• LLM Features Can Hurt GNNs: Concatenation Interference on Homophilous Graph Benchmarks [10.283803346813047]
  Adding LLM-generated node features to graph neural networks (GNNs) is widely reported to improve accuracy on standard benchmarks.<n>We document a contrasting observation: when LLM features are introduced through pure input concatenation, they can systematically degrade accuracy on homophilous benchmarks where end-to-end pipelines succeed.
  arXiv  Detail & Related papers  (2026-06-16T06:30:30Z)
• Selecting Samples on Graphs: A Unified Dataset Pruning Framework for Lossless Training Acceleration [71.77541419885925]
  We present a unified graph-based DP framework.<n>By modeling the dataset as a weighted graph, DP can be cast as a Maximum Weight Clique Problem (MWCP)<n>Our method outperforms existing DP methods while substantially reducing training cost, reducing training time by over 40% without sacrificing accuracy on ImageNet-1k with ResNet-50.
  arXiv  Detail & Related papers  (2026-06-11T05:13:32Z)
• vstash: Local-First Hybrid Retrieval with Adaptive Fusion for LLM Agents [0.0]
  We present **vstash**, a local-first document memory system that combines vector similarity search with fulltext keyword matching.<n>All data resides in a single file using sqlite-vec for approximate nearest neighbor search and FTS5 for keyword matching.
  arXiv  Detail & Related papers  (2026-04-16T19:22:58Z)
• Edge-aware baselines for ogbn-proteins in PyTorch Geometric: species-wise normalization, post-hoc calibration, and cost-accuracy trade-offs [45.88028371034407]
  We present reproducible, edge-aware baselines for ogbn-proteins in PyTorch Geometric.<n>Our strongest baseline is GraphSAGE with sum-based edge-to-node features.<n>Post-hoc per-label temperature scaling plus per-label thresholds substantially improves micro-F1 and expected calibration error (ECE) with negligible AUC change.
  arXiv  Detail & Related papers  (2025-11-17T11:09:46Z)
• Identity-Link IRT for Label-Free LLM Evaluation: Preserving Additivity in TVD-MI Scores [3.959606869996232]
  We show that averaging TVD-MI's binary trials yields centered-probability scores with additive structure suitable for item-response theory (IRT) without nonlinear link functions.<n>We derive this clipped-linear evaluations from Gini entropy, yielding a box-constrained least-squares formulation that handles boundary saturation.
  arXiv  Detail & Related papers  (2025-10-16T17:59:25Z)
• Revisiting Rotation Averaging: Uncertainties and Robust Losses [51.64986160468128]
  We argue that the main problem of current methods is the minimized cost function that is only weakly connected with the input data via the estimated epipolar. We propose to better model the underlying noise distributions by directly propagating the uncertainty from the point correspondences into the rotation averaging.
  arXiv  Detail & Related papers  (2023-03-09T11:51:20Z)
• On Learning Mixture of Linear Regressions in the Non-Realizable Setting [44.307245411703704]
  We show that mixture of linear regressions (MLR) can be used for prediction where instead of predicting a label, the model predicts a list of values. In this paper we show that a version of the popular minimization (AM) algorithm finds the best fit lines in a dataset even when a realizable model is not assumed.
  arXiv  Detail & Related papers  (2022-05-26T05:34:57Z)
• Predict then Interpolate: A Simple Algorithm to Learn Stable Classifiers [59.06169363181417]
  Predict then Interpolate (PI) is an algorithm for learning correlations that are stable across environments. We prove that by interpolating the distributions of the correct predictions and the wrong predictions, we can uncover an oracle distribution where the unstable correlation vanishes.
  arXiv  Detail & Related papers  (2021-05-26T15:37:48Z)
• Comprehensive Graph-conditional Similarity Preserving Network for Unsupervised Cross-modal Hashing [97.44152794234405]
  Unsupervised cross-modal hashing (UCMH) has become a hot topic recently. In this paper, we devise a deep graph-neighbor coherence preserving network (DGCPN) DGCPN regulates comprehensive similarity preserving losses by exploiting three types of data similarities.
  arXiv  Detail & Related papers  (2020-12-25T07:40:59Z)
• Cluster-Based Information Retrieval by using (K-means)- Hierarchical Parallel Genetic Algorithms Approach [0.0]
  Cluster-based information retrieval is fast in processing large datasets of document. To improve the quality of retrieved documents, increase the efficiency of IR and reduce irrelevant documents from user search. Three common datasets (NLP, CISI, and CACM) are used to compute the recall, precision, and F-measure averages.
  arXiv  Detail & Related papers  (2020-08-01T02:05:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.