Persistence Homology of TEDtalk: Do Sentence Embeddings Have a Topological Shape?

• URL: http://arxiv.org/abs/2103.14131v1
• Date: Thu, 25 Mar 2021 20:52:17 GMT
• Title: Persistence Homology of TEDtalk: Do Sentence Embeddings Have a Topological Shape?
• Authors: Shouman Das, Syed A. Haque, Md. Iftekhar Tanveer
• Abstract summary: We investigate the possibility of applying TDA to improve the classification accuracy of public speaking rating. We calculated emphpersistence image vectors for the sentence embeddings of TEDtalk data and feed this vectors as additional inputs to our machine learning models. From our results, we could not conclude that the topological shapes of the sentence embeddings can help us train a better model for public speaking rating.
• Score: 3.1675545188012078
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: \emph{Topological data analysis} (TDA) has recently emerged as a new technique to extract meaningful discriminitve features from high dimensional data. In this paper, we investigate the possibility of applying TDA to improve the classification accuracy of public speaking rating. We calculated \emph{persistence image vectors} for the sentence embeddings of TEDtalk data and feed this vectors as additional inputs to our machine learning models. We have found a negative result that this topological information does not improve the model accuracy significantly. In some cases, it makes the accuracy slightly worse than the original one. From our results, we could not conclude that the topological shapes of the sentence embeddings can help us train a better model for public speaking rating.

Related papers

• The Right to be Forgotten in Pruning: Unveil Machine Unlearning on Sparse Models [18.728123679646398]
  Machine unlearning aims to efficiently eliminate the memory about deleted data from trained models and address the right to be forgotten.<n>In this paper, we empirically find that the deleted data has an impact on the pruned topology in a sparse model.<n>Motivated by the observation and the right to be forgotten, we define a new terminology un-pruning" to eliminate the impact of deleted data on model pruning.
  arXiv  Detail & Related papers  (2025-07-24T18:13:26Z)
• Pruning Literals for Highly Efficient Explainability at Word Level [13.249876381579158]
  Tsetlin Machine(TM) is promising because of its capability of providing word-level explanation using proposition logic. In this paper, we design a post-hoc pruning of clauses that eliminate the randomly placed literals in the clause. Experiments on the publicly available YELP-HAT dataset demonstrate that the proposed pruned TM's attention map aligns more with the human attention map than the vanilla TM's attention map.
  arXiv  Detail & Related papers  (2024-11-07T09:28:38Z)
• The Persian Rug: solving toy models of superposition using large-scale symmetries [0.0]
  We present a complete mechanistic description of the algorithm learned by a minimal non-linear sparse data autoencoder in the limit of large input dimension. Our work contributes to neural network interpretability by introducing techniques for understanding the structure of autoencoders.
  arXiv  Detail & Related papers  (2024-10-15T22:52:45Z)
• Towards Model-Agnostic Posterior Approximation for Fast and Accurate Variational Autoencoders [22.77397537980102]
  We show that we can compute a deterministic, model-agnostic posterior approximation (MAPA) of the true model's posterior. We present preliminary results on low-dimensional synthetic data that (1) MAPA captures the trend of the true posterior, and (2) our MAPA-based inference performs better density estimation with less computation than baselines.
  arXiv  Detail & Related papers  (2024-03-13T20:16:21Z)
• Towards Fine-Grained Information: Identifying the Type and Location of Translation Errors [80.22825549235556]
  Existing approaches can not synchronously consider error position and type. We build an FG-TED model to predict the textbf addition and textbfomission errors. Experiments show that our model can identify both error type and position concurrently, and gives state-of-the-art results.
  arXiv  Detail & Related papers  (2023-02-17T16:20:33Z)
• Topological Data Analysis for Speech Processing [10.00176964652466]
  We show that a simple linear classifier built on top of such features outperforms a fine-tuned classification head. We also show that topological features are able to reveal functional roles of speech Transformer heads.
  arXiv  Detail & Related papers  (2022-11-30T18:22:37Z)
• Evaluating State-of-the-Art Classification Models Against Bayes Optimality [106.50867011164584]
  We show that we can compute the exact Bayes error of generative models learned using normalizing flows. We use our approach to conduct a thorough investigation of state-of-the-art classification models.
  arXiv  Detail & Related papers  (2021-06-07T06:21:20Z)
• Generative Text Modeling through Short Run Inference [47.73892773331617]
  The present work proposes a short run dynamics for inference. It is variation from the prior distribution of the latent variable and then runs a small number of Langevin dynamics steps guided by its posterior distribution. We show that the models trained with short run dynamics more accurately model the data, compared to strong language model and VAE baselines, and exhibit no sign of posterior collapse.
  arXiv  Detail & Related papers  (2021-05-27T09:14:35Z)
• Learnable Boundary Guided Adversarial Training [66.57846365425598]
  We use the model logits from one clean model to guide learning of another one robust model. We achieve new state-of-the-art robustness on CIFAR-100 without additional real or synthetic data.
  arXiv  Detail & Related papers  (2020-11-23T01:36:05Z)
• Improving Robustness by Augmenting Training Sentences with Predicate-Argument Structures [62.562760228942054]
  Existing approaches to improve robustness against dataset biases mostly focus on changing the training objective. We propose to augment the input sentences in the training data with their corresponding predicate-argument structures. We show that without targeting a specific bias, our sentence augmentation improves the robustness of transformer models against multiple biases.
  arXiv  Detail & Related papers  (2020-10-23T16:22:05Z)
• Evaluating Prediction-Time Batch Normalization for Robustness under Covariate Shift [81.74795324629712]
  We call prediction-time batch normalization, which significantly improves model accuracy and calibration under covariate shift. We show that prediction-time batch normalization provides complementary benefits to existing state-of-the-art approaches for improving robustness. The method has mixed results when used alongside pre-training, and does not seem to perform as well under more natural types of dataset shift.
  arXiv  Detail & Related papers  (2020-06-19T05:08:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.