Dynamic transformation of prior knowledge into Bayesian models for data streams

• URL: http://arxiv.org/abs/2003.06123v4
• Date: Sun, 26 Dec 2021 06:58:29 GMT
• Title: Dynamic transformation of prior knowledge into Bayesian models for data streams
• Authors: Tran Xuan Bach, Nguyen Duc Anh, Ngo Van Linh and Khoat Than
• Abstract summary: We consider how to effectively use prior knowledge when learning a Bayesian model from streaming environments where data come infinitely and sequentially. We propose a novel framework that enables to incorporate the prior knowledge of different forms into a base Bayesian model for data streams.
• Score: 2.294014185517203
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We consider how to effectively use prior knowledge when learning a Bayesian model from streaming environments where the data come infinitely and sequentially. This problem is highly important in the era of data explosion and rich sources of precious external knowledge such as pre-trained models, ontologies, Wikipedia, etc. We show that some existing approaches can forget any knowledge very fast. We then propose a novel framework that enables to incorporate the prior knowledge of different forms into a base Bayesian model for data streams. Our framework subsumes some existing popular models for time-series/dynamic data. Extensive experiments show that our framework outperforms existing methods with a large margin. In particular, our framework can help Bayesian models generalize well on extremely short text while other methods overfit. The implementation of our framework is available at https://github.com/bachtranxuan/TPS.git.

Related papers

• Open-source framework for detecting bias and overfitting for large pathology images [0.0]
  Even foundational models that are trained on datasets with billions of data samples may develop shortcuts that lead to overfitting and bias. We propose a generalized, model-agnostic framework to debug deep learning models. Our framework is available as an open-source tool available on GitHub.
  arXiv  Detail & Related papers  (2025-03-03T18:52:53Z)
• TimeRAF: Retrieval-Augmented Foundation model for Zero-shot Time Series Forecasting [59.702504386429126]
  TimeRAF is a Retrieval-Augmented Forecasting model that enhance zero-shot time series forecasting through retrieval-augmented techniques. TimeRAF employs an end-to-end learnable retriever to extract valuable information from the knowledge base.
  arXiv  Detail & Related papers  (2024-12-30T09:06:47Z)
• A Practitioner's Guide to Continual Multimodal Pretraining [83.63894495064855]
  Multimodal foundation models serve numerous applications at the intersection of vision and language. To keep models updated, research into continual pretraining mainly explores scenarios with either infrequent, indiscriminate updates on large-scale new data, or frequent, sample-level updates. We introduce FoMo-in-Flux, a continual multimodal pretraining benchmark with realistic compute constraints and practical deployment requirements.
  arXiv  Detail & Related papers  (2024-08-26T17:59:01Z)
• Toward a Foundation Model for Time Series Data [34.1973242428317]
  A foundation model is a machine learning model trained on a large and diverse set of data. We develop an effective time series foundation model by leveraging unlabeled samples from multiple domains.
  arXiv  Detail & Related papers  (2023-10-05T21:44:50Z)
• Catastrophic Forgetting in the Context of Model Updates [0.360953887026184]
  Deep neural networks can cost many thousands of dollars to train. When new data comes in the pipeline, you can train a new model from scratch on all existing data. The former is costly and slow. The latter is cheap and fast, but catastrophic forgetting generally causes the new model to 'forget' how to classify older data well.
  arXiv  Detail & Related papers  (2023-06-16T21:21:41Z)
• Universal Domain Adaptation from Foundation Models: A Baseline Study [58.51162198585434]
  We make empirical studies of state-of-the-art UniDA methods using foundation models. We introduce textitCLIP distillation, a parameter-free method specifically designed to distill target knowledge from CLIP models. Although simple, our method outperforms previous approaches in most benchmark tasks.
  arXiv  Detail & Related papers  (2023-05-18T16:28:29Z)
• Dataless Knowledge Fusion by Merging Weights of Language Models [51.8162883997512]
  Fine-tuning pre-trained language models has become the prevalent paradigm for building downstream NLP models. This creates a barrier to fusing knowledge across individual models to yield a better single model. We propose a dataless knowledge fusion method that merges models in their parameter space.
  arXiv  Detail & Related papers  (2022-12-19T20:46:43Z)
• dpart: Differentially Private Autoregressive Tabular, a General Framework for Synthetic Data Generation [8.115937653695884]
  dpart is an open source Python library for differentially private synthetic data generation. The library has been created with a view to serve as a quick and accessible baseline. Specific instances of dpart include Independent, an optimized version of PrivBayes, and a newly proposed model, dp-synthpop.
  arXiv  Detail & Related papers  (2022-07-12T19:55:21Z)
• Visualising Deep Network's Time-Series Representations [93.73198973454944]
  Despite the popularisation of machine learning models, more often than not they still operate as black boxes with no insight into what is happening inside the model. In this paper, a method that addresses that issue is proposed, with a focus on visualising multi-dimensional time-series data. Experiments on a high-frequency stock market dataset show that the method provides fast and discernible visualisations.
  arXiv  Detail & Related papers  (2021-03-12T09:53:34Z)
• KGPT: Knowledge-Grounded Pre-Training for Data-to-Text Generation [100.79870384880333]
  We propose a knowledge-grounded pre-training (KGPT) to generate knowledge-enriched text. We adopt three settings, namely fully-supervised, zero-shot, few-shot to evaluate its effectiveness. Under zero-shot setting, our model achieves over 30 ROUGE-L on WebNLG while all other baselines fail.
  arXiv  Detail & Related papers  (2020-10-05T19:59:05Z)
• GRAFFL: Gradient-free Federated Learning of a Bayesian Generative Model [8.87104231451079]
  This paper presents the first gradient-free federated learning framework called GRAFFL. It uses implicit information derived from each participating institution to learn posterior distributions of parameters. We propose the GRAFFL-based Bayesian mixture model to serve as a proof-of-concept of the framework.
  arXiv  Detail & Related papers  (2020-08-29T07:19:44Z)
• $n$-Reference Transfer Learning for Saliency Prediction [73.17061116358036]
  We propose a few-shot transfer learning paradigm for saliency prediction. The proposed framework is gradient-based and model-agnostic. The results show that the proposed framework achieves a significant performance improvement.
  arXiv  Detail & Related papers  (2020-07-09T23:20:44Z)
• A Generic and Model-Agnostic Exemplar Synthetization Framework for Explainable AI [29.243901669124515]
  We focus on explainable AI and propose a novel generic and model-agnostic framework for synthesizing input exemplars. We use a generative model, which acts as a prior for generating data, and traverse its latent space using a novel evolutionary strategy. Our framework is model-agnostic, in the sense that the machine learning model that we aim to explain is a black-box.
  arXiv  Detail & Related papers  (2020-06-06T15:46:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.