Knowledge Infused Policy Gradients with Upper Confidence Bound for Relational Bandits

• URL: http://arxiv.org/abs/2106.13895v1
• Date: Fri, 25 Jun 2021 21:54:08 GMT
• Title: Knowledge Infused Policy Gradients with Upper Confidence Bound for Relational Bandits
• Authors: Kaushik Roy and Qi Zhang and Manas Gaur and Amit Sheth
• Abstract summary: Contextual Bandits find important use cases in various real-life scenarios such as online advertising, recommendation systems, healthcare, etc. Most of the algorithms use flat feature vectors to represent context whereas, in the real world, there is a varying number of objects and relations among them to model in the context. Adding richer relational context representations also introduces a much larger context space making exploration-exploitation harder.
• Score: 14.316482550910587
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Contextual Bandits find important use cases in various real-life scenarios such as online advertising, recommendation systems, healthcare, etc. However, most of the algorithms use flat feature vectors to represent context whereas, in the real world, there is a varying number of objects and relations among them to model in the context. For example, in a music recommendation system, the user context contains what music they listen to, which artists create this music, the artist albums, etc. Adding richer relational context representations also introduces a much larger context space making exploration-exploitation harder. To improve the efficiency of exploration-exploitation knowledge about the context can be infused to guide the exploration-exploitation strategy. Relational context representations allow a natural way for humans to specify knowledge owing to their descriptive nature. We propose an adaptation of Knowledge Infused Policy Gradients to the Contextual Bandit setting and a novel Knowledge Infused Policy Gradients Upper Confidence Bound algorithm and perform an experimental analysis of a simulated music recommendation dataset and various real-life datasets where expert knowledge can drastically reduce the total regret and where it cannot.

Related papers

• Augmented Commonsense Knowledge for Remote Object Grounding [67.30864498454805]
  We propose an augmented commonsense knowledge model (ACK) to leverage commonsense information as atemporal knowledge graph for improving agent navigation. ACK consists of knowledge graph-aware cross-modal and concept aggregation modules to enhance visual representation and visual-textual data alignment. We add a new pipeline for the commonsense-based decision-making process which leads to more accurate local action prediction.
  arXiv  Detail & Related papers  (2024-06-03T12:12:33Z)
• VLLMs Provide Better Context for Emotion Understanding Through Common Sense Reasoning [66.23296689828152]
  We leverage the capabilities of Vision-and-Large-Language Models to enhance in-context emotion classification. In the first stage, we propose prompting VLLMs to generate descriptions in natural language of the subject's apparent emotion. In the second stage, the descriptions are used as contextual information and, along with the image input, are used to train a transformer-based architecture.
  arXiv  Detail & Related papers  (2024-04-10T15:09:15Z)
• Knowledge Graphs and Pre-trained Language Models enhanced Representation Learning for Conversational Recommender Systems [58.561904356651276]
  We introduce the Knowledge-Enhanced Entity Representation Learning (KERL) framework to improve the semantic understanding of entities for Conversational recommender systems. KERL uses a knowledge graph and a pre-trained language model to improve the semantic understanding of entities. KERL achieves state-of-the-art results in both recommendation and response generation tasks.
  arXiv  Detail & Related papers  (2023-12-18T06:41:23Z)
• Crafting In-context Examples according to LMs' Parametric Knowledge [42.41668563471588]
  In-context examples trigger a language model (LM) to surface information stored in its parametric knowledge. We identify 'known' examples, where models can correctly answer from their parametric knowledge, and 'unknown' ones. Our experiments show that prompting with 'unknown' examples decreases the performance, potentially as it encourages hallucination.
  arXiv  Detail & Related papers  (2023-11-16T05:30:07Z)
• Combo of Thinking and Observing for Outside-Knowledge VQA [13.838435454270014]
  Outside-knowledge visual question answering is a challenging task that requires both the acquisition and the use of open-ended real-world knowledge. In this paper, we are inspired to constrain the cross-modality space into the same space of natural-language space. We propose a novel framework consisting of a multimodal encoder, a textual encoder and an answer decoder.
  arXiv  Detail & Related papers  (2023-05-10T18:32:32Z)
• Context-faithful Prompting for Large Language Models [51.194410884263135]
  Large language models (LLMs) encode parametric knowledge about world facts. Their reliance on parametric knowledge may cause them to overlook contextual cues, leading to incorrect predictions in context-sensitive NLP tasks. We assess and enhance LLMs' contextual faithfulness in two aspects: knowledge conflict and prediction with abstention.
  arXiv  Detail & Related papers  (2023-03-20T17:54:58Z)
• Multi-Sentence Knowledge Selection in Open-Domain Dialogue [11.936691632841388]
  We evaluate the existing state of open-domain conversation knowledge selection. We create an augmented dataset based on the Wizard of Wikipedia (WOW) corpus. WOW++ averages 8 relevant knowledge sentences per dialogue context.
  arXiv  Detail & Related papers  (2022-03-01T22:07:05Z)
• A General Framework for Learning Prosodic-Enhanced Representation of Rap Lyrics [21.944835086749375]
  Learning and analyzing rap lyrics is a significant basis for many web applications. We propose a hierarchical attention variational autoencoder framework (HAVAE) A feature aggregation strategy is proposed to appropriately integrate various features and generate prosodic-enhanced representation.
  arXiv  Detail & Related papers  (2021-03-23T15:13:21Z)
• Wisdom of the Contexts: Active Ensemble Learning for Contextual Anomaly Detection [7.87320844079302]
  In contextual anomaly detection (CAD), an object is only considered anomalous within a specific context. We propose a novel approach, called WisCon, that automatically creates contexts from the feature set. Our method constructs an ensemble of multiple contexts, with varying importance scores, based on the assumption that not all useful contexts are equally so.
  arXiv  Detail & Related papers  (2021-01-27T17:34:13Z)
• Learning Contextual Causality from Time-consecutive Images [84.26437953699444]
  Causality knowledge is crucial for many artificial intelligence systems. In this paper, we investigate the possibility of learning contextual causality from the visual signal. We first propose a high-quality dataset Vis-Causal and then conduct experiments to demonstrate that it is possible to automatically discover meaningful causal knowledge from the videos.
  arXiv  Detail & Related papers  (2020-12-13T20:24:48Z)
• CokeBERT: Contextual Knowledge Selection and Embedding towards Enhanced Pre-Trained Language Models [103.18329049830152]
  We propose a novel framework named Coke to dynamically select contextual knowledge and embed knowledge context according to textual context. Our experimental results show that Coke outperforms various baselines on typical knowledge-driven NLP tasks. Coke can describe the semantics of text-related knowledge in a more interpretable form than the conventional PLMs.
  arXiv  Detail & Related papers  (2020-09-29T12:29:04Z)
• Improving Machine Reading Comprehension with Contextualized Commonsense Knowledge [62.46091695615262]
  We aim to extract commonsense knowledge to improve machine reading comprehension. We propose to represent relations implicitly by situating structured knowledge in a context. We employ a teacher-student paradigm to inject multiple types of contextualized knowledge into a student machine reader.
  arXiv  Detail & Related papers  (2020-09-12T17:20:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.