Leveraging Pre-Trained Language Models to Streamline Natural Language Interaction for Self-Tracking

• URL: http://arxiv.org/abs/2205.15503v2
• Date: Wed, 1 Jun 2022 01:52:31 GMT
• Title: Leveraging Pre-Trained Language Models to Streamline Natural Language Interaction for Self-Tracking
• Authors: Young-Ho Kim, Sungdong Kim, Minsuk Chang, Sang-Woo Lee
• Abstract summary: We propose a novel NLP task for self-tracking that extracts close- and open-ended information from a retrospective activity log. The framework augments the prompt using synthetic samples to transform the task into 10-shot learning, to address a cold-start problem in bootstrapping a new tracking topic.
• Score: 25.28975864365579
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Current natural language interaction for self-tracking tools largely depends on bespoke implementation optimized for a specific tracking theme and data format, which is neither generalizable nor scalable to a tremendous design space of self-tracking. However, training machine learning models in the context of self-tracking is challenging due to the wide variety of tracking topics and data formats. In this paper, we propose a novel NLP task for self-tracking that extracts close- and open-ended information from a retrospective activity log described as a plain text, and a domain-agnostic, GPT-3-based NLU framework that performs this task. The framework augments the prompt using synthetic samples to transform the task into 10-shot learning, to address a cold-start problem in bootstrapping a new tracking topic. Our preliminary evaluation suggests that our approach significantly outperforms the baseline QA models. Going further, we discuss future application domains toward which the NLP and HCI researchers can collaborate.

Related papers

• Context is Key: A Benchmark for Forecasting with Essential Textual Information [87.3175915185287]
  "Context is Key" (CiK) is a time series forecasting benchmark that pairs numerical data with diverse types of carefully crafted textual context. We evaluate a range of approaches, including statistical models, time series foundation models, and LLM-based forecasters. Our experiments highlight the importance of incorporating contextual information, demonstrate surprising performance when using LLM-based forecasting models, and also reveal some of their critical shortcomings.
  arXiv  Detail & Related papers  (2024-10-24T17:56:08Z)
• CELA: Cost-Efficient Language Model Alignment for CTR Prediction [71.85120354973073]
  Click-Through Rate (CTR) prediction holds a paramount position in recommender systems. Recent efforts have sought to mitigate these challenges by integrating Pre-trained Language Models (PLMs) We propose textbfCost-textbfEfficient textbfLanguage Model textbfAlignment (textbfCELA) for CTR prediction.
  arXiv  Detail & Related papers  (2024-05-17T07:43:25Z)
• Towards Unified Token Learning for Vision-Language Tracking [65.96561538356315]
  We present a vision-language (VL) tracking pipeline, termed textbfMMTrack, which casts VL tracking as a token generation task. Our proposed framework serializes language description and bounding box into a sequence of discrete tokens. In this new design paradigm, all token queries are required to perceive the desired target and directly predict spatial coordinates of the target.
  arXiv  Detail & Related papers  (2023-08-27T13:17:34Z)
• ChatGPT for Zero-shot Dialogue State Tracking: A Solution or an Opportunity? [2.3555053092246125]
  We present preliminary experimental results on the ChatGPT research preview, showing that ChatGPT achieves state-of-the-art performance in zero-shot DST. We theorize that the in-context learning capabilities of such models will likely become powerful tools to support the development of dedicated and dynamic dialogue state trackers.
  arXiv  Detail & Related papers  (2023-06-02T09:15:01Z)
• Annotated Dataset Creation through General Purpose Language Models for non-English Medical NLP [0.5482532589225552]
  In our work we suggest to leverage pretrained language models for training data acquisition. We create a custom dataset which we use to train a medical NER model for German texts, GPTNERMED.
  arXiv  Detail & Related papers  (2022-08-30T18:42:55Z)
• Leveraging Natural Supervision for Language Representation Learning and Generation [8.083109555490475]
  We describe three lines of work that seek to improve the training and evaluation of neural models using naturally-occurring supervision. We first investigate self-supervised training losses to help enhance the performance of pretrained language models for various NLP tasks. We propose a framework that uses paraphrase pairs to disentangle semantics and syntax in sentence representations.
  arXiv  Detail & Related papers  (2022-07-21T17:26:03Z)
• Leveraging pre-trained language models for conversational information seeking from text [2.8425118603312]
  In this paper we investigate the usage of in-context learning and pre-trained language representation models to address the problem of information extraction from process description documents. The results highlight the potential of the approach and the usefulness of the in-context learning customizations.
  arXiv  Detail & Related papers  (2022-03-31T09:00:46Z)
• Efficient Nearest Neighbor Language Models [114.40866461741795]
  Non-parametric neural language models (NLMs) learn predictive distributions of text utilizing an external datastore. We show how to achieve up to a 6x speed-up in inference speed while retaining comparable performance.
  arXiv  Detail & Related papers  (2021-09-09T12:32:28Z)
• Prompt-Learning for Fine-Grained Entity Typing [40.983849729537795]
  We investigate the application of prompt-learning on fine-grained entity typing in fully supervised, few-shot and zero-shot scenarios. We propose a self-supervised strategy that carries out distribution-level optimization in prompt-learning to automatically summarize the information of entity types.
  arXiv  Detail & Related papers  (2021-08-24T09:39:35Z)
• Unsupervised Paraphrasing with Pretrained Language Models [85.03373221588707]
  We propose a training pipeline that enables pre-trained language models to generate high-quality paraphrases in an unsupervised setting. Our recipe consists of task-adaptation, self-supervision, and a novel decoding algorithm named Dynamic Blocking. We show with automatic and human evaluations that our approach achieves state-of-the-art performance on both the Quora Question Pair and the ParaNMT datasets.
  arXiv  Detail & Related papers  (2020-10-24T11:55:28Z)
• Few-shot Natural Language Generation for Task-Oriented Dialog [113.07438787659859]
  We present FewShotWoz, the first NLG benchmark to simulate the few-shot learning setting in task-oriented dialog systems. We develop the SC-GPT model, which is pre-trained on a large set of annotated NLG corpus to acquire the controllable generation ability. Experiments on FewShotWoz and the large Multi-Domain-WOZ datasets show that the proposed SC-GPT significantly outperforms existing methods.
  arXiv  Detail & Related papers  (2020-02-27T18:48:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.