Explorers at #SMM4H 2023: Enhancing BERT for Health Applications through Knowledge and Model Fusion

• URL: http://arxiv.org/abs/2312.10652v1
• Date: Sun, 17 Dec 2023 08:52:05 GMT
• Title: Explorers at #SMM4H 2023: Enhancing BERT for Health Applications through Knowledge and Model Fusion
• Authors: Xutong Yue, Xilai Wang, Yuxin He, Zhenkun Zhou
• Abstract summary: Social media has become a valuable data resource for studying human health. This paper outlines the methods in our participation in the #SMM4H 2023 Shared Tasks.
• Score: 3.386401892906348
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: An increasing number of individuals are willing to post states and opinions in social media, which has become a valuable data resource for studying human health. Furthermore, social media has been a crucial research point for healthcare now. This paper outlines the methods in our participation in the #SMM4H 2023 Shared Tasks, including data preprocessing, continual pre-training and fine-tuned optimization strategies. Especially for the Named Entity Recognition (NER) task, we utilize the model architecture named W2NER that effectively enhances the model generalization ability. Our method achieved first place in the Task 3. This paper has been peer-reviewed and accepted for presentation at the #SMM4H 2023 Workshop.

Related papers

• Enriched BERT Embeddings for Scholarly Publication Classification [0.13654846342364302]
  The NSLP 2024 FoRC Task I addresses this challenge organized as a competition. The goal is to develop a classifier capable of predicting one of 123 predefined classes from the Open Research Knowledge Graph (ORKG) taxonomy of research fields for a given article.
  arXiv  Detail & Related papers  (2024-05-07T09:05:20Z)
• Data and models for stance and premise detection in COVID-19 tweets: insights from the Social Media Mining for Health (SMM4H) 2022 shared task [7.559611243635055]
  We organize the Social Media Mining for Health (SMM4H) 2022 Shared Task 2. This competition utilized manually annotated posts on three COVID-19-related topics: school closures, stay-at-home orders, and wearing masks. We present newly collected data on vaccination from Twitter to assess the performance of models on a different topic.
  arXiv  Detail & Related papers  (2023-11-14T10:30:49Z)
• PULSAR at MEDIQA-Sum 2023: Large Language Models Augmented by Synthetic Dialogue Convert Patient Dialogues to Medical Records [23.25763256861649]
  This paper describes PULSAR, our system submission at the ImageClef 2023 MediQA-Sum task on summarising patient-doctor dialogues into clinical records. The proposed framework relies on domain-specific pre-training, to produce a specialised language model which is trained on task-specific natural data. We find limited evidence towards the efficacy of domain-specific pre-training and data augmentation, while scaling up the language model yields the best performance gains.
  arXiv  Detail & Related papers  (2023-07-05T03:31:12Z)
• PULSAR: Pre-training with Extracted Healthcare Terms for Summarising Patients' Problems and Data Augmentation with Black-box Large Language Models [25.363775123262307]
  Automatic summarisation of a patient's problems in the form of a problem list can aid stakeholders in understanding a patient's condition, reducing workload and cognitive bias. BioNLP 2023 Shared Task 1A focuses on generating a list of diagnoses and problems from the provider's progress notes during hospitalisation. One component employs large language models (LLMs) for data augmentation; the other is an abstractive summarisation LLM with a novel pre-training objective for generating the patients' problems summarised as a list. Our approach was ranked second among all submissions to the shared task.
  arXiv  Detail & Related papers  (2023-06-05T10:17:50Z)
• TMR: Text-to-Motion Retrieval Using Contrastive 3D Human Motion Synthesis [59.465092047829835]
  We present TMR, a simple yet effective approach for text to 3D human motion retrieval. Our method extends the state-of-the-art text-to-motion synthesis model TEMOS. We show that maintaining the motion generation loss, along with the contrastive training, is crucial to obtain good performance.
  arXiv  Detail & Related papers  (2023-05-02T17:52:41Z)
• Large-scale Multi-Modal Pre-trained Models: A Comprehensive Survey [66.18478838828231]
  Multi-modal pre-trained big models have drawn more and more attention in recent years. This paper introduces the background of multi-modal pre-training by reviewing the conventional deep, pre-training works in natural language process, computer vision, and speech. Then, we introduce the task definition, key challenges, and advantages of multi-modal pre-training models (MM-PTMs), and discuss the MM-PTMs with a focus on data, objectives, network, and knowledge enhanced pre-training.
  arXiv  Detail & Related papers  (2023-02-20T15:34:03Z)
• Multi-task Learning for Personal Health Mention Detection on Social Media [70.23889100356091]
  This research employs a multitask learning framework to leverage available annotated data to improve the performance on the main task. We focus on incorporating emotional information into our target task by using emotion detection as an auxiliary task.
  arXiv  Detail & Related papers  (2022-12-09T23:49:00Z)
• Incorporating Emotions into Health Mention Classification Task on Social Media [70.23889100356091]
  We present a framework for health mention classification that incorporates affective features. We evaluate our approach on 5 HMC-related datasets from different social media platforms. Our results indicate that HMC models infused with emotional knowledge are an effective alternative.
  arXiv  Detail & Related papers  (2022-12-09T18:38:41Z)
• Prompt-based Pre-trained Model for Personality and Interpersonal Reactivity Prediction [19.288384399961867]
  This paper describes the LingJing team's method to the Workshop on Computational Approaches to Subjectivity, Sentiment & Social Media Analysis (WASSA) 2022 shared task on Personality Prediction (PER) and Reactivity Index Prediction (IRI)
  arXiv  Detail & Related papers  (2022-03-23T15:22:34Z)
• An Analysis of a BERT Deep Learning Strategy on a Technology Assisted Review Task [91.3755431537592]
  Document screening is a central task within Evidenced Based Medicine. I propose a DL document classification approach with BERT or PubMedBERT embeddings and a DL similarity search path. I test and evaluate the retrieval effectiveness of my DL strategy on the 2017 and 2018 CLEF eHealth collections.
  arXiv  Detail & Related papers  (2021-04-16T19:45:27Z)
• MET: Multimodal Perception of Engagement for Telehealth [52.54282887530756]
  We present MET, a learning-based algorithm for perceiving a human's level of engagement from videos. We release a new dataset, MEDICA, for mental health patient engagement detection.
  arXiv  Detail & Related papers  (2020-11-17T15:18:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.