Punctuation Prediction for Polish Texts using Transformers

• URL: http://arxiv.org/abs/2410.04621v1
• Date: Sun, 6 Oct 2024 20:51:02 GMT
• Title: Punctuation Prediction for Polish Texts using Transformers
• Authors: Jakub Pokrywka,
• Abstract summary: This paper describes a solution for Poleval 2022 Task 1: Punctuation Prediction for Polish Texts, which scores 71.44 Weighted F1. The method utilizes a single HerBERT model finetuned to the competition data and an external dataset.
• Score: 0.7252027234425334
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Speech recognition systems typically output text lacking punctuation. However, punctuation is crucial for written text comprehension. To tackle this problem, Punctuation Prediction models are developed. This paper describes a solution for Poleval 2022 Task 1: Punctuation Prediction for Polish Texts, which scores 71.44 Weighted F1. The method utilizes a single HerBERT model finetuned to the competition data and an external dataset.

Related papers

• Misspellings in Natural Language Processing: A survey [52.419589623702336]
  misspellings have become ubiquitous in digital communication. We reconstruct a history of misspellings as a scientific problem. We discuss the latest advancements to address the challenge of misspellings in NLP.
  arXiv  Detail & Related papers  (2025-01-28T10:26:04Z)
• LibriSpeech-PC: Benchmark for Evaluation of Punctuation and Capitalization Capabilities of end-to-end ASR Models [58.790604613878216]
  We introduce a LibriSpeech-PC benchmark designed to assess the punctuation and capitalization prediction capabilities of end-to-end ASR models. The benchmark includes a LibriSpeech-PC dataset with restored punctuation and capitalization, a novel evaluation metric called Punctuation Error Rate (PER) that focuses on punctuation marks, and initial baseline models.
  arXiv  Detail & Related papers  (2023-10-04T16:23:37Z)
• Improved Training for End-to-End Streaming Automatic Speech Recognition Model with Punctuation [0.08602553195689511]
  We propose a method for predicting punctuated text from input speech using a chunk-based Transformer encoder trained with Connectionist Temporal Classification (CTC) loss. By combining CTC losses on the chunks and utterances, we achieved both the improved F1 score of punctuation prediction and Word Error Rate (WER)
  arXiv  Detail & Related papers  (2023-06-02T06:46:14Z)
• token2vec: A Joint Self-Supervised Pre-training Framework Using Unpaired Speech and Text [65.04385919645395]
  token2vec is a novel joint pre-training framework for unpaired speech and text based on discrete representations of speech. Experiments show that token2vec is significantly superior to various speech-only pre-training baselines, with up to 17.7% relative WER reduction.
  arXiv  Detail & Related papers  (2022-10-30T06:38:19Z)
• End-to-end Speech-to-Punctuated-Text Recognition [23.44236710364419]
  punctuation marks are important for the readability of the speech recognition results. Conventional automatic speech recognition systems do not produce punctuation marks. We propose an end-to-end model that takes speech as input and outputs punctuated texts.
  arXiv  Detail & Related papers  (2022-07-07T08:58:01Z)
• M-Adapter: Modality Adaptation for End-to-End Speech-to-Text Translation [66.92823764664206]
  We propose M-Adapter, a novel Transformer-based module, to adapt speech representations to text. While shrinking the speech sequence, M-Adapter produces features desired for speech-to-text translation. Our experimental results show that our model outperforms a strong baseline by up to 1 BLEU.
  arXiv  Detail & Related papers  (2022-07-03T04:26:53Z)
• RuArg-2022: Argument Mining Evaluation [69.87149207721035]
  This paper is a report of the organizers on the first competition of argumentation analysis systems dealing with Russian language texts. A corpus containing 9,550 sentences (comments on social media posts) on three topics related to the COVID-19 pandemic was prepared. The system that won the first place in both tasks used the NLI (Natural Language Inference) variant of the BERT architecture.
  arXiv  Detail & Related papers  (2022-06-18T17:13:37Z)
• Punctuation restoration in Swedish through fine-tuned KB-BERT [0.0]
  The method is based on KB-BERT, a publicly available, neural network language model pre-trained on a Swedish corpus. With a lower-case and unpunctuated Swedish text as input, the model is supposed to return a grammatically correct punctuated copy of the text as output.
  arXiv  Detail & Related papers  (2022-02-14T14:39:40Z)
• Improving Punctuation Restoration for Speech Transcripts via External Data [1.4335946386597276]
  We tackle the punctuation restoration problem specifically for the noisy text. We introduce a data sampling technique based on an n-gram language model to sample more training data. The proposed approach outperforms the baseline with an improvement of 1:12% F1 score.
  arXiv  Detail & Related papers  (2021-10-01T17:40:55Z)
• Zero-Shot Text-to-Speech for Text-Based Insertion in Audio Narration [62.75234183218897]
  We propose a one-stage context-aware framework to generate natural and coherent target speech without any training data of the speaker. We generate the mel-spectrogram of the edited speech with a transformer-based decoder. It outperforms a recent zero-shot TTS engine by a large margin.
  arXiv  Detail & Related papers  (2021-09-12T04:17:53Z)
• Token-Level Supervised Contrastive Learning for Punctuation Restoration [7.9713449581347104]
  Punctuation is critical in understanding natural language text. Most automatic speech recognition systems do not generate punctuation. Recent work in punctuation restoration heavily utilizes pre-trained language models.
  arXiv  Detail & Related papers  (2021-07-19T18:24:33Z)
• Adversarial Transfer Learning for Punctuation Restoration [58.2201356693101]
  Adversarial multi-task learning is introduced to learn task invariant knowledge for punctuation prediction. Experiments are conducted on IWSLT2011 datasets.
  arXiv  Detail & Related papers  (2020-04-01T06:19:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.