'Beach' to 'Bitch': Inadvertent Unsafe Transcription of Kids' Content on YouTube

• URL: http://arxiv.org/abs/2203.04837v1
• Date: Thu, 17 Feb 2022 19:19:09 GMT
• Title: 'Beach' to 'Bitch': Inadvertent Unsafe Transcription of Kids' Content on YouTube
• Authors: Krithika Ramesh, Ashiqur R. KhudaBukhsh, Sumeet Kumar
• Abstract summary: Well-known automatic speech recognition (ASR) systems may produce text content highly inappropriate for kids while transcribing YouTube Kids' videos. We release a first-of-its-kind data set of audios for which the existing state-of-the-art ASR systems hallucinate inappropriate content for kids.
• Score: 13.116806430326513
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Over the last few years, YouTube Kids has emerged as one of the highly competitive alternatives to television for children's entertainment. Consequently, YouTube Kids' content should receive an additional level of scrutiny to ensure children's safety. While research on detecting offensive or inappropriate content for kids is gaining momentum, little or no current work exists that investigates to what extent AI applications can (accidentally) introduce content that is inappropriate for kids. In this paper, we present a novel (and troubling) finding that well-known automatic speech recognition (ASR) systems may produce text content highly inappropriate for kids while transcribing YouTube Kids' videos. We dub this phenomenon as \emph{inappropriate content hallucination}. Our analyses suggest that such hallucinations are far from occasional, and the ASR systems often produce them with high confidence. We release a first-of-its-kind data set of audios for which the existing state-of-the-art ASR systems hallucinate inappropriate content for kids. In addition, we demonstrate that some of these errors can be fixed using language models.

Related papers

• Does Object Grounding Really Reduce Hallucination of Large Vision-Language Models? [53.89380284760555]
  Large vision-language models (LVLMs) produce captions that mention concepts that cannot be found in the image. These hallucinations erode the trustworthiness of LVLMs and are arguably among the main obstacles to their ubiquitous adoption. Recent work suggests that addition of grounding objectives -- those that explicitly align image regions or objects to text spans -- reduces the amount of LVLM hallucination.
  arXiv  Detail & Related papers  (2024-06-20T16:56:11Z)
• Enhanced Multimodal Content Moderation of Children's Videos using Audiovisual Fusion [0.6963971634605796]
  We present an efficient adaptation of CLIP that can leverage contextual audio cues for enhanced content moderation. We conduct experiments on a multimodal version of the MOB (Malicious or Benign) dataset in supervised and few-shot settings.
  arXiv  Detail & Related papers  (2024-05-09T22:19:40Z)
• Security Advice for Parents and Children About Content Filtering and Circumvention as Found on YouTube and TikTok [2.743215038883957]
  We examine the advice available to parents and children regarding content filtering and circumvention as found on YouTube and TikTok. Our results show that of these videos, roughly three-quarters are accurate, with the remaining one-fourth containing factually incorrect advice. We find that videos targeting children are both more likely to be incorrect and actionable than videos targeting parents, leaving children at increased risk of taking harmful action.
  arXiv  Detail & Related papers  (2024-02-05T18:12:33Z)
• An Image is Worth a Thousand Toxic Words: A Metamorphic Testing Framework for Content Moderation Software [64.367830425115]
  Social media platforms are being increasingly misused to spread toxic content, including hate speech, malicious advertising, and pornography. Despite tremendous efforts in developing and deploying content moderation methods, malicious users can evade moderation by embedding texts into images. We propose a metamorphic testing framework for content moderation software.
  arXiv  Detail & Related papers  (2023-08-18T20:33:06Z)
• LLM Censorship: A Machine Learning Challenge or a Computer Security Problem? [52.71988102039535]
  We show that semantic censorship can be perceived as an undecidable problem. We argue that the challenges extend beyond semantic censorship, as knowledgeable attackers can reconstruct impermissible outputs.
  arXiv  Detail & Related papers  (2023-07-20T09:25:02Z)
• Malicious or Benign? Towards Effective Content Moderation for Children's Videos [1.0323063834827415]
  This paper introduces our toolkit Malicious or Benign for promoting research on automated content moderation of children's videos. We present 1) a customizable annotation tool for videos, 2) a new dataset with difficult to detect test cases of malicious content, and 3) a benchmark suite of state-of-the-art video classification models.
  arXiv  Detail & Related papers  (2023-05-24T20:33:38Z)
• Seeing What You Said: Talking Face Generation Guided by a Lip Reading Expert [89.07178484337865]
  Talking face generation, also known as speech-to-lip generation, reconstructs facial motions concerning lips given coherent speech input. Previous studies revealed the importance of lip-speech synchronization and visual quality. We propose using a lip-reading expert to improve the intelligibility of the generated lip regions.
  arXiv  Detail & Related papers  (2023-03-29T07:51:07Z)
• Constructing Highly Inductive Contexts for Dialogue Safety through Controllable Reverse Generation [65.48908724440047]
  We propose a method called emphreverse generation to construct adversarial contexts conditioned on a given response. We test three popular pretrained dialogue models (Blender, DialoGPT, and Plato2) and find that BAD+ can largely expose their safety problems.
  arXiv  Detail & Related papers  (2022-12-04T12:23:41Z)
• Did your child get disturbed by an inappropriate advertisement on YouTube? [3.2113789596629503]
  We analyze the advertising patterns of 24.6 K diverse YouTube videos appropriate for young children. We find that 9.9% of the 4.6 K unique advertisements shown on these 24.6 K videos contain inappropriate content for young children.
  arXiv  Detail & Related papers  (2022-11-04T10:28:54Z)
• YouTubers Not madeForKids: Detecting Channels Sharing Inappropriate Videos Targeting Children [3.936965297430477]
  We study YouTube channels found to post suitable or disturbing videos targeting kids in the past. We identify a clear discrepancy between what YouTube assumes and flags as inappropriate content and channel, vs. what is found to be disturbing content and still available on the platform.
  arXiv  Detail & Related papers  (2022-05-27T10:34:15Z)
• Audio-Visual Speech Codecs: Rethinking Audio-Visual Speech Enhancement by Re-Synthesis [67.73554826428762]
  We propose a novel audio-visual speech enhancement framework for high-fidelity telecommunications in AR/VR. Our approach leverages audio-visual speech cues to generate the codes of a neural speech, enabling efficient synthesis of clean, realistic speech from noisy signals.
  arXiv  Detail & Related papers  (2022-03-31T17:57:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.