Beat-Aligned Spectrogram-to-Sequence Generation of Rhythm-Game Charts

• URL: http://arxiv.org/abs/2311.13687v1
• Date: Wed, 22 Nov 2023 20:47:52 GMT
• Title: Beat-Aligned Spectrogram-to-Sequence Generation of Rhythm-Game Charts
• Authors: Jayeon Yi and Sungho Lee and Kyogu Lee
• Abstract summary: We formulate chart generation as a sequence generation task and train a Transformer using a large dataset. We also introduce tempo-informed preprocessing and training procedures, some of which are suggested to be integral for a successful training.
• Score: 18.938897917126408
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In the heart of "rhythm games" - games where players must perform actions in sync with a piece of music - are "charts", the directives to be given to players. We newly formulate chart generation as a sequence generation task and train a Transformer using a large dataset. We also introduce tempo-informed preprocessing and training procedures, some of which are suggested to be integral for a successful training. Our model is found to outperform the baselines on a large dataset, and is also found to benefit from pretraining and finetuning.

Related papers

• ChartAssisstant: A Universal Chart Multimodal Language Model via Chart-to-Table Pre-training and Multitask Instruction Tuning [54.89249749894061]
  ChartAssistant is a vision-language model for universal chart comprehension and reasoning. It undergoes a two-stage training process, starting with pre-training on chart-to-table parsing to align chart and text. Experimental results demonstrate significant performance gains over the state-of-the-art UniChart and Chartllama method.
  arXiv  Detail & Related papers  (2024-01-04T17:51:48Z)
• Efficient Grammatical Error Correction Via Multi-Task Training and Optimized Training Schedule [55.08778142798106]
  We propose auxiliary tasks that exploit the alignment between the original and corrected sentences. We formulate each task as a sequence-to-sequence problem and perform multi-task training. We find that the order of datasets used for training and even individual instances within a dataset may have important effects on the final performance.
  arXiv  Detail & Related papers  (2023-11-20T14:50:12Z)
• ChartReader: A Unified Framework for Chart Derendering and Comprehension without Heuristic Rules [89.75395046894809]
  We present ChartReader, a unified framework that seamlessly integrates chart derendering and comprehension tasks. Our approach includes a transformer-based chart component detection module and an extended pre-trained vision-language model for chart-to-X tasks. Our proposed framework can significantly reduce the manual effort involved in chart analysis, providing a step towards a universal chart understanding model.
  arXiv  Detail & Related papers  (2023-04-05T00:25:27Z)
• SGL-PT: A Strong Graph Learner with Graph Prompt Tuning [36.650472660276]
  We propose a novel framework named SGL-PT which follows the learning strategy Pre-train, Prompt, and Predict''. Specifically, we raise a strong and universal pre-training task coined as SGL that acquires the complementary merits of generative and contrastive self-supervised graph learning. And aiming for graph classification task, we unify pre-training and fine-tuning by designing a novel verbalizer-free prompting function, which reformulates the downstream task in a similar format as pretext task.
  arXiv  Detail & Related papers  (2023-02-24T04:31:18Z)
• Event Detection in Football using Graph Convolutional Networks [0.0]
  We show how to model the players and the ball in each frame of the video sequence as a graph. We present the results for graph convolutional layers and pooling methods that can be used to model the temporal context present around each action.
  arXiv  Detail & Related papers  (2023-01-24T14:52:54Z)
• Neural Graph Matching for Pre-training Graph Neural Networks [72.32801428070749]
  Graph neural networks (GNNs) have been shown powerful capacity at modeling structural data. We present a novel Graph Matching based GNN Pre-Training framework, called GMPT. The proposed method can be applied to fully self-supervised pre-training and coarse-grained supervised pre-training.
  arXiv  Detail & Related papers  (2022-03-03T09:53:53Z)
• Gen\'eLive! Generating Rhythm Actions in Love Live! [1.3912598476882783]
  A rhythm action game is a music-based video game in which the player is challenged to issue commands at the right timings during a music session. Before this work, the company generated the charts manually, which resulted in a costly business operation. This paper presents how KLab applied a deep generative model for synthesizing charts, and shows how it has improved the chart production process.
  arXiv  Detail & Related papers  (2022-02-25T17:03:36Z)
• TaikoNation: Patterning-focused Chart Generation for Rhythm Action Games [1.590611306750623]
  Patterning is a key identifier of high quality rhythm game content, seen as a necessary component in human rankings. We establish a new approach for chart generation that produces charts with more congruent, human-like patterning than seen in prior work.
  arXiv  Detail & Related papers  (2021-07-26T22:55:57Z)
• How Well Self-Supervised Pre-Training Performs with Streaming Data? [73.5362286533602]
  In real-world scenarios where data are collected in a streaming fashion, the joint training scheme is usually storage-heavy and time-consuming. It is unclear how well sequential self-supervised pre-training performs with streaming data. We find sequential self-supervised learning exhibits almost the same performance as the joint training when the distribution shifts within streaming data are mild.
  arXiv  Detail & Related papers  (2021-04-25T06:56:48Z)
• Improving Semantic Segmentation via Self-Training [75.07114899941095]
  We show that we can obtain state-of-the-art results using a semi-supervised approach, specifically a self-training paradigm. We first train a teacher model on labeled data, and then generate pseudo labels on a large set of unlabeled data. Our robust training framework can digest human-annotated and pseudo labels jointly and achieve top performances on Cityscapes, CamVid and KITTI datasets.
  arXiv  Detail & Related papers  (2020-04-30T17:09:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.