Can pre-trained language models generate titles for research papers?

• URL: http://arxiv.org/abs/2409.14602v2
• Date: Sun, 13 Oct 2024 18:35:04 GMT
• Title: Can pre-trained language models generate titles for research papers?
• Authors: Tohida Rehman, Debarshi Kumar Sanyal, Samiran Chattopadhyay,
• Abstract summary: In this paper, we fine-tune pre-trained language models to generate titles of papers from their abstracts. We also use GPT-3.5-turbo in a zero-shot setting to generate paper titles. Our observations suggest that AI-generated paper titles are generally accurate and appropriate.
• Score: 3.3489486000815765
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: The title of a research paper communicates in a succinct style the main theme and, sometimes, the findings of the paper. Coming up with the right title is often an arduous task, and therefore, it would be beneficial to authors if title generation can be automated. In this paper, we fine-tune pre-trained language models to generate titles of papers from their abstracts. Additionally, we use GPT-3.5-turbo in a zero-shot setting to generate paper titles. The performance of the models is measured with ROUGE, METEOR, MoverScore, BERTScore and SciBERTScore metrics. We find that fine-tuned PEGASUS-large outperforms the other models, including fine-tuned LLaMA-3-8B and GPT-3.5-turbo, across most metrics. We also demonstrate that ChatGPT can generate creative titles for papers. Our observations suggest that AI-generated paper titles are generally accurate and appropriate.

Related papers

• Good things come in three: Generating SO Post Titles with Pre-Trained Models, Self Improvement and Post Ranking [5.874782446136913]
  Stack Overflow is a prominent Q and A forum, supporting developers in seeking suitable resources on programming-related matters. Having high-quality question titles is an effective means to attract developers' attention. Research has been conducted, predominantly leveraging pre-trained models to generate titles from code snippets and problem descriptions. We present FILLER as a solution to generating Stack Overflow post titles using a fine-tuned language model with self-improvement and post ranking.
  arXiv  Detail & Related papers  (2024-06-21T20:18:34Z)
• Shall We Pretrain Autoregressive Language Models with Retrieval? A Comprehensive Study [115.96080028033904]
  We study a scalable pre-trained retrieval-augmented LM (i.e., RETRO) compared with standard GPT and retrieval-augmented GPT. Our findings highlight the promising direction of pretraining autoregressive LMs with retrieval as future foundation models.
  arXiv  Detail & Related papers  (2023-04-13T18:04:19Z)
• Large Language Models in the Workplace: A Case Study on Prompt Engineering for Job Type Classification [58.720142291102135]
  This case study investigates the task of job classification in a real-world setting. The goal is to determine whether an English-language job posting is appropriate for a graduate or entry-level position.
  arXiv  Detail & Related papers  (2023-03-13T14:09:53Z)
• Large-scale Multi-Modal Pre-trained Models: A Comprehensive Survey [66.18478838828231]
  Multi-modal pre-trained big models have drawn more and more attention in recent years. This paper introduces the background of multi-modal pre-training by reviewing the conventional deep, pre-training works in natural language process, computer vision, and speech. Then, we introduce the task definition, key challenges, and advantages of multi-modal pre-training models (MM-PTMs), and discuss the MM-PTMs with a focus on data, objectives, network, and knowledge enhanced pre-training.
  arXiv  Detail & Related papers  (2023-02-20T15:34:03Z)
• The Next Chapter: A Study of Large Language Models in Storytelling [51.338324023617034]
  The application of prompt-based learning with large language models (LLMs) has exhibited remarkable performance in diverse natural language processing (NLP) tasks. This paper conducts a comprehensive investigation, utilizing both automatic and human evaluation, to compare the story generation capacity of LLMs with recent models. The results demonstrate that LLMs generate stories of significantly higher quality compared to other story generation models.
  arXiv  Detail & Related papers  (2023-01-24T02:44:02Z)
• Transformers Go for the LOLs: Generating (Humourous) Titles from Scientific Abstracts End-to-End [26.53850343633923]
  We consider the end-to-end abstract-to-title generation problem, exploring seven recent transformer based models. We also consider the harder problem of generating humorous paper titles. For the latter, we compile the first large-scale humor annotated dataset for scientific papers in the NLP/ML domains.
  arXiv  Detail & Related papers  (2022-12-20T18:37:11Z)
• How Large Language Models are Transforming Machine-Paraphrased Plagiarism [3.8768839735240737]
  This work explores T5 and GPT-3 for machine-paraphrase generation on scientific articles from arXiv, student theses, and Wikipedia. We evaluate the detection performance of six automated solutions and one commercial plagiarism detection software. Human experts rate the quality of paraphrases generated by GPT-3 as high as original texts.
  arXiv  Detail & Related papers  (2022-10-07T14:08:57Z)
• Development of an Extractive Title Generation System Using Titles of Papers of Top Conferences for Intermediate English Students [0.0]
  This study develops an extractive title generation system that formulates titles from keywords extracted from an abstract. We also realize a title evaluation model that can evaluate the appropriateness of paper titles. The results show that our evaluation model can identify top-conference titles more effectively than intermediate English and beginner students.
  arXiv  Detail & Related papers  (2021-10-08T15:41:27Z)
• Fine-tuning GPT-3 for Russian Text Summarization [77.34726150561087]
  This paper showcases ruGPT3 ability to summarize texts, fine-tuning it on the corpora of Russian news with their corresponding human-generated summaries. We evaluate the resulting texts with a set of metrics, showing that our solution can surpass the state-of-the-art model's performance without additional changes in architecture or loss function.
  arXiv  Detail & Related papers  (2021-08-07T19:01:40Z)
• Domain Controlled Title Generation with Human Evaluation [2.5505887482902287]
  A good title allows you to get the attention that your research deserves. For domain-controlled titles, we used the pre-trained text-to-text transformer model and the additional token technique. Title tokens are sampled from a local distribution (which is a subset of global vocabulary) of the domain-specific vocabulary and not global vocabulary.
  arXiv  Detail & Related papers  (2021-03-08T20:55:55Z)
• Hooks in the Headline: Learning to Generate Headlines with Controlled Styles [69.30101340243375]
  We propose a new task, Stylistic Headline Generation (SHG), to enrich the headlines with three style options. TitleStylist generates style-specific headlines by combining the summarization and reconstruction tasks into a multitasking framework. The attraction score of our model generated headlines surpasses that of the state-of-the-art summarization model by 9.68%, and even outperforms human-written references.
  arXiv  Detail & Related papers  (2020-04-04T17:24:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.