LLM-Collaboration on Automatic Science Journalism for the General Audience

• URL: http://arxiv.org/abs/2407.09756v1
• Date: Sat, 13 Jul 2024 03:31:35 GMT
• Title: LLM-Collaboration on Automatic Science Journalism for the General Audience
• Authors: Gongyao Jiang, Xinran Shi, Qiong Luo,
• Abstract summary: Science journalism reports current scientific discoveries to non-specialists. This task can be challenging as the audience often lacks specific knowledge about the presented research. We propose a framework that integrates three LLMs mimicking the real-world writing-reading-feedback-revision workflow.
• Score: 3.591143309194537
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Science journalism reports current scientific discoveries to non-specialists, aiming to enable public comprehension of the state of the art. However, this task can be challenging as the audience often lacks specific knowledge about the presented research. To address this challenge, we propose a framework that integrates three LLMs mimicking the real-world writing-reading-feedback-revision workflow, with one LLM acting as the journalist, a smaller LLM as the general public reader, and the third LLM as an editor. The journalist's writing is iteratively refined by feedback from the reader and suggestions from the editor. Our experiments demonstrate that by leveraging the collaboration of two 7B and one 1.8B open-source LLMs, we can generate articles that are more accessible than those generated by existing methods, including advanced models such as GPT-4.

Related papers

• "Ownership, Not Just Happy Talk": Co-Designing a Participatory Large Language Model for Journalism [7.25169954977234]
  Journalism has emerged as an essential domain for understanding the uses, limitations, and impacts of large language models (LLMs) in the workplace. How might a journalist-led LLM work, and what can participatory design illuminate about the present-day challenges about adapting one-size-fits-all'' foundation models to a given context of use? Our 20 interviews with reporters, data journalists, editors, labor organizers, product leads, and executives highlight macro, meso, and micro tensions that designing for this opportunity space must address.
  arXiv  Detail & Related papers  (2025-01-28T21:06:52Z)
• JRE-L: Journalist, Reader, and Editor LLMs in the Loop for Science Journalism for the General Audience [3.591143309194537]
  Science journalism reports current scientific discoveries to non-specialists, aiming to enable public comprehension of the state of the art. We propose a JRE-L framework that integrates three LLMs mimicking the writing-reading-feedback-revision loop. Our code is publicly available at accessible.com/Zzoay/JRE-L.
  arXiv  Detail & Related papers  (2025-01-28T11:30:35Z)
• Understanding the Dark Side of LLMs' Intrinsic Self-Correction [55.51468462722138]
  Intrinsic self-correction was proposed to improve LLMs' responses via feedback prompts solely based on their inherent capability. Recent works show that LLMs' intrinsic self-correction fails without oracle labels as feedback prompts. We identify intrinsic self-correction can cause LLMs to waver both intermedia and final answers and lead to prompt bias on simple factual questions.
  arXiv  Detail & Related papers  (2024-12-19T15:39:31Z)
• From Test-Taking to Test-Making: Examining LLM Authoring of Commonsense Assessment Items [0.18416014644193068]
  We consider LLMs as authors of commonsense assessment items. We prompt LLMs to generate items in the style of a prominent benchmark for commonsense reasoning. We find that LLMs that succeed in answering the original COPA benchmark are also more successful in authoring their own items.
  arXiv  Detail & Related papers  (2024-10-18T22:42:23Z)
• LLMs Assist NLP Researchers: Critique Paper (Meta-)Reviewing [106.45895712717612]
  Large language models (LLMs) have shown remarkable versatility in various generative tasks. This study focuses on the topic of LLMs assist NLP Researchers. To our knowledge, this is the first work to provide such a comprehensive analysis.
  arXiv  Detail & Related papers  (2024-06-24T01:30:22Z)
• Developing Story: Case Studies of Generative AI's Use in Journalism [18.67676679963561]
  We conduct a study of journalist-AI interactions by two news agencies through browsing the WildChat dataset. Our analysis uncovers instances where journalists provide sensitive material such as confidential correspondence with sources or articles from other agencies to the LLM as stimuli and prompt it to generate articles. Based on our findings, we call for further research into what constitutes responsible use of AI, and the establishment of clear guidelines and best practices on using LLMs in a journalistic context.
  arXiv  Detail & Related papers  (2024-06-19T16:58:32Z)
• LLMs as Meta-Reviewers' Assistants: A Case Study [4.345138609587135]
  Large Language Models (LLMs) can be used to generate a controlled multi-perspective summary (MPS) of experts opinions. This paper performs a case study with three popular LLMs, i.e., GPT-3.5, LLaMA2, and PaLM2, to assist meta-reviewers in better comprehending experts perspectives.
  arXiv  Detail & Related papers  (2024-02-23T20:14:16Z)
• DELL: Generating Reactions and Explanations for LLM-Based Misinformation Detection [50.805599761583444]
  Large language models are limited by challenges in factuality and hallucinations to be directly employed off-the-shelf for judging the veracity of news articles. We propose Dell that identifies three key stages in misinformation detection where LLMs could be incorporated as part of the pipeline.
  arXiv  Detail & Related papers  (2024-02-16T03:24:56Z)
• Guiding LLM to Fool Itself: Automatically Manipulating Machine Reading Comprehension Shortcut Triggers [76.77077447576679]
  Shortcuts, mechanisms triggered by features spuriously correlated to the true label, has emerged as a potential threat to Machine Reading (MRC) systems. We introduce a framework that guides an editor to add potential shortcuts-triggers to samples. Using GPT4 as the editor, we find it can successfully edit trigger shortcut in samples that fool LLMs.
  arXiv  Detail & Related papers  (2023-10-24T12:37:06Z)
• Eva-KELLM: A New Benchmark for Evaluating Knowledge Editing of LLMs [54.22416829200613]
  Eva-KELLM is a new benchmark for evaluating knowledge editing of large language models. Experimental results indicate that the current methods for knowledge editing using raw documents are not effective in yielding satisfactory results.
  arXiv  Detail & Related papers  (2023-08-19T09:17:19Z)
• A Comprehensive Overview of Large Language Models [68.22178313875618]
  Large Language Models (LLMs) have recently demonstrated remarkable capabilities in natural language processing tasks. This article provides an overview of the existing literature on a broad range of LLM-related concepts.
  arXiv  Detail & Related papers  (2023-07-12T20:01:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.