JRE-L: Journalist, Reader, and Editor LLMs in the Loop for Science Journalism for the General Audience

• URL: http://arxiv.org/abs/2501.16865v1
• Date: Tue, 28 Jan 2025 11:30:35 GMT
• Title: JRE-L: Journalist, Reader, and Editor LLMs in the Loop for Science Journalism for the General Audience
• Authors: Gongyao Jiang, Xinran Shi, Qiong Luo,
• Abstract summary: Science journalism reports current scientific discoveries to non-specialists, aiming to enable public comprehension of the state of the art. We propose a JRE-L framework that integrates three LLMs mimicking the writing-reading-feedback-revision loop. Our code is publicly available at accessible.com/Zzoay/JRE-L.
• Score: 3.591143309194537
• License:
• Abstract: Science journalism reports current scientific discoveries to non-specialists, aiming to enable public comprehension of the state of the art. This task is challenging as the audience often lacks specific knowledge about the presented research. We propose a JRE-L framework that integrates three LLMs mimicking the writing-reading-feedback-revision loop. In JRE-L, one LLM acts as the journalist, another LLM as the general public reader, and the third LLM as an editor. The journalist's writing is iteratively refined by feedback from the reader and suggestions from the editor. Our experiments demonstrate that by leveraging the collaboration of two 7B and one 1.8B open-source LLMs, we can generate articles that are more accessible than those generated by existing methods, including prompting single advanced models such as GPT-4 and other LLM-collaboration strategies. Our code is publicly available at github.com/Zzoay/JRE-L.

Related papers

• "Ownership, Not Just Happy Talk": Co-Designing a Participatory Large Language Model for Journalism [7.25169954977234]
  Journalism has emerged as an essential domain for understanding the uses, limitations, and impacts of large language models (LLMs) in the workplace. How might a journalist-led LLM work, and what can participatory design illuminate about the present-day challenges about adapting one-size-fits-all'' foundation models to a given context of use? Our 20 interviews with reporters, data journalists, editors, labor organizers, product leads, and executives highlight macro, meso, and micro tensions that designing for this opportunity space must address.
  arXiv  Detail & Related papers  (2025-01-28T21:06:52Z)
• Understanding the LLM-ification of CHI: Unpacking the Impact of LLMs at CHI through a Systematic Literature Review [8.524433537542484]
  Large language models (LLMs) have been positioned to revolutionize HCI. To-date, there has been little understanding of LLMs' uptake in HCI. We review 153 CHI papers from 2020-24 that engage with LLMs.
  arXiv  Detail & Related papers  (2025-01-22T00:31:51Z)
• From Test-Taking to Test-Making: Examining LLM Authoring of Commonsense Assessment Items [0.18416014644193068]
  We consider LLMs as authors of commonsense assessment items. We prompt LLMs to generate items in the style of a prominent benchmark for commonsense reasoning. We find that LLMs that succeed in answering the original COPA benchmark are also more successful in authoring their own items.
  arXiv  Detail & Related papers  (2024-10-18T22:42:23Z)
• LLM-Collaboration on Automatic Science Journalism for the General Audience [3.591143309194537]
  Science journalism reports current scientific discoveries to non-specialists. This task can be challenging as the audience often lacks specific knowledge about the presented research. We propose a framework that integrates three LLMs mimicking the real-world writing-reading-feedback-revision workflow.
  arXiv  Detail & Related papers  (2024-07-13T03:31:35Z)
• LLMs Assist NLP Researchers: Critique Paper (Meta-)Reviewing [106.45895712717612]
  Large language models (LLMs) have shown remarkable versatility in various generative tasks. This study focuses on the topic of LLMs assist NLP Researchers. To our knowledge, this is the first work to provide such a comprehensive analysis.
  arXiv  Detail & Related papers  (2024-06-24T01:30:22Z)
• Developing Story: Case Studies of Generative AI's Use in Journalism [18.67676679963561]
  We conduct a study of journalist-AI interactions by two news agencies through browsing the WildChat dataset. Our analysis uncovers instances where journalists provide sensitive material such as confidential correspondence with sources or articles from other agencies to the LLM as stimuli and prompt it to generate articles. Based on our findings, we call for further research into what constitutes responsible use of AI, and the establishment of clear guidelines and best practices on using LLMs in a journalistic context.
  arXiv  Detail & Related papers  (2024-06-19T16:58:32Z)
• DELL: Generating Reactions and Explanations for LLM-Based Misinformation Detection [50.805599761583444]
  Large language models are limited by challenges in factuality and hallucinations to be directly employed off-the-shelf for judging the veracity of news articles. We propose Dell that identifies three key stages in misinformation detection where LLMs could be incorporated as part of the pipeline.
  arXiv  Detail & Related papers  (2024-02-16T03:24:56Z)
• See the Unseen: Better Context-Consistent Knowledge-Editing by Noises [73.54237379082795]
  Knowledge-editing updates knowledge of large language models (LLMs) Existing works ignore this property and the editing lacks generalization. We empirically find that the effects of different contexts upon LLMs in recalling the same knowledge follow a Gaussian-like distribution.
  arXiv  Detail & Related papers  (2024-01-15T09:09:14Z)
• Guiding LLM to Fool Itself: Automatically Manipulating Machine Reading Comprehension Shortcut Triggers [76.77077447576679]
  Shortcuts, mechanisms triggered by features spuriously correlated to the true label, has emerged as a potential threat to Machine Reading (MRC) systems. We introduce a framework that guides an editor to add potential shortcuts-triggers to samples. Using GPT4 as the editor, we find it can successfully edit trigger shortcut in samples that fool LLMs.
  arXiv  Detail & Related papers  (2023-10-24T12:37:06Z)
• Eva-KELLM: A New Benchmark for Evaluating Knowledge Editing of LLMs [54.22416829200613]
  Eva-KELLM is a new benchmark for evaluating knowledge editing of large language models. Experimental results indicate that the current methods for knowledge editing using raw documents are not effective in yielding satisfactory results.
  arXiv  Detail & Related papers  (2023-08-19T09:17:19Z)
• A Comprehensive Overview of Large Language Models [68.22178313875618]
  Large Language Models (LLMs) have recently demonstrated remarkable capabilities in natural language processing tasks. This article provides an overview of the existing literature on a broad range of LLM-related concepts.
  arXiv  Detail & Related papers  (2023-07-12T20:01:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.