Know Your Audience: Do LLMs Adapt to Different Age and Education Levels?

• URL: http://arxiv.org/abs/2312.02065v1
• Date: Mon, 4 Dec 2023 17:19:53 GMT
• Title: Know Your Audience: Do LLMs Adapt to Different Age and Education Levels?
• Authors: Donya Rooein, Amanda Cercas Curry, Dirk Hovy
• Abstract summary: We evaluate the readability of answers generated by four state-of-the-art large language models (LLMs) We compare the readability scores of the generated responses against the recommended comprehension level of each age and education group. Our results suggest LLM answers need to be better adapted to the intended audience to be more comprehensible.
• Score: 21.302967282814784
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) offer a range of new possibilities, including adapting the text to different audiences and their reading needs. But how well do they adapt? We evaluate the readability of answers generated by four state-of-the-art LLMs (commercial and open-source) to science questions when prompted to target different age groups and education levels. To assess the adaptability of LLMs to diverse audiences, we compare the readability scores of the generated responses against the recommended comprehension level of each age and education group. We find large variations in the readability of the answers by different LLMs. Our results suggest LLM answers need to be better adapted to the intended audience demographics to be more comprehensible. They underline the importance of enhancing the adaptability of LLMs in education settings to cater to diverse age and education levels. Overall, current LLMs have set readability ranges and do not adapt well to different audiences, even when prompted. That limits their potential for educational purposes.

Related papers

• Alignment Revisited: Are Large Language Models Consistent in Stated and Revealed Preferences? [5.542420010310746]
  A critical, yet understudied, issue is the potential divergence between an LLM's stated preferences and its revealed preferences.<n>This work formally defines and proposes a method to measure this preference deviation.<n>Our study will be crucial for integrating LLMs into services, especially those that interact directly with humans.
  arXiv  Detail & Related papers  (2025-05-31T23:38:48Z)
• Embracing AI in Education: Understanding the Surge in Large Language Model Use by Secondary Students [53.20318273452059]
  Large language models (LLMs) like OpenAI's ChatGPT have opened up new avenues in education. Despite school restrictions, our survey of over 300 middle and high school students revealed that a remarkable 70% of students have utilized LLMs. We propose a few ideas to address such issues, including subject-specific models, personalized learning, and AI classrooms.
  arXiv  Detail & Related papers  (2024-11-27T19:19:34Z)
• What can LLM tell us about cities? [6.405546719612814]
  This study explores the capabilities of large language models (LLMs) in providing knowledge about cities and regions on a global scale. Experiments reveal that LLMs embed a broad but varying degree of knowledge across global cities, with ML models trained on LLM-derived features consistently leading to improved predictive accuracy.
  arXiv  Detail & Related papers  (2024-11-25T09:07:56Z)
• Hate Personified: Investigating the role of LLMs in content moderation [64.26243779985393]
  For subjective tasks such as hate detection, where people perceive hate differently, the Large Language Model's (LLM) ability to represent diverse groups is unclear. By including additional context in prompts, we analyze LLM's sensitivity to geographical priming, persona attributes, and numerical information to assess how well the needs of various groups are reflected.
  arXiv  Detail & Related papers  (2024-10-03T16:43:17Z)
• Are LLMs Aware that Some Questions are not Open-ended? [58.93124686141781]
  We study whether Large Language Models are aware that some questions have limited answers and need to respond more deterministically. The lack of question awareness in LLMs leads to two phenomena: (1) too casual to answer non-open-ended questions or (2) too boring to answer open-ended questions.
  arXiv  Detail & Related papers  (2024-10-01T06:07:00Z)
• AI Meets the Classroom: When Do Large Language Models Harm Learning? [0.0]
  We show that the effect of large language models (LLMs) on learning outcomes depends on usage behavior. While LLMs show great potential to improve learning, their use must be tailored to the educational context.
  arXiv  Detail & Related papers  (2024-08-29T17:07:46Z)
• Beyond Flesch-Kincaid: Prompt-based Metrics Improve Difficulty Classification of Educational Texts [20.933548500888595]
  Using large language models (LLMs) for educational applications like dialogue-based teaching is a hot topic. Current Static metrics for text difficulty, like the Flesch-Kincaid Reading Ease score, are known to be crude and brittle. We introduce and evaluate a new set of Prompt-based metrics for text difficulty.
  arXiv  Detail & Related papers  (2024-05-15T16:22:16Z)
• When Do LLMs Need Retrieval Augmentation? Mitigating LLMs' Overconfidence Helps Retrieval Augmentation [66.01754585188739]
  Large Language Models (LLMs) have been found to have difficulty knowing they do not possess certain knowledge. Retrieval Augmentation (RA) has been extensively studied to mitigate LLMs' hallucinations. We propose several methods to enhance LLMs' perception of knowledge boundaries and show that they are effective in reducing overconfidence.
  arXiv  Detail & Related papers  (2024-02-18T04:57:19Z)
• When LLMs Meet Cunning Texts: A Fallacy Understanding Benchmark for Large Language Models [59.84769254832941]
  We propose a FaLlacy Understanding Benchmark (FLUB) containing cunning texts that are easy for humans to understand but difficult for models to grasp. Specifically, the cunning texts that FLUB focuses on mainly consist of the tricky, humorous, and misleading texts collected from the real internet environment. Based on FLUB, we investigate the performance of multiple representative and advanced LLMs.
  arXiv  Detail & Related papers  (2024-02-16T22:12:53Z)
• Large Language Models: A Survey [69.72787936480394]
  Large Language Models (LLMs) have drawn a lot of attention due to their strong performance on a wide range of natural language tasks. LLMs' ability of general-purpose language understanding and generation is acquired by training billions of model's parameters on massive amounts of text data.
  arXiv  Detail & Related papers  (2024-02-09T05:37:09Z)
• See the Unseen: Better Context-Consistent Knowledge-Editing by Noises [73.54237379082795]
  Knowledge-editing updates knowledge of large language models (LLMs) Existing works ignore this property and the editing lacks generalization. We empirically find that the effects of different contexts upon LLMs in recalling the same knowledge follow a Gaussian-like distribution.
  arXiv  Detail & Related papers  (2024-01-15T09:09:14Z)
• Investigating Answerability of LLMs for Long-Form Question Answering [35.41413072729483]
  We focus on long-form question answering (LFQA) because it has several practical and impactful applications. We propose a question-generation method from abstractive summaries and show that generating follow-up questions from summaries of long documents can create a challenging setting.
  arXiv  Detail & Related papers  (2023-09-15T07:22:56Z)
• Head-to-Tail: How Knowledgeable are Large Language Models (LLMs)? A.K.A. Will LLMs Replace Knowledge Graphs? [24.931467926497152]
  Head-to-Tail is a benchmark that consists of 18K question-answer pairs regarding head, torso, and tail facts in terms of popularity. We show that existing LLMs are still far from being perfect in terms of their grasp of factual knowledge, especially for facts of torso-to-tail entities.
  arXiv  Detail & Related papers  (2023-08-20T05:31:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.