Purposeful remixing with generative AI: Constructing designer voice in multimodal composing

• URL: http://arxiv.org/abs/2403.19095v1
• Date: Thu, 28 Mar 2024 02:15:03 GMT
• Title: Purposeful remixing with generative AI: Constructing designer voice in multimodal composing
• Authors: Xiao Tan, Wei Xu, Chaoran Wang,
• Abstract summary: This study investigates whether the use of generative AI tools could help student authors construct a more consistent voice in multimodal writing. The study sheds light on the intentional and discursive nature of multimodal writing with AI as afforded by the technological flexibility.
• Score: 16.24460569356749
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Voice, the discursive construction of the writer's identity, has been extensively studied and theorized in composition studies. In multimodal writing, students are able to mobilize both linguistic and non linguistic resources to express their real or imagined identities. But at the same time, when students are limited to choose from available online resources, their voices might be compromised due to the incompatibility between their authorial intentions and the existing materials. This study, therefore, investigates whether the use of generative AI tools could help student authors construct a more consistent voice in multimodal writing. In this study, we have designed a photo essay assignment where students recount a story in the form of photo essays and prompt AI image generating tools to create photos for their storytelling. Drawing on interview data, written reflection, written annotation, and multimodal products from seven focal participants, we have identified two remixing practices, through which students attempted to establish a coherent and unique voice in writing. The study sheds light on the intentional and discursive nature of multimodal writing with AI as afforded by the technological flexibility, while also highlighting the practical and ethical challenges that could be attributed to students insufficient prompt and multimodal literacy and the innate limitations of AI systems. This study provides important implications for incorporating AI tools in designing multimodal writing tasks.

Related papers

• SARD: A Human-AI Collaborative Story Generation [0.0]
  We propose SARD, a drag-and-drop visual interface for generating a multi-chapter story using large language models. Our evaluation of the usability of SARD and its creativity support shows that while node-based visualization of the narrative may help writers build a mental model, it exerts unnecessary mental overhead to the writer. We also found that AI generates stories that are less lexically diverse, irrespective of the complexity of the story.
  arXiv  Detail & Related papers  (2024-03-03T17:48:42Z)
• Techniques for supercharging academic writing with generative AI [0.0]
  This Perspective maps out principles and methods for using generative artificial intelligence (AI) to elevate the quality and efficiency of academic writing. We introduce a human-AI collaborative framework that delineates the rationale (why), process (how), and nature (what) of AI engagement in writing.
  arXiv  Detail & Related papers  (2023-10-26T04:35:00Z)
• TextBind: Multi-turn Interleaved Multimodal Instruction-following in the Wild [102.93338424976959]
  We introduce TextBind, an almost annotation-free framework for empowering larger language models with the multi-turn interleaved instruction-following capabilities. Our approach requires only image-caption pairs and generates multi-turn multimodal instruction-response conversations from a language model. To accommodate interleaved image-text inputs and outputs, we devise MIM, a language model-centric architecture that seamlessly integrates image encoder and decoder models.
  arXiv  Detail & Related papers  (2023-09-14T15:34:01Z)
• Exploring EFL students' prompt engineering in human-AI story writing: an Activity Theory perspective [4.0109641418513355]
  This study applies Activity Theory to investigate how English as a foreign language (EFL) students prompt generative artificial intelligence (AI) tools during short story writing. The study collected and analyzed the students' generative-AI tools, short stories, and written reflections on their conditions or purposes for prompting.
  arXiv  Detail & Related papers  (2023-06-01T14:52:28Z)
• The Role of AI in Human-AI Creative Writing for Hong Kong Secondary Students [4.739597165434651]
  Recent advancement in Natural Language Processing has led to the development of language models capable of generating human-like language. Our empirical findings show that language models play different roles in helping student writers to be more creative.
  arXiv  Detail & Related papers  (2023-04-21T23:50:09Z)
• Beyond Summarization: Designing AI Support for Real-World Expository Writing Tasks [28.702425557409516]
  Large language models have introduced exciting new opportunities and challenges in designing and developing new AI-assisted writing support tools. Recent work has shown that leveraging this new technology can transform writing in many scenarios such as ideation during creative writing, editing support, and summarization. We argue that developing AI supports for expository writing has unique and exciting research challenges and can lead to high real-world impacts.
  arXiv  Detail & Related papers  (2023-04-05T17:47:11Z)
• Vision+X: A Survey on Multimodal Learning in the Light of Data [64.03266872103835]
  multimodal machine learning that incorporates data from various sources has become an increasingly popular research area. We analyze the commonness and uniqueness of each data format mainly ranging from vision, audio, text, and motions. We investigate the existing literature on multimodal learning from both the representation learning and downstream application levels.
  arXiv  Detail & Related papers  (2022-10-05T13:14:57Z)
• Multimodal Lecture Presentations Dataset: Understanding Multimodality in Educational Slides [57.86931911522967]
  We test the capabilities of machine learning models in multimodal understanding of educational content. Our dataset contains aligned slides and spoken language, for 180+ hours of video and 9000+ slides, with 10 lecturers from various subjects. We introduce PolyViLT, a multimodal transformer trained with a multi-instance learning loss that is more effective than current approaches.
  arXiv  Detail & Related papers  (2022-08-17T05:30:18Z)
• Personalized Multimodal Feedback Generation in Education [50.95346877192268]
  The automatic evaluation for school assignments is an important application of AI in the education field. We propose a novel Personalized Multimodal Feedback Generation Network (PMFGN) armed with a modality gate mechanism and a personalized bias mechanism. Our model significantly outperforms several baselines by generating more accurate and diverse feedback.
  arXiv  Detail & Related papers  (2020-10-31T05:26:49Z)
• Filling the Gap of Utterance-aware and Speaker-aware Representation for Multi-turn Dialogue [76.88174667929665]
  A multi-turn dialogue is composed of multiple utterances from two or more different speaker roles. In the existing retrieval-based multi-turn dialogue modeling, the pre-trained language models (PrLMs) as encoder represent the dialogues coarsely. We propose a novel model to fill such a gap by modeling the effective utterance-aware and speaker-aware representations entailed in a dialogue history.
  arXiv  Detail & Related papers  (2020-09-14T15:07:19Z)
• Disentangled Speech Embeddings using Cross-modal Self-supervision [119.94362407747437]
  We develop a self-supervised learning objective that exploits the natural cross-modal synchrony between faces and audio in video. We construct a two-stream architecture which: (1) shares low-level features common to both representations; and (2) provides a natural mechanism for explicitly disentangling these factors.
  arXiv  Detail & Related papers  (2020-02-20T14:13:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.