Language as a Material Interface for Creative LLM Interaction

• URL: http://arxiv.org/abs/2607.24753v1
• Date: Mon, 25 May 2026 02:28:44 GMT
• Title: Language as a Material Interface for Creative LLM Interaction
• Abstract summary: We explore how creative practitioners might work with AI systems when language is treated not merely as instruction but as material.<n>We conducted an ecological two-week study with four creative practitioners using a design probe: the Memetic Mixer.<n>Analysis of post-study interviews and device logs identified distinct modes of material language use and temporalities that shaped each participant's engagement with AI and their creative practice.
• Score: 5.306307106311875
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Although directive prompting is the predominant way to interact with Large Language Models (LLMs), many creative practices rely on language that is open-ended, associative, phonaesthetic, symbolic, and that unfolds across multiple temporalities. In this work, we explore how creative practitioners might work with AI systems when language is treated not merely as instruction but as material. We conducted an ecological two-week study with four creative practitioners using a design probe: the Memetic Mixer, a tangible interactive device that constrains interaction with an LLM. Analysis of post-study interviews and device logs identified distinct modes of material language use and temporalities that shaped each participant's engagement with AI and their creative practice. We reflect on these findings and contribute design considerations that support open-ended interaction with AI in creative practice.

Related papers

• Shaping Schema via Language Representation as the Next Frontier for LLM Intelligence Expanding [88.11781604392606]
  This paper defines language representation as the linguistic and symbolic constructs used to map and model the real world.<n>We argue that shaping schemas through advanced language representation is the next frontier for expanding Large Language Models.
  arXiv  Detail & Related papers  (2026-05-10T02:42:29Z)
• Rapport du Projet de Recherche TRAIMA [0.7440170908149745]
  The project addresses a central methodological challenge in educational and interactional research.<n>The analysis of verbal, paraverbal, and non-verbal data is currently carried out manually, making it extremely time-consuming and difficult to scale.<n>The project focuses specifically on explanatory and collaborative sequences occurring in classroom interactions.
  arXiv  Detail & Related papers  (2026-01-19T08:55:50Z)
• A vibe coding learning design to enhance EFL students' talking to, through, and about AI [5.671810777243196]
  This article reports on the piloting of vibe coding (using natural language to create software applications with AI) for English as a Foreign Language education.<n>We developed a human-AI meta-languaging framework with three dimensions: talking to AI (prompt engineering), talking through AI (negotiating authorship), and talking about AI.
  arXiv  Detail & Related papers  (2025-09-09T00:27:04Z)
• ThematicPlane: Bridging Tacit User Intent and Latent Spaces for Image Generation [49.805992099208595]
  We introduce ThematicPlane, a system that enables users to navigate and manipulate high-level semantic concepts.<n>This interface bridges the gap between tacit creative intent and system control.
  arXiv  Detail & Related papers  (2025-08-08T06:57:14Z)
• A Novel Idea Generation Tool using a Structured Conversational AI (CAI) System [0.0]
  This paper presents a novel conversational AI-enabled active ideation interface as a creative idea-generation tool to assist novice designers. It is a dynamic, interactive, and contextually responsive approach, actively involving a large language model (LLM) from the domain of natural language processing (NLP) in artificial intelligence (AI) Integrating such AI models with ideation creates what we refer to as an Active Ideation scenario, which helps foster continuous dialogue-based interaction, context-sensitive conversation, and prolific idea generation.
  arXiv  Detail & Related papers  (2024-09-09T16:02:27Z)
• BiosERC: Integrating Biography Speakers Supported by LLMs for ERC Tasks [2.9873893715462176]
  This work introduces a novel framework named BiosERC, which investigates speaker characteristics in a conversation. By employing Large Language Models (LLMs), we extract the "biographical information" of the speaker within a conversation. Our proposed method achieved state-of-the-art (SOTA) results on three famous benchmark datasets.
  arXiv  Detail & Related papers  (2024-07-05T06:25:34Z)
• Purposeful remixing with generative AI: Constructing designer voice in multimodal composing [16.24460569356749]
  This study investigates whether the use of generative AI tools could help student authors construct a more consistent voice in multimodal writing. The study sheds light on the intentional and discursive nature of multimodal writing with AI as afforded by the technological flexibility.
  arXiv  Detail & Related papers  (2024-03-28T02:15:03Z)
• Detecting Any Human-Object Interaction Relationship: Universal HOI Detector with Spatial Prompt Learning on Foundation Models [55.20626448358655]
  This study explores the universal interaction recognition in an open-world setting through the use of Vision-Language (VL) foundation models and large language models (LLMs) Our design includes an HO Prompt-guided Decoder (HOPD), facilitates the association of high-level relation representations in the foundation model with various HO pairs within the image. For open-category interaction recognition, our method supports either of two input types: interaction phrase or interpretive sentence.
  arXiv  Detail & Related papers  (2023-11-07T08:27:32Z)
• LLaVA-Interactive: An All-in-One Demo for Image Chat, Segmentation, Generation and Editing [99.80742991922992]
  The system can have multi-turn dialogues with human users by taking multimodal user inputs and generating multimodal responses. LLaVA-Interactive goes beyond language prompt, where visual prompt is enabled to align human intents in the interaction.
  arXiv  Detail & Related papers  (2023-11-01T15:13:43Z)
• Interactive Natural Language Processing [67.87925315773924]
  Interactive Natural Language Processing (iNLP) has emerged as a novel paradigm within the field of NLP. This paper offers a comprehensive survey of iNLP, starting by proposing a unified definition and framework of the concept.
  arXiv  Detail & Related papers  (2023-05-22T17:18:29Z)
• Channel-aware Decoupling Network for Multi-turn Dialogue Comprehension [81.47133615169203]
  We propose compositional learning for holistic interaction across utterances beyond the sequential contextualization from PrLMs. We employ domain-adaptive training strategies to help the model adapt to the dialogue domains. Experimental results show that our method substantially boosts the strong PrLM baselines in four public benchmark datasets.
  arXiv  Detail & Related papers  (2023-01-10T13:18:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.