MCP2OSC: Parametric Control by Natural Language

• URL: http://arxiv.org/abs/2508.10414v1
• Date: Thu, 14 Aug 2025 07:38:01 GMT
• Title: MCP2OSC: Parametric Control by Natural Language
• Abstract summary: This study focuses on exploring parametric OSC (OpenSoundControl) control by natural language prompts.<n>Claude is integrated with the MCP2OSC server effective in generating OSC messages by natural language.<n>Results suggest its potential for a LLM-based universal control mechanism for multimedia devices.
• Score: 0.8158530638728501
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Text prompts enable intuitive content creation but may fall short in achieving high precision for intricate tasks; knob or slider controls offer precise adjustments at the cost of increased complexity. To address the gap between knobs and prompts, a new MCP (Model Context Protocol) server and a unique set of prompt design criteria are presented to enable exploring parametric OSC (OpenSoundControl) control by natural language prompts. Demonstrated by 14 practical QA examples with best practices and the generalized prompt templates, this study finds Claude integrated with the MCP2OSC server effective in generating OSC messages by natural language, interpreting, searching, and visualizing OSC messages, validating and debugging OSC messages, and managing OSC address patterns. MCP2OSC enhances human-machine collaboration by leveraging LLM (Large Language Model) to handle intricate OSC development tasks, and by empowering human creativity with an intuitive language interface featuring flexible precision controls: a prompt-based OSC tool. This study provides a novel perspective on the creative MCP application at the network protocol level by utilizing LLM's strength in directly processing and generating human-readable OSC messages. The results suggest its potential for a LLM-based universal control mechanism for multimedia devices.

Related papers

• When LLMs Develop Languages: Symbolic Communication for Efficient Multi-Agent Reasoning [85.36421257648294]
  Chain-of-Thought (CoT) improves large language models (LLMs) on difficult reasoning tasks.<n>We propose Communicative Language Symbolism Routing (R)<n>R treats each Language Symbolism Framework as a reusable symbolic protocol with compact symbols, usage rules, and a message-passing contract.<n>R reduces latency-oriented generated token completion by $3sim 6times$ compared to standard CoT.
  arXiv  Detail & Related papers  (2026-06-28T12:02:42Z)
• From Words to Widgets for Controllable LLM Generation [23.17092768490849]
  We propose Malleable Prompting, a new interactive prompting technique for controllable large language models (LLMs) generation.<n>It reifies preference expressions in natural language prompts into GUI widgets that users can configure to steer generation.<n>We show that Malleable Prompting helps participants achieve target preferences more precisely and is perceived as more controllable and transparent than natural language prompting alone.
  arXiv  Detail & Related papers  (2026-04-13T02:46:31Z)
• CMLFormer: A Dual Decoder Transformer with Switching Point Learning for Code-Mixed Language Modeling [0.0]
  CMLFormer is an enhanced multi-layer dual-decoder Transformer with a shared encoder and synchronized decoder cross-attention.<n>CMLFormer is pre-trained on an augmented Hinglish corpus with switching point and translation annotations.<n>Experiments show that CMLFormer improves F1 score, precision, and accuracy over other approaches.
  arXiv  Detail & Related papers  (2025-05-19T00:50:49Z)
• ESPnet-SpeechLM: An Open Speech Language Model Toolkit [98.4525334631522]
  We present ESPnet-SpeechLM, an open toolkit designed to democratize the development of speech language models (SpeechLMs)<n>The toolkit standardizes speech processing tasks by framing them as universal sequential modeling problems.<n>With ESPnet-SpeechLM, users can easily define task templates and configure key settings, enabling seamless and streamlined SpeechLM development.
  arXiv  Detail & Related papers  (2025-02-21T05:21:58Z)
• Boosting Code-Switching ASR with Mixture of Experts Enhanced Speech-Conditioned LLM [1.3089936156875277]
  We introduce a speech-conditioned Large Language Model (LLM) integrated with a Mixture of Experts (MoE) based connector. We propose an Insertion and Deletion of Interruption Token (IDIT) mechanism for better transfer text generation ability of LLM to speech recognition task. We also present a connecter with MoE architecture that manages multiple languages efficiently.
  arXiv  Detail & Related papers  (2024-09-24T09:20:22Z)
• Natural Language as Policies: Reasoning for Coordinate-Level Embodied Control with LLMs [7.746160514029531]
  We demonstrate experimental results with LLMs that address robotics task planning problems. Our approach acquires text descriptions of the task and scene objects, then formulates task planning through natural language reasoning. Our approach is evaluated on a multi-modal prompt simulation benchmark.
  arXiv  Detail & Related papers  (2024-03-20T17:58:12Z)
• CoCo-Agent: A Comprehensive Cognitive MLLM Agent for Smartphone GUI Automation [61.68049335444254]
  Multimodal large language models (MLLMs) have shown remarkable potential as human-like autonomous language agents to interact with real-world environments. We propose a Comprehensive Cognitive LLM Agent, CoCo-Agent, with two novel approaches, comprehensive environment perception (CEP) and conditional action prediction (CAP) With our technical design, our agent achieves new state-of-the-art performance on AITW and META-GUI benchmarks, showing promising abilities in realistic scenarios.
  arXiv  Detail & Related papers  (2024-02-19T08:29:03Z)
• kNN-ICL: Compositional Task-Oriented Parsing Generalization with Nearest Neighbor In-Context Learning [50.40636157214161]
  Task-Oriented Parsing (TOP) enables conversational assistants to interpret user commands expressed in natural language. LLMs have achieved impressive performance in computer programs based on a natural language prompt. This paper focuses on harnessing the capabilities of LLMs for semantic parsing tasks.
  arXiv  Detail & Related papers  (2023-12-17T17:26:50Z)
• Human-Centric Autonomous Systems With LLMs for User Command Reasoning [16.452638202694246]
  We propose to leverage the reasoning capabilities of Large Language Models to infer system requirements from in-cabin users' commands. We confirm the general ability of LLMs to understand and reason about prompts but underline that their effectiveness is conditioned on the quality of both the LLM model and the design of appropriate sequential prompts.
  arXiv  Detail & Related papers  (2023-11-14T14:42:28Z)
• Language-Oriented Communication with Semantic Coding and Knowledge Distillation for Text-to-Image Generation [53.97155730116369]
  We put forward a novel framework of language-oriented semantic communication (LSC) In LSC, machines communicate using human language messages that can be interpreted and manipulated via natural language processing (NLP) techniques for SC efficiency. We introduce three innovative algorithms: 1) semantic source coding (SSC), which compresses a text prompt into its key head words capturing the prompt's syntactic essence; 2) semantic channel coding ( SCC), that improves robustness against errors by substituting head words with their lenghthier synonyms; and 3) semantic knowledge distillation (SKD), that produces listener-customized prompts via in-context learning the listener's
  arXiv  Detail & Related papers  (2023-09-20T08:19:05Z)
• Sequential Monte Carlo Steering of Large Language Models using Probabilistic Programs [46.721838623748816]
  We propose a new inference-time approach to enforcing syntactic and semantic constraints on the outputs of large language models. Key idea is to specify language generation tasks as posterior inference problems in a class of discrete probabilistic sequence models. For a computational cost similar to that of beam search, SMC can steer LLMs to solve diverse tasks.
  arXiv  Detail & Related papers  (2023-06-05T17:55:05Z)
• Low-code LLM: Graphical User Interface over Large Language Models [115.08718239772107]
  This paper introduces a novel human-LLM interaction framework, Low-code LLM. It incorporates six types of simple low-code visual programming interactions to achieve more controllable and stable responses. We highlight three advantages of the low-code LLM: user-friendly interaction, controllable generation, and wide applicability.
  arXiv  Detail & Related papers  (2023-04-17T09:27:40Z)
• Learning Combinatorial Prompts for Universal Controllable Image Captioning [46.34624079520254]
  Controllable Image Captioning -- generating natural language descriptions about images under the guidance of given control signals -- is one of the most promising directions towards next-generation captioning systems. We propose a novel prompt-based framework for CIC by learning Combinatorial Prompts, dubbed as ComPro. Our ComPro can be further extended to more kinds of combined control signals by concatenating these prompts.
  arXiv  Detail & Related papers  (2023-03-11T07:53:15Z)
• PADL: Language-Directed Physics-Based Character Control [66.517142635815]
  We present PADL, which allows users to issue natural language commands for specifying high-level tasks and low-level skills that a character should perform. We show that our framework can be applied to effectively direct a simulated humanoid character to perform a diverse array of complex motor skills.
  arXiv  Detail & Related papers  (2023-01-31T18:59:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.