Meaning-Typed Programming: Language Abstraction and Runtime for Model-Integrated Applications

• URL: http://arxiv.org/abs/2405.08965v4
• Date: Wed, 04 Jun 2025 00:51:13 GMT
• Title: Meaning-Typed Programming: Language Abstraction and Runtime for Model-Integrated Applications
• Authors: Jayanaka L. Dantanarayana, Yiping Kang, Kugesan Sivasothynathan, Christopher Clarke, Baichuan Li, Savini Kashmira, Krisztian Flautner, Lingjia Tang, Jason Mars,
• Abstract summary: This paper presents Meaning-Typed Programming (MTP) model, a novel paradigm that abstracts large language models (LLMs) integration through intuitive language-level constructs.<n>We implement MTP in Jac, a Python superset language, and find that MTP significantly reduces coding complexity while maintaining accuracy and efficiency.<n>For math problems from the GSM8k dataset, MTP achieves accuracy rates approaching 90%, while reducing token usage in 10 out of 13 benchmarks.
• Score: 8.007302441327214
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Software development is shifting from traditional logical programming to model-integrated applications that leverage generative AI and large language models (LLMs) during runtime. However, integrating LLMs remains complex, requiring developers to manually craft prompts and process outputs. Existing tools attempt to assist with prompt engineering, but often introduce additional complexity. This paper presents Meaning-Typed Programming (MTP) model, a novel paradigm that abstracts LLM integration through intuitive language-level constructs. By leveraging the inherent semantic richness of code, MTP automates prompt generation and response handling without additional developer effort. We introduce the by operator for seamless LLM invocation, MT-IR, a meaning-based intermediate representation for semantic extraction, and MT-Runtime, an automated system for managing LLM interactions. We implement MTP in Jac, a Python superset language and find that MTP significantly reduces coding complexity while maintaining accuracy and efficiency. Our evaluation across diverse benchmarks and user studies demonstrates that MTP outperforms existing frameworks such as DSPy and LMQL by reducing lines of code by factors of 2.3-7.5X and 1.3-10.7X respectively. For math problems from the GSM8k dataset, MTP achieves accuracy rates approaching 90%, while reducing token usage in 10 out of 13 benchmarks. This leads to cost savings up to 4.5X and runtime speedups as high as 4.75X. Additionally, MTP demonstrates resilience even when 50% of naming conventions are suboptimal, establishing it as a practical, efficient solution for streamlining model-integrated application development.

Related papers

• VeriOpt: PPA-Aware High-Quality Verilog Generation via Multi-Role LLMs [41.94295877935867]
  VeriOpt is a novel framework that leverages role-based prompting and PPA-aware optimization to produce high-quality, synthesizable Verilog.<n>Our work advances the state-of-the-art AI-driven hardware design by addressing the critical gap between correctness and quality.
  arXiv  Detail & Related papers  (2025-07-20T00:28:55Z)
• Pangu Embedded: An Efficient Dual-system LLM Reasoner with Metacognition [95.54406667705999]
  Pangu Embedded is an efficient Large Language Model (LLM) reasoner developed on Ascend Neural Processing Units (NPUs)<n>It addresses the significant computational costs and inference latency challenges prevalent in existing reasoning-optimized LLMs.<n>It delivers rapid responses and state-of-the-art reasoning quality within a single, unified model architecture.
  arXiv  Detail & Related papers  (2025-05-28T14:03:02Z)
• Self-Steering Language Models [113.96916935955842]
  DisCIPL is a method for "self-steering" language models.<n>DisCIPL uses a Planner model to generate a task-specific inference program.<n>Our work opens up a design space of highly-parallelized Monte Carlo inference strategies.
  arXiv  Detail & Related papers  (2025-04-09T17:54:22Z)
• Modularization is Better: Effective Code Generation with Modular Prompting [9.955541341324007]
  We propose a novel prompting technique, called MoT, to enhance the code generation performance of Large Language Models.<n>MoT exploits modularization principles to decompose complex programming problems into smaller, independent reasoning steps.<n>It structures the reasoning process using an MLR Graph, which hierarchically organizes reasoning steps.
  arXiv  Detail & Related papers  (2025-03-16T12:23:23Z)
• VeriMind: Agentic LLM for Automated Verilog Generation with a Novel Evaluation Metric [4.590930025882158]
  We propose VeriMind, an agentic LLM framework for Verilog code generation.<n>We introduce a novel evaluation metric-pass@ARC-which combines the conventional pass@k measure with Average Refinement Cycles (ARC) to capture both success rate and the efficiency of iterative refinement.<n> Experimental results on diverse hardware design tasks demonstrated that our approach achieved up to $8.3%$ improvement on pass@k metric and $8.1%$ on pass@ARC metric.
  arXiv  Detail & Related papers  (2025-03-15T23:43:06Z)
• AutoIOT: LLM-Driven Automated Natural Language Programming for AIoT Applications [16.47929288038498]
  Large Language Models (LLMs) have profoundly transformed our lives, revolutionizing interactions with AI and lowering the barrier to AI usage. This paper introduces AutoIOT, an automated program generator for AIoT applications.
  arXiv  Detail & Related papers  (2025-03-07T11:40:52Z)
• Promptware Engineering: Software Engineering for LLM Prompt Development [22.788377588087894]
  Large Language Models (LLMs) are increasingly integrated into software applications, with prompts serving as the primary 'programming' interface. As a result, a new software paradigm, promptware, has emerged, using natural language prompts to interact with LLMs. Unlike traditional software, which relies on formal programming languages and deterministic runtime environments, promptware is based on ambiguous, unstructured, and context-dependent natural language.
  arXiv  Detail & Related papers  (2025-03-04T08:43:16Z)
• LLM-AutoDiff: Auto-Differentiate Any LLM Workflow [58.56731133392544]
  We introduce LLM-AutoDiff: a novel framework for Automatic Prompt Engineering (APE)<n>LLMs-AutoDiff treats each textual input as a trainable parameter and uses a frozen backward engine to generate feedback-akin to textual gradients.<n>It consistently outperforms existing textual gradient baselines in both accuracy and training cost.
  arXiv  Detail & Related papers  (2025-01-28T03:18:48Z)
• Creating an LLM-based AI-agent: A high-level methodology towards enhancing LLMs with APIs [0.0]
  Large Language Models (LLMs) have revolutionized various aspects of engineering and science. This thesis serves as a comprehensive guide that elucidates a multi-faceted approach for empowering LLMs with the capability to leverage Application Programming Interfaces (APIs) We propose an on-device architecture that aims to exploit the functionality of carry-on devices by using small models from the Hugging Face community.
  arXiv  Detail & Related papers  (2024-12-17T14:14:04Z)
• Interactive and Expressive Code-Augmented Planning with Large Language Models [62.799579304821826]
  Large Language Models (LLMs) demonstrate strong abilities in common-sense reasoning and interactive decision-making. Recent techniques have sought to structure LLM outputs using control flow and other code-adjacent techniques to improve planning performance. We propose REPL-Plan, an LLM planning approach that is fully code-expressive and dynamic.
  arXiv  Detail & Related papers  (2024-11-21T04:23:17Z)
• Meaning Typed Prompting: A Technique for Efficient, Reliable Structured Output Generation [0.0]
  We introduce Meaning Typed Prompting (MTP), a technique for efficient structured output generation. By utilizing expressive type definitions, MTP enhances output clarity and reduces dependence on complex abstractions. We present Semantix, a framework that implements MTP, providing practical insights into its application.
  arXiv  Detail & Related papers  (2024-10-22T20:43:50Z)
• Codellm-Devkit: A Framework for Contextualizing Code LLMs with Program Analysis Insights [9.414198519543564]
  We present codellm-devkit (hereafter, CLDK'), an open-source library that significantly simplifies the process of performing program analysis. CLDK offers developers an intuitive and user-friendly interface, making it incredibly easy to provide rich program analysis context to code LLMs.
  arXiv  Detail & Related papers  (2024-10-16T20:05:59Z)
• Optimizing Token Usage on Large Language Model Conversations Using the Design Structure Matrix [49.1574468325115]
  Large Language Models become ubiquitous in many sectors and tasks. There is a need to reduce token usage, overcoming challenges such as short context windows, limited output sizes, and costs associated with token intake and generation. This work brings the Design Structure Matrix from the engineering design discipline into LLM conversation optimization.
  arXiv  Detail & Related papers  (2024-10-01T14:38:36Z)
• Reference Trustable Decoding: A Training-Free Augmentation Paradigm for Large Language Models [79.41139393080736]
  Large language models (LLMs) have rapidly advanced and demonstrated impressive capabilities. In-Context Learning (ICL) and. Efficient Fine-Tuning (PEFT) are currently two mainstream methods for augmenting. LLMs to downstream tasks. We propose Reference Trustable Decoding (RTD), a paradigm that allows models to quickly adapt to new tasks without fine-tuning.
  arXiv  Detail & Related papers  (2024-09-30T10:48:20Z)
• Large Language Models are Interpretable Learners [53.56735770834617]
  In this paper, we show a combination of Large Language Models (LLMs) and symbolic programs can bridge the gap between expressiveness and interpretability. The pretrained LLM with natural language prompts provides a massive set of interpretable modules that can transform raw input into natural language concepts. As the knowledge learned by LSP is a combination of natural language descriptions and symbolic rules, it is easily transferable to humans (interpretable) and other LLMs.
  arXiv  Detail & Related papers  (2024-06-25T02:18:15Z)
• APPL: A Prompt Programming Language for Harmonious Integration of Programs and Large Language Model Prompts [21.819126948549766]
  Large Language Models (LLMs) have become increasingly capable of handling diverse tasks with the aid of well-crafted prompts. APPL acts as a bridge between computer programs and LLMs, allowing seamless embedding of prompts into Python functions.
  arXiv  Detail & Related papers  (2024-06-19T02:29:59Z)
• Synthetic Programming Elicitation for Text-to-Code in Very Low-Resource Programming and Formal Languages [21.18996339478024]
  We introduce emphsynthetic programming elicitation and compilation (SPEAC) SPEAC produces syntactically correct programs more frequently and without sacrificing semantic correctness. We empirically evaluate the performance of SPEAC in a case study for the UCLID5 formal verification language.
  arXiv  Detail & Related papers  (2024-06-05T22:16:19Z)
• A Framework to Model ML Engineering Processes [1.9744907811058787]
  Development of Machine Learning (ML) based systems is complex and requires multidisciplinary teams with diverse skill sets. Current process modeling languages are not suitable for describing the development of such systems. We introduce a framework for modeling ML-based software development processes, built around a domain-specific language.
  arXiv  Detail & Related papers  (2024-04-29T09:17:36Z)
• An Empirical Study on Usage and Perceptions of LLMs in a Software Engineering Project [1.433758865948252]
  Large Language Models (LLMs) represent a leap in artificial intelligence, excelling in tasks using human language(s) In this paper, we analyze the AI-generated code, prompts used for code generation, and the human intervention levels to integrate the code into the code base. Our findings suggest that LLMs can play a crucial role in the early stages of software development.
  arXiv  Detail & Related papers  (2024-01-29T14:32:32Z)
• If LLM Is the Wizard, Then Code Is the Wand: A Survey on How Code Empowers Large Language Models to Serve as Intelligent Agents [81.60906807941188]
  Large language models (LLMs) are trained on a combination of natural language and formal language (code) Code translates high-level goals into executable steps, featuring standard syntax, logical consistency, abstraction, and modularity.
  arXiv  Detail & Related papers  (2024-01-01T16:51:20Z)
• LLM4EDA: Emerging Progress in Large Language Models for Electronic Design Automation [74.7163199054881]
  Large Language Models (LLMs) have demonstrated their capability in context understanding, logic reasoning and answer generation. We present a systematic study on the application of LLMs in the EDA field. We highlight the future research direction, focusing on applying LLMs in logic synthesis, physical design, multi-modal feature extraction and alignment of circuits.
  arXiv  Detail & Related papers  (2023-12-28T15:09:14Z)
• LLMind: Orchestrating AI and IoT with LLM for Complex Task Execution [18.816077341295628]
  We present LLMind, a task-oriented AI framework that enables effective collaboration among IoT devices. Inspired by the functional specialization theory of the brain, our framework integrates an LLM with domain-specific AI modules. Complex tasks, which may involve collaborations of multiple domain-specific AI modules and IoT devices, are executed through a control script.
  arXiv  Detail & Related papers  (2023-12-14T14:57:58Z)
• LMRL Gym: Benchmarks for Multi-Turn Reinforcement Learning with Language Models [56.25156596019168]
  This paper introduces the LMRL-Gym benchmark for evaluating multi-turn RL for large language models (LLMs) Our benchmark consists of 8 different language tasks, which require multiple rounds of language interaction and cover a range of tasks in open-ended dialogue and text games.
  arXiv  Detail & Related papers  (2023-11-30T03:59:31Z)
• ML-Bench: Evaluating Large Language Models and Agents for Machine Learning Tasks on Repository-Level Code [76.84199699772903]
  ML-Bench is a benchmark rooted in real-world programming applications that leverage existing code repositories to perform tasks. To evaluate both Large Language Models (LLMs) and AI agents, two setups are employed: ML-LLM-Bench for assessing LLMs' text-to-code conversion within a predefined deployment environment, and ML-Agent-Bench for testing autonomous agents in an end-to-end task execution within a Linux sandbox environment.
  arXiv  Detail & Related papers  (2023-11-16T12:03:21Z)
• Simultaneous Machine Translation with Large Language Models [51.470478122113356]
  We investigate the possibility of applying Large Language Models to SimulMT tasks. We conducted experiments using the textttLlama2-7b-chat model on nine different languages from the MUST-C dataset. The results show that LLM outperforms dedicated MT models in terms of BLEU and LAAL metrics.
  arXiv  Detail & Related papers  (2023-09-13T04:06:47Z)
• When Do Program-of-Thoughts Work for Reasoning? [51.2699797837818]
  We propose complexity-impacted reasoning score (CIRS) to measure correlation between code and reasoning abilities. Specifically, we use the abstract syntax tree to encode the structural information and calculate logical complexity. Code will be integrated into the EasyInstruct framework at https://github.com/zjunlp/EasyInstruct.
  arXiv  Detail & Related papers  (2023-08-29T17:22:39Z)
• Low-code LLM: Graphical User Interface over Large Language Models [115.08718239772107]
  This paper introduces a novel human-LLM interaction framework, Low-code LLM. It incorporates six types of simple low-code visual programming interactions to achieve more controllable and stable responses. We highlight three advantages of the low-code LLM: user-friendly interaction, controllable generation, and wide applicability.
  arXiv  Detail & Related papers  (2023-04-17T09:27:40Z)
• Neuro-Symbolic Causal Language Planning with Commonsense Prompting [67.06667162430118]
  Language planning aims to implement complex high-level goals by decomposition into simpler low-level steps. Previous methods require either manual exemplars or annotated programs to acquire such ability from large language models. This paper proposes Neuro-Symbolic Causal Language Planner (CLAP) that elicits procedural knowledge from the LLMs with commonsense-infused prompting.
  arXiv  Detail & Related papers  (2022-06-06T22:09:52Z)
• Technology Readiness Levels for AI & ML [79.22051549519989]
  Development of machine learning systems can be executed easily with modern tools, but the process is typically rushed and means-to-an-end. Engineering systems follow well-defined processes and testing standards to streamline development for high-quality, reliable results. We propose a proven systems engineering approach for machine learning development and deployment.
  arXiv  Detail & Related papers  (2020-06-21T17:14:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.