Flama: a Python framework for development and deployment of production-ready APIs, machine learning, and LLM services

• URL: http://arxiv.org/abs/2608.18733v1
• Date: Wed, 19 Aug 2026 09:37:00 GMT
• Title: Flama: a Python framework for development and deployment of production-ready APIs, machine learning, and LLM services
• Abstract summary: Flama is an open-source Python framework for developing and deploying production-ready web APIs, machine learning services, and large-model (LLM) applications.<n>It offers a type-driven, async-first programming model that unifies REST API development, predictive model serving, and generative AI inference in one architecture.
• Score: 0.9668407688201359
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present Flama, an open-source Python framework for developing and deploying production-ready web APIs, machine learning services, and large-language-model (LLM) applications. Built on the Asynchronous Server Gateway Interface (ASGI), Flama offers a type-driven, async-first programming model that unifies REST API development, predictive model serving, and generative AI inference in one architecture. It is organised around seven subsystems: a component-based dependency injection system resolving handler parameters from type annotations at startup; a pluggable schema layer supporting Pydantic, Marshmallow and Typesystem behind a single adapter; an automatic CRUD generator turning a SQLAlchemy table and a schema class into REST endpoints backed by the Repository and Unit of Work patterns; a portable binary format (.flm) packaging models from scikit-learn, TensorFlow, PyTorch and Hugging Face Transformers with their metadata for zero-code deployment; a multi-backend LLM server running vLLM (Linux/CUDA) or MLX (Apple Silicon) and exposing four wire protocols (OpenAI, Anthropic, Ollama, and a native streaming dialect) through a shared codec; a Rust-accelerated core compiled via Maturin for routing, JSON encoding, compression and parsing; and a Model Context Protocol module turning any application into an MCP server over JSON-RPC 2.0. Built-in capabilities include JWT authentication, two pagination strategies, background tasks in threads or processes, WebSocket endpoints, Server-Sent Event and NDJSON streaming, OpenAPI 3.2.0 generation from handler signatures, and a command-line interface for running applications and for serving, packaging and inspecting models. We describe the architecture, present the programming model through worked examples, and compare Flama with existing frameworks, model serving platforms and LLM inference engines.

Related papers

• NVIDIA-labs OO Agents: Native Python Object-Oriented Agents [8.777370925453148]
  We present NVIDIA Object-Oriented Agents (NOOA), a model-agnostic Python framework for building reliable AI agents.<n>NOOA takes a simpler approach: an agent is a Python object. Its methods are the actions the model can take, fields are its state, docstrings are its prompts, and its type annotations are contracts.
  arXiv  Detail & Related papers  (2026-07-22T20:25:55Z)
• Environment-free Synthetic Data Generation for API-Calling Agents [68.95520739708304]
  Training API-calling large language model (LLM) agents demands massive amounts of high-quality trajectories.<n>We propose an environment-free synthetic data generation approach that leverages LLMs as on-the-fly digital world models.<n>We evaluate our approach on the challenging AppWorld and OfficeBench benchmarks, which include both information-retrieval and state-changing tasks.
  arXiv  Detail & Related papers  (2026-07-18T17:36:44Z)
• minWM: A Full-Stack Open-Source Framework for Real-Time Interactive Video World Models [51.45338589543413]
  minWM is a full-stack open-source framework for building real-time interactive video world models.<n> minWM converts existing bidirectional T2V/TI2V video foundation models into camera-controllable few-step autoregressive world models.
  arXiv  Detail & Related papers  (2026-05-28T17:27:03Z)
• HarnessAPI: A Skill-First Framework for Unified Streaming APIs and MCP Tools [0.0]
  We present HarnessAPI, a Python framework that eliminates this duplication.<n>From one handler.py plus Pydantic schemas, the framework automatically derives a streaming HTTP endpoint.<n>It reduces framework-facing validation by 74% compared with a manually maintained dual-stack implementation.
  arXiv  Detail & Related papers  (2026-05-21T17:03:44Z)
• Apple Intelligence Foundation Language Models: Tech Report 2025 [230.86919268176976]
  We introduce two foundation language models that power Apple Intelligence features across Apple devices and services.<n>Both models are trained on large-scale multilingual and multimodal datasets sourced via responsible web crawling.<n>A new Swift-centric Foundation Models framework exposes guided generation, constrained tool calling, and LoRA adapter fine-tuning.
  arXiv  Detail & Related papers  (2025-07-17T23:37:19Z)
• Two are better than one: Context window extension with multi-grained self-injection [111.1376461868317]
  SharedLLM is a novel approach grounded in the design philosophy of multi-grained context compression and query-aware information retrieval. We introduce a specialized tree-style data structure to efficiently encode, store and retrieve multi-grained contextual information for text chunks.
  arXiv  Detail & Related papers  (2024-10-25T06:08:59Z)
• DistML.js: Installation-free Distributed Deep Learning Framework for Web Browsers [40.48978035180545]
  "DistML.js" is a library designed for training and inference of machine learning models within web browsers. We provide a comprehensive explanation of DistML.js's design, API, and implementation, alongside practical applications.
  arXiv  Detail & Related papers  (2024-07-01T07:13:14Z)
• KerasCV and KerasNLP: Vision and Language Power-Ups [9.395199188271254]
  KerasCV and KerasNLP are extensions of the Keras API for Computer Vision and Natural Language Processing. These domain packages are designed to enable fast experimentation, with a focus on ease-of-use and performance. The libraries are fully open-source (Apache 2.0 license) and available on GitHub.
  arXiv  Detail & Related papers  (2024-05-30T16:58:34Z)
• ModelScope-Agent: Building Your Customizable Agent System with Open-source Large Language Models [74.64651681052628]
  We introduce ModelScope-Agent, a customizable agent framework for real-world applications based on open-source LLMs as controllers. It provides a user-friendly system library, with customizable engine design to support model training on multiple open-source LLMs. A comprehensive framework has been proposed spanning over tool-use data collection, tool retrieval, tool registration, memory control, customized model training, and evaluation.
  arXiv  Detail & Related papers  (2023-09-02T16:50:30Z)
• OSFormer: One-Stage Camouflaged Instance Segmentation with Transformers [79.09486658392626]
  OSFormer is a one-stage transformer framework for camouflaged instance segmentation (CIS) We design a location-sensing transformer (LST) to obtain the location label and instance-aware parameters. We develop a coarse-to-fine fusion (CFF) to merge diverse context information from the LST encoder and CNN backbone.
  arXiv  Detail & Related papers  (2022-07-05T18:32:21Z)
• CodeReef: an open platform for portable MLOps, reusable automation actions and reproducible benchmarking [0.2148535041822524]
  We present CodeReef - an open platform to share all the components necessary to enable cross-platform MLOps (MLSysOps) We also introduce the CodeReef solution - a way to package and share models as non-virtualized, portable, customizable archive files.
  arXiv  Detail & Related papers  (2020-01-22T09:52:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.