GPIoT: Tailoring Small Language Models for IoT Program Synthesis and Development

• URL: http://arxiv.org/abs/2503.00686v1
• Date: Sun, 02 Mar 2025 01:55:40 GMT
• Title: GPIoT: Tailoring Small Language Models for IoT Program Synthesis and Development
• Authors: Leming Shen, Qiang Yang, Xinyu Huang, Zijing Ma, Yuanqing Zheng,
• Abstract summary: GPIoT is a code generation system for IoT applications by fine-tuning locally deployable Small Language Models (SLMs)<n>We propose GPIoT, a code generation system for IoT applications by fine-tuning locally deployable Small Language Models (SLMs) on IoT-specialized datasets.
• Score: 15.109121724888382
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: Code Large Language Models (LLMs) enhance software development efficiency by automatically generating code and documentation in response to user requirements. However, code LLMs cannot synthesize specialized programs when tasked with IoT applications that require domain knowledge. While Retrieval-Augmented Generation (RAG) offers a promising solution by fetching relevant domain knowledge, it necessitates powerful cloud LLMs (e.g., GPT-4) to process user requirements and retrieved contents, which raises significant privacy concerns. This approach also suffers from unstable networks and prohibitive LLM query costs. Moreover, it is challenging to ensure the correctness and relevance of the fetched contents. To address these issues, we propose GPIoT, a code generation system for IoT applications by fine-tuning locally deployable Small Language Models (SLMs) on IoT-specialized datasets. SLMs have smaller model sizes, allowing efficient local deployment and execution to mitigate privacy concerns and network uncertainty. Furthermore, by fine-tuning the SLMs with our IoT-specialized datasets, the SLMs' ability to synthesize IoT-related programs can be substantially improved. To evaluate GPIoT's capability in synthesizing programs for IoT applications, we develop a benchmark, IoTBench. Extensive experiments and user trials demonstrate the effectiveness of GPIoT in generating IoT-specialized code, outperforming state-of-the-art code LLMs with an average task accuracy increment of 64.7% and significant improvements in user satisfaction.

Related papers

• LLM-Based Threat Detection and Prevention Framework for IoT Ecosystems [6.649910168731417]
  This paper presents a novel Large Language Model (LLM)-based framework for comprehensive threat detection and prevention in IoT environments. The system integrates lightweight LLMs fine-tuned on IoT-specific datasets for real-time anomaly detection and automated, context-aware mitigation strategies. Experimental results in simulated IoT environments demonstrate significant improvements in detection accuracy, response latency, and resource efficiency over traditional security methods.
  arXiv  Detail & Related papers  (2025-05-01T01:18:54Z)
• LLMs meet Federated Learning for Scalable and Secure IoT Management [6.649910168731417]
  Traditional centralized architectures struggle with latency, privacy concerns, and excessive resource consumption. This paper presents a novel Federated Learning-driven Large Language Model (FL-LLM) framework, designed to enhance IoT system intelligence.
  arXiv  Detail & Related papers  (2025-04-22T16:56:59Z)
• PLM: Efficient Peripheral Language Models Hardware-Co-Designed for Ubiquitous Computing [48.30406812516552]
  We introduce the PLM, a Peripheral Language Model, developed through a co-design process that jointly optimize model architecture and edge system constraints. PLM employs a Multi-head Latent Attention mechanism and employs the squared ReLU activation function to encourage sparsity, thereby reducing peak memory footprint. evaluation results demonstrate that PLM outperforms existing small language models trained on publicly available data.
  arXiv  Detail & Related papers  (2025-03-15T15:11:17Z)
• AutoIOT: LLM-Driven Automated Natural Language Programming for AIoT Applications [16.47929288038498]
  Large Language Models (LLMs) have profoundly transformed our lives, revolutionizing interactions with AI and lowering the barrier to AI usage. This paper introduces AutoIOT, an automated program generator for AIoT applications.
  arXiv  Detail & Related papers  (2025-03-07T11:40:52Z)
• ChatIoT: Large Language Model-based Security Assistant for Internet of Things with Retrieval-Augmented Generation [6.39666247062118]
  ChatIoT is a large language model (LLM)-based IoT security assistant designed to disseminate IoT security and threat intelligence.<n>We develop an end-to-end data processing toolkit to handle heterogeneous datasets.
  arXiv  Detail & Related papers  (2025-02-14T04:00:18Z)
• When IoT Meet LLMs: Applications and Challenges [0.5461938536945723]
  We show how Large Language Models (LLMs) can facilitate advanced decision making and contextual understanding in the Internet of Things (IoT)<n>This is the first comprehensive study covering IoT-LLM integration between edge, fog, and cloud systems.<n>We propose a novel system model for industrial IoT applications that leverages LLM-based collective intelligence to enable predictive maintenance and condition monitoring.
  arXiv  Detail & Related papers  (2024-11-20T23:44:51Z)
• Large Language Model as a Catalyst: A Paradigm Shift in Base Station Siting Optimization [62.16747639440893]
  Large language models (LLMs) and their associated technologies advance, particularly in the realms of prompt engineering and agent engineering. Our proposed framework incorporates retrieval-augmented generation (RAG) to enhance the system's ability to acquire domain-specific knowledge and generate solutions.
  arXiv  Detail & Related papers  (2024-08-07T08:43:32Z)
• IoT-LM: Large Multisensory Language Models for the Internet of Things [70.74131118309967]
  IoT ecosystem provides rich source of real-world modalities such as motion, thermal, geolocation, imaging, depth, sensors, and audio. Machine learning presents a rich opportunity to automatically process IoT data at scale. We introduce IoT-LM, an open-source large multisensory language model tailored for the IoT ecosystem.
  arXiv  Detail & Related papers  (2024-07-13T08:20:37Z)
• Efficient Prompting for LLM-based Generative Internet of Things [88.84327500311464]
  Large language models (LLMs) have demonstrated remarkable capacities on various tasks, and integrating the capacities of LLMs into the Internet of Things (IoT) applications has drawn much research attention recently. Due to security concerns, many institutions avoid accessing state-of-the-art commercial LLM services, requiring the deployment and utilization of open-source LLMs in a local network setting. We propose a LLM-based Generative IoT (GIoT) system deployed in the local network setting in this study.
  arXiv  Detail & Related papers  (2024-06-14T19:24:00Z)
• LLMind: Orchestrating AI and IoT with LLM for Complex Task Execution [18.816077341295628]
  We present LLMind, a task-oriented AI framework that enables effective collaboration among IoT devices. Inspired by the functional specialization theory of the brain, our framework integrates an LLM with domain-specific AI modules. Complex tasks, which may involve collaborations of multiple domain-specific AI modules and IoT devices, are executed through a control script.
  arXiv  Detail & Related papers  (2023-12-14T14:57:58Z)
• FederatedScope-LLM: A Comprehensive Package for Fine-tuning Large Language Models in Federated Learning [70.38817963253034]
  This paper first discusses these challenges of federated fine-tuning LLMs, and introduces our package FS-LLM as a main contribution. We provide comprehensive federated parameter-efficient fine-tuning algorithm implementations and versatile programming interfaces for future extension in FL scenarios. We conduct extensive experiments to validate the effectiveness of FS-LLM and benchmark advanced LLMs with state-of-the-art parameter-efficient fine-tuning algorithms in FL settings.
  arXiv  Detail & Related papers  (2023-09-01T09:40:36Z)
• Exploring Parameter-Efficient Fine-Tuning Techniques for Code Generation with Large Language Models [11.845239346943067]
  parameter-efficient fine-tuning (PEFT) is a promising approach to efficiently specialize large language models (LLMs) to task-specific data.<n>Our study highlights the potential for tuning larger LLMs and significant reductions in memory usage by combining PEFT with quantization.
  arXiv  Detail & Related papers  (2023-08-21T04:31:06Z)
• Optimizing Resource-Efficiency for Federated Edge Intelligence in IoT Networks [96.24723959137218]
  We study an edge intelligence-based IoT network in which a set of edge servers learn a shared model using federated learning (FL) We propose a novel framework, called federated edge intelligence (FEI), that allows edge servers to evaluate the required number of data samples according to the energy cost of the IoT network. We prove that our proposed algorithm does not cause any data leakage nor disclose any topological information of the IoT network.
  arXiv  Detail & Related papers  (2020-11-25T12:51:59Z)
• Machine learning and data analytics for the IoT [8.39035688352917]
  We review how IoT-generated data are processed for machine learning analysis. We propose a framework to enable IoT applications to adaptively learn from other IoT applications.
  arXiv  Detail & Related papers  (2020-06-30T07:38:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.