Characterizing Tests in IoT Software: Practices, Challenges and Opportunities

• URL: http://arxiv.org/abs/2606.12592v1
• Date: Wed, 10 Jun 2026 18:39:13 GMT
• Title: Characterizing Tests in IoT Software: Practices, Challenges and Opportunities
• Abstract summary: The Internet of Things (IoT) is experiencing rapid growth.<n>Bugs in IoT software can lead to severe consequences.<n>Little is known about how IoT software is tested.
• Score: 4.966939549153776
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: The Internet of Things (IoT) is experiencing rapid growth. Smart devices are emerging in smart homes and industrial applications, performing mission-critical tasks. Bugs in IoT software can lead to severe consequences. For example, a buggy smart lock can allow unauthorized access to a private property. Testing is a primary practice to expose software bugs and ensure software quality. However, little is known about how IoT software is tested. To bridge this gap, we conducted the first empirical study on test cases in open-source IoT software. Specifically, we evaluated the effectiveness of test cases in IoT software, explored the challenges inherent in testing IoT software, and analyzed the usage of mock objects. Our results indicate that while IoT software often contains a considerable number of tests, their effectiveness remains limited. We identified the primary challenges in testing IoT software as managing complex interactions with various external dependencies, such as other network-reliant IoT components, file systems, operating systems, and databases. We also observed that the use of mock objects in IoT software closely aligns with our identified testing challenges. This alignment demonstrates the potential of mocking as a solution to enhance test coverage and address the complexities of IoT software testing.

Related papers

• WebTestBench: Evaluating Computer-Use Agents towards End-to-End Automated Web Testing [57.7131457251794]
  We introduce WebTestBench, a benchmark for evaluating end-to-end automated web testing.<n>We decompose the testing process into two cascaded sub-tasks, checklist generation and defect detection, and propose WebTester.<n>Our findings expose a substantial gap between current computer-use agent capabilities and industrial-grade deployment demands.
  arXiv  Detail & Related papers  (2026-03-26T09:27:29Z)
• Intelligent Detection of Non-Essential IoT Traffic on the Home Gateway [45.70482328441101]
  This work presents ML-IoTrim, a system for detecting and mitigating non-essential IoT traffic by analyzing network behavior at the edge.<n>We test our framework in a consumer smart home setup with IoT devices from five categories, demonstrating that the model can accurately identify and block non-essential traffic.<n>This research advances privacy-aware traffic control in smart homes, paving the way for future developments in IoT device privacy.
  arXiv  Detail & Related papers  (2025-04-22T09:40:05Z)
• Testing Practices, Challenges, and Developer Perspectives in Open-Source IoT Platforms [12.319289129092011]
  This paper is the first to uncover both the practices and perceptions behind testing in IoT platforms.<n>We analyze the code and integrations of two popular and well-maintained open-source IoT platforms, OpenHab and HomeAssitant.<n>Our analysis reveals a significant focus on automated (unit) testing, and a lack of manual testing.
  arXiv  Detail & Related papers  (2025-02-11T04:52:27Z)
• Comparison of Code Quality and Best Practices in IoT and non-IoT Software [3.0711566483997075]
  We compare the code quality of two equivalent sets of IoT and non-IoT systems. We then select and present a list of best practices to address the observed difference between IoT and non-IoT code.
  arXiv  Detail & Related papers  (2024-08-05T16:39:04Z)
• IoT-LM: Large Multisensory Language Models for the Internet of Things [70.74131118309967]
  IoT ecosystem provides rich source of real-world modalities such as motion, thermal, geolocation, imaging, depth, sensors, and audio. Machine learning presents a rich opportunity to automatically process IoT data at scale. We introduce IoT-LM, an open-source large multisensory language model tailored for the IoT ecosystem.
  arXiv  Detail & Related papers  (2024-07-13T08:20:37Z)
• A Lean Simulation Framework for Stress Testing IoT Cloud Systems [16.004920480830858]
  The Internet of Things connects a plethora of smart devices globally across various applications like smart cities, autonomous vehicles and health monitoring. Simulation plays a key role in the testing of IoT systems, noting that field testing of a complete IoT product may be infeasible or expensive. Existing stress testing solutions for IoT demand significant computational resources, making them ill-suited and costly. We propose a lean simulation framework designed for IoT cloud stress testing which enables efficient simulation of a large array of IoT and edge devices that communicate with the cloud.
  arXiv  Detail & Related papers  (2024-04-17T16:36:55Z)
• Effective Intrusion Detection in Heterogeneous Internet-of-Things Networks via Ensemble Knowledge Distillation-based Federated Learning [52.6706505729803]
  We introduce Federated Learning (FL) to collaboratively train a decentralized shared model of Intrusion Detection Systems (IDS) FLEKD enables a more flexible aggregation method than conventional model fusion techniques. Experiment results show that the proposed approach outperforms local training and traditional FL in terms of both speed and performance.
  arXiv  Detail & Related papers  (2024-01-22T14:16:37Z)
• MultiIoT: Benchmarking Machine Learning for the Internet of Things [70.74131118309967]
  The next generation of machine learning systems must be adept at perceiving and interacting with the physical world. sensory data from motion, thermal, geolocation, depth, wireless signals, video, and audio are increasingly used to model the states of physical environments. Existing efforts are often specialized to a single sensory modality or prediction task. This paper proposes MultiIoT, the most expansive and unified IoT benchmark to date, encompassing over 1.15 million samples from 12 modalities and 8 real-world tasks.
  arXiv  Detail & Related papers  (2023-11-10T18:13:08Z)
• IoTScent: Enhancing Forensic Capabilities in Internet of Things Gateways [45.44831696628473]
  This paper presents IoTScent, an open-source forensic tool that enables IoT gateways and Home Automation platforms to perform IoT traffic capture and analysis. IoTScent is specifically designed to operate over IEEE5.4-based traffic, which is the basis for many IoT-specific protocols such as Zigbee, 6LoWPAN and Thread. This work provides a comprehensive description of the IoTScent tool, including a practical use case that demonstrates the use of the tool to perform device identification from Zigbee traffic.
  arXiv  Detail & Related papers  (2023-10-05T09:10:05Z)
• Optimizing Resource-Efficiency for Federated Edge Intelligence in IoT Networks [96.24723959137218]
  We study an edge intelligence-based IoT network in which a set of edge servers learn a shared model using federated learning (FL) We propose a novel framework, called federated edge intelligence (FEI), that allows edge servers to evaluate the required number of data samples according to the energy cost of the IoT network. We prove that our proposed algorithm does not cause any data leakage nor disclose any topological information of the IoT network.
  arXiv  Detail & Related papers  (2020-11-25T12:51:59Z)
• Machine learning and data analytics for the IoT [8.39035688352917]
  We review how IoT-generated data are processed for machine learning analysis. We propose a framework to enable IoT applications to adaptively learn from other IoT applications.
  arXiv  Detail & Related papers  (2020-06-30T07:38:31Z)
• Personalized Federated Learning for Intelligent IoT Applications: A Cloud-Edge based Framework [12.199870302894439]
  Internet of Things (IoT) have widely penetrated in different aspects of modern life. In this article we advocate a personalized federated learning framework in a cloud-edge architecture for intelligent IoT applications.
  arXiv  Detail & Related papers  (2020-02-25T05:11:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.