MultiIoT: Benchmarking Machine Learning for the Internet of Things

• URL: http://arxiv.org/abs/2311.06217v2
• Date: Thu, 4 Jul 2024 05:16:47 GMT
• Title: MultiIoT: Benchmarking Machine Learning for the Internet of Things
• Authors: Shentong Mo, Louis-Philippe Morency, Russ Salakhutdinov, Paul Pu Liang,
• Abstract summary: The next generation of machine learning systems must be adept at perceiving and interacting with the physical world. sensory data from motion, thermal, geolocation, depth, wireless signals, video, and audio are increasingly used to model the states of physical environments. Existing efforts are often specialized to a single sensory modality or prediction task. This paper proposes MultiIoT, the most expansive and unified IoT benchmark to date, encompassing over 1.15 million samples from 12 modalities and 8 real-world tasks.
• Score: 70.74131118309967
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The next generation of machine learning systems must be adept at perceiving and interacting with the physical world through a diverse array of sensory channels. Commonly referred to as the `Internet of Things (IoT)' ecosystem, sensory data from motion, thermal, geolocation, depth, wireless signals, video, and audio are increasingly used to model the states of physical environments and the humans inside them. Despite the potential for understanding human wellbeing, controlling physical devices, and interconnecting smart cities, the community has seen limited benchmarks for building machine learning systems for IoT. Existing efforts are often specialized to a single sensory modality or prediction task, which makes it difficult to study and train large-scale models across many IoT sensors and tasks. To accelerate the development of new machine learning technologies for IoT, this paper proposes MultiIoT, the most expansive and unified IoT benchmark to date, encompassing over 1.15 million samples from 12 modalities and 8 real-world tasks. MultiIoT introduces unique challenges involving (1) generalizable learning from many sensory modalities, (2) multimodal interactions across long temporal ranges, (3) extreme heterogeneity due to unique structure and noise topologies in real-world sensors, and (4) complexity during training and inference. We evaluate a comprehensive set of models on MultiIoT, including modality and task-specific methods, multisensory and multitask supervised models, and large multisensory foundation models. Our results highlight opportunities for ML to make a significant impact in IoT, but many challenges in scalable learning from heterogeneous, long-range, and imperfect sensory modalities still persist. We release all code and data to accelerate future research in machine learning for IoT.

Related papers

• IoT-LM: Large Multisensory Language Models for the Internet of Things [70.74131118309967]
  IoT ecosystem provides rich source of real-world modalities such as motion, thermal, geolocation, imaging, depth, sensors, and audio. Machine learning presents a rich opportunity to automatically process IoT data at scale. We introduce IoT-LM, an open-source large multisensory language model tailored for the IoT ecosystem.
  arXiv  Detail & Related papers  (2024-07-13T08:20:37Z)
• Foundations of Multisensory Artificial Intelligence [32.56967614091527]
  This thesis aims to advance the machine learning foundations of multisensory AI. In the first part, we present a theoretical framework formalizing how modalities interact with each other to give rise to new information for a task. In the second part, we study the design of practical multimodal foundation models that generalize over many modalities and tasks.
  arXiv  Detail & Related papers  (2024-04-29T14:45:28Z)
• Effective Intrusion Detection in Heterogeneous Internet-of-Things Networks via Ensemble Knowledge Distillation-based Federated Learning [52.6706505729803]
  We introduce Federated Learning (FL) to collaboratively train a decentralized shared model of Intrusion Detection Systems (IDS) FLEKD enables a more flexible aggregation method than conventional model fusion techniques. Experiment results show that the proposed approach outperforms local training and traditional FL in terms of both speed and performance.
  arXiv  Detail & Related papers  (2024-01-22T14:16:37Z)
• Random resistive memory-based deep extreme point learning machine for unified visual processing [67.51600474104171]
  We propose a novel hardware-software co-design, random resistive memory-based deep extreme point learning machine (DEPLM) Our co-design system achieves huge energy efficiency improvements and training cost reduction when compared to conventional systems.
  arXiv  Detail & Related papers  (2023-12-14T09:46:16Z)
• Towards Artificial General Intelligence (AGI) in the Internet of Things (IoT): Opportunities and Challenges [55.82853124625841]
  Artificial General Intelligence (AGI) possesses the capacity to comprehend, learn, and execute tasks with human cognitive abilities. This research embarks on an exploration of the opportunities and challenges towards achieving AGI in the context of the Internet of Things. The application spectrum for AGI-infused IoT is broad, encompassing domains ranging from smart grids, residential environments, manufacturing, and transportation to environmental monitoring, agriculture, healthcare, and education.
  arXiv  Detail & Related papers  (2023-09-14T05:43:36Z)
• Ensemble Learning based Anomaly Detection for IoT Cybersecurity via Bayesian Hyperparameters Sensitivity Analysis [2.3226893628361682]
  Internet of Things (IoT) integrates more than billions of intelligent devices over the globe with the capability of communicating with other connected devices. Data collected by IoT contain a tremendous amount of information for anomaly detection. In this paper, we present a study on using ensemble machine learning methods for enhancing IoT cybersecurity via anomaly detection.
  arXiv  Detail & Related papers  (2023-07-20T05:23:49Z)
• Graph Neural Networks in IoT: A Survey [9.257834364029547]
  The Internet of Things (IoT) boom has revolutionized almost every corner of people's daily lives. Deep learning models have been extensively employed in solving IoT tasks. Graph Neural Networks (GNNs) have been demonstrated to achieve state-of-the-art results in numerous IoT learning tasks.
  arXiv  Detail & Related papers  (2022-03-29T22:27:59Z)
• ThreeDWorld: A Platform for Interactive Multi-Modal Physical Simulation [75.0278287071591]
  ThreeDWorld (TDW) is a platform for interactive multi-modal physical simulation. TDW enables simulation of high-fidelity sensory data and physical interactions between mobile agents and objects in rich 3D environments. We present initial experiments enabled by TDW in emerging research directions in computer vision, machine learning, and cognitive science.
  arXiv  Detail & Related papers  (2020-07-09T17:33:27Z)
• Personalized Federated Learning for Intelligent IoT Applications: A Cloud-Edge based Framework [12.199870302894439]
  Internet of Things (IoT) have widely penetrated in different aspects of modern life. In this article we advocate a personalized federated learning framework in a cloud-edge architecture for intelligent IoT applications.
  arXiv  Detail & Related papers  (2020-02-25T05:11:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.