MultiCore+TPU Accelerated Multi-Modal TinyML for Livestock Behaviour Recognition

• URL: http://arxiv.org/abs/2504.11467v2
• Date: Sat, 19 Apr 2025 02:53:51 GMT
• Title: MultiCore+TPU Accelerated Multi-Modal TinyML for Livestock Behaviour Recognition
• Abstract summary: In recent years, more intelligent livestock monitoring solutions have been proposed to enhance farming efficiency and productivity.<n>This work presents a novel approach to animal activity recognition and movement tracking, leveraging tiny machine learning (TinyML) techniques.<n>It collects and fuses accelerometer data and vision inputs to build a multi-modal network for three tasks: image classification, object detection, and behaviour recognition.
• Score: 0.5524804393257919
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The advancement of technology has revolutionised the agricultural industry, transitioning it from labour-intensive farming practices to automated, AI-powered management systems. In recent years, more intelligent livestock monitoring solutions have been proposed to enhance farming efficiency and productivity. This work presents a novel approach to animal activity recognition and movement tracking, leveraging tiny machine learning (TinyML) techniques, wireless communication framework, and microcontroller platforms to develop an efficient, cost-effective livestock sensing system. It collects and fuses accelerometer data and vision inputs to build a multi-modal network for three tasks: image classification, object detection, and behaviour recognition. The system is deployed and evaluated on commercial microcontrollers for real-time inference using embedded applications, demonstrating up to 270$\times$ model size reduction, less than 80ms response latency, and on-par performance comparable to existing methods. The incorporation of the TinyML technique allows for seamless data transmission between devices, benefiting use cases in remote locations with poor Internet connectivity. This work delivers a robust, scalable IoT-edge livestock monitoring solution adaptable to diverse farming needs, offering flexibility for future extensions.

Related papers

• Design and Evaluation of an AI-Enabled Cloud-Edge Architecture for Connected Precision Agriculture Farms [0.28312565521926425]
  Plant diseases cause significant yield losses worldwide, with tomato crops particularly susceptible to early blight, late blight, and leaf mold.<n>To tackle this limitation, an artificial intelligence (AI) enabled cloud-edge architecture is proposed for autonomous crop monitoring.<n>This proposed architecture integrates Internet of Things (IoT) sensors, unmanned aerial vehicles (UAVs), deep learning, Azure IoT Hub-based cloud analytics, and multi-platform interfaces.
  arXiv  Detail & Related papers  (2026-08-04T15:25:02Z)
• Leveraging Large Vision Model for Multi-UAV Co-perception in Low-Altitude Wireless Networks [51.323657802729535]
  Multi-uncrewed aerial vehicles (UAV) cooperative perception has emerged as a promising paradigm for diverse low-altitude economy applications.<n>This paper proposes a communication-efficient cooperative perception framework, termed Base-Station-Helped UAV (BHU)<n>We employ a Top-K selection mechanism to identify the most informative pixels from UAV-captured RGB images, enabling sparsified visual transmission with reduced data volume and latency.
  arXiv  Detail & Related papers  (2026-03-13T17:49:28Z)
• MOTION: ML-Assisted On-Device Low-Latency Motion Recognition [5.0385144315892925]
  We use WeBe Band, a multi-sensor wearable device equipped with a powerful enough MCU to effectively perform gesture recognition entirely on the device.<n>We found that the neural network provided the best balance between accuracy, latency, and memory use.<n>Our results also demonstrate that reliable real-time gesture recognition can be achieved in WeBe Band, with great potential for real-time medical monitoring solutions.
  arXiv  Detail & Related papers  (2025-10-14T01:15:47Z)
• Dexplore: Scalable Neural Control for Dexterous Manipulation from Reference-Scoped Exploration [58.4036440289082]
  Hand-object motion-capture (MoCap) offer large-scale, contact-rich demonstrations and hold promise for dexterous robotic scopes.<n>We introduce Dexplore, a unified single-loop optimization that performs repositories and tracking to learn robot control policies directly from MoCap at scale.
  arXiv  Detail & Related papers  (2025-09-11T17:59:07Z)
• D-CAT: Decoupled Cross-Attention Transfer between Sensor Modalities for Unimodal Inference [3.6344649347926326]
  Cross-modal transfer learning is used to improve multi-modal classification models.<n>Existing methods require paired sensor data at both training and inference.<n>We propose Decoupled Cross-Attention Transfer (D-CAT), a framework that aligns modality-specific representations without requiring joint sensor modality during inference.
  arXiv  Detail & Related papers  (2025-09-11T10:54:07Z)
• An Explainable AI based approach for Monitoring Animal Health [0.2749898166276853]
  Monitoring cattle health and optimizing yield are key challenges faced by dairy farmers due to difficulties in tracking all animals on the farm.<n>This work aims to showcase modern data-driven farming practices based on explainable machine learning(ML) methods that explain the activity and behaviour of dairy cattle (cows)
  arXiv  Detail & Related papers  (2025-08-13T21:40:35Z)
• Multimodal AI Systems for Enhanced Laying Hen Welfare Assessment and Productivity Optimization [0.0]
  Future of poultry production depends on replacing subjective, labor-intensive welfare checks with data-driven, intelligent monitoring ecosystems.<n>Traditional welfare assessments-limited by human observation and single-sensor data-cannot fully capture the complex, multidimensional nature of laying hen welfare in modern farms.<n>This work lays the foundation for a transition from reactive, unimodal monitoring to proactive, precision-driven welfare systems that unite productivity with ethical, science based animal care.
  arXiv  Detail & Related papers  (2025-08-11T05:17:16Z)
• Gait Recognition Based on Tiny ML and IMU Sensors [1.2289361708127877]
  This project presents the development of a gait recognition system using Tiny Machine Learning (Tiny ML) and Inertial Measurement Unit (IMU) sensors.<n>The system leverages the XIAO-nRF52840 Sense microcontroller and the LSM6DS3 IMU sensor to capture motion data, including acceleration and angular velocity, from four distinct activities.<n>The data collected is processed through Edge Impulse, an edge AI platform, which enables the training of machine learning models that can be deployed directly onto the microcontroller for real-time activity classification.
  arXiv  Detail & Related papers  (2025-07-24T17:59:08Z)
• AI/ML Life Cycle Management for Interoperable AI Native RAN [50.61227317567369]
  Artificial intelligence (AI) and machine learning (ML) models are rapidly permeating the 5G Radio Access Network (RAN)<n>These developments lay the foundation for AI-native transceivers as a key enabler for 6G.
  arXiv  Detail & Related papers  (2025-07-24T16:04:59Z)
• CAML: Collaborative Auxiliary Modality Learning for Multi-Agent Systems [38.20651868834145]
  Collaborative Auxiliary Modality Learning ($textbfCAML$) is a novel multi-agent multi-modality framework.<n>It enables agents to collaborate and share multimodal data during training while allowing inference with reduced modalities per agent during testing.
  arXiv  Detail & Related papers  (2025-02-25T03:59:40Z)
• On-device Online Learning and Semantic Management of TinyML Systems [8.183732025472766]
  This study aims to bridge the gap between prototyping single TinyML models and developing reliable TinyML systems in production. We propose online learning to enable training on constrained devices, adapting local models towards the latest field conditions. We present semantic management for the joint management of models and devices at scale.
  arXiv  Detail & Related papers  (2024-05-13T10:03:34Z)
• GeRM: A Generalist Robotic Model with Mixture-of-experts for Quadruped Robot [27.410618312830497]
  In this paper, we propose GeRM (Generalist Robotic Model) We utilize offline reinforcement learning to optimize data utilization strategies. We employ a transformer-based VLA network to process multi-modal inputs and output actions.
  arXiv  Detail & Related papers  (2024-03-20T07:36:43Z)
• Effective Intrusion Detection in Heterogeneous Internet-of-Things Networks via Ensemble Knowledge Distillation-based Federated Learning [52.6706505729803]
  We introduce Federated Learning (FL) to collaboratively train a decentralized shared model of Intrusion Detection Systems (IDS) FLEKD enables a more flexible aggregation method than conventional model fusion techniques. Experiment results show that the proposed approach outperforms local training and traditional FL in terms of both speed and performance.
  arXiv  Detail & Related papers  (2024-01-22T14:16:37Z)
• Robot Fine-Tuning Made Easy: Pre-Training Rewards and Policies for Autonomous Real-World Reinforcement Learning [58.3994826169858]
  We introduce RoboFuME, a reset-free fine-tuning system for robotic reinforcement learning. Our insights are to utilize offline reinforcement learning techniques to ensure efficient online fine-tuning of a pre-trained policy. Our method can incorporate data from an existing robot dataset and improve on a target task within as little as 3 hours of autonomous real-world experience.
  arXiv  Detail & Related papers  (2023-10-23T17:50:08Z)
• Agile gesture recognition for capacitive sensing devices: adapting on-the-job [55.40855017016652]
  We demonstrate a hand gesture recognition system that uses signals from capacitive sensors embedded into the etee hand controller. The controller generates real-time signals from each of the wearer five fingers. We use a machine learning technique to analyse the time series signals and identify three features that can represent 5 fingers within 500 ms.
  arXiv  Detail & Related papers  (2023-05-12T17:24:02Z)
• Output Feedback Tube MPC-Guided Data Augmentation for Robust, Efficient Sensorimotor Policy Learning [49.05174527668836]
  Imitation learning (IL) can generate computationally efficient sensorimotor policies from demonstrations provided by computationally expensive model-based sensing and control algorithms. In this work, we combine IL with an output feedback robust tube model predictive controller to co-generate demonstrations and a data augmentation strategy to efficiently learn neural network-based sensorimotor policies. We numerically demonstrate that our method can learn a robust visuomotor policy from a single demonstration--a two-orders of magnitude improvement in demonstration efficiency compared to existing IL methods.
  arXiv  Detail & Related papers  (2022-10-18T19:59:17Z)
• Online Data Selection for Federated Learning with Limited Storage [53.46789303416799]
  Federated Learning (FL) has been proposed to achieve distributed machine learning among networked devices. The impact of on-device storage on the performance of FL is still not explored. In this work, we take the first step to consider the online data selection for FL with limited on-device storage.
  arXiv  Detail & Related papers  (2022-09-01T03:27:33Z)
• Self-Supervised Multimodal Fusion Transformer for Passive Activity Recognition [2.35066982314539]
  Wi-Fi signals provide significant opportunities for human sensing and activity recognition in fields such as healthcare. Current systems do not effectively exploit the information acquired through multiple sensors to recognise the different activities. We propose the Fusion Transformer, an attention-based model for multimodal and multi-sensor fusion.
  arXiv  Detail & Related papers  (2022-08-15T15:38:10Z)
• The Synergy of Complex Event Processing and Tiny Machine Learning in Industrial IoT [7.172671995820974]
  The Industrial Internet-of-Things (IIoT) facilitates efficiency and robustness in factory operations. Various sensors and field devices play a central role, as they generate a vast amount of real-time data that can provide insights into manufacturing. The synergy of complex event processing (CEP) and machine learning (ML) has been developed actively in the last years in IIoT to identify patterns in heterogeneous data streams and fuse raw data into tangible facts. This paper proposes a framework that exploits ML and CEP's synergy at the edge in distributed sensor networks.
  arXiv  Detail & Related papers  (2021-05-04T14:58:48Z)
• Domain Adaptive Robotic Gesture Recognition with Unsupervised Kinematic-Visual Data Alignment [60.31418655784291]
  We propose a novel unsupervised domain adaptation framework which can simultaneously transfer multi-modality knowledge, i.e., both kinematic and visual data, from simulator to real robot. It remedies the domain gap with enhanced transferable features by using temporal cues in videos, and inherent correlations in multi-modal towards recognizing gesture. Results show that our approach recovers the performance with great improvement gains, up to 12.91% in ACC and 20.16% in F1score without using any annotations in real robot.
  arXiv  Detail & Related papers  (2021-03-06T09:10:03Z)
• Moving Object Classification with a Sub-6 GHz Massive MIMO Array using Real Data [64.48836187884325]
  Classification between different activities in an indoor environment using wireless signals is an emerging technology for various applications. In this paper, we analyze classification of moving objects by employing machine learning on real data from a massive multi-input-multi-output (MIMO) system in an indoor environment.
  arXiv  Detail & Related papers  (2021-02-09T15:48:35Z)
• YOLOpeds: Efficient Real-Time Single-Shot Pedestrian Detection for Smart Camera Applications [2.588973722689844]
  This work addresses the challenge of achieving a good trade-off between accuracy and speed for efficient deployment of deep-learning-based pedestrian detection in smart camera applications. A computationally efficient architecture is introduced based on separable convolutions and proposes integrating dense connections across layers and multi-scale feature fusion. Overall, YOLOpeds provides real-time sustained operation of over 30 frames per second with detection rates in the range of 86% outperforming existing deep learning models.
  arXiv  Detail & Related papers  (2020-07-27T09:50:11Z)
• Machine learning approaches for identifying prey handling activity in otariid pinnipeds [12.814241588031685]
  This paper focuses on the identification of prey handling activity in seals. Data taken into consideration are streams of 3D accelerometers and depth sensors values collected by devices attached directly on seals. We propose an automatic model based on Machine Learning (ML) algorithms.
  arXiv  Detail & Related papers  (2020-02-10T15:30:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.