DynaMark: A Reinforcement Learning Framework for Dynamic Watermarking in Industrial Machine Tool Controllers

• URL: http://arxiv.org/abs/2508.21797v1
• Date: Fri, 29 Aug 2025 17:24:00 GMT
• Title: DynaMark: A Reinforcement Learning Framework for Dynamic Watermarking in Industrial Machine Tool Controllers
• Abstract summary: DynaMark is a reinforcement learning framework that models dynamic watermarking as a Markov decision process (MDP)<n>It maximizes a unique reward function balancing control performance, energy consumption, and detection confidence dynamically.<n>On a Siemens Sinumerik 828D controller digital twin, DynaMark achieves a reduction in watermark energy by 70% while preserving the nominal trajectory.
• Score: 2.440379640936221
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Industry 4.0's highly networked Machine Tool Controllers (MTCs) are prime targets for replay attacks that use outdated sensor data to manipulate actuators. Dynamic watermarking can reveal such tampering, but current schemes assume linear-Gaussian dynamics and use constant watermark statistics, making them vulnerable to the time-varying, partly proprietary behavior of MTCs. We close this gap with DynaMark, a reinforcement learning framework that models dynamic watermarking as a Markov decision process (MDP). It learns an adaptive policy online that dynamically adapts the covariance of a zero-mean Gaussian watermark using available measurements and detector feedback, without needing system knowledge. DynaMark maximizes a unique reward function balancing control performance, energy consumption, and detection confidence dynamically. We develop a Bayesian belief updating mechanism for real-time detection confidence in linear systems. This approach, independent of specific system assumptions, underpins the MDP for systems with linear dynamics. On a Siemens Sinumerik 828D controller digital twin, DynaMark achieves a reduction in watermark energy by 70% while preserving the nominal trajectory, compared to constant variance baselines. It also maintains an average detection delay equivalent to one sampling interval. A physical stepper-motor testbed validates these findings, rapidly triggering alarms with less control performance decline and exceeding existing benchmarks.

Related papers

• Performance Drift Detection in Machine Learning as a Service (MLaaS) for IoT Environments [0.06157382820537719]
  We propose a novel ML Performance Drift Detection framework for IoT environments.<n>The framework first employs an ML extraction model that learns service behavior from input-output pairs.<n>We further design an Adaptive Performance Drift Detection Mechanism that adjusts monitoring frequency.
  arXiv  Detail & Related papers  (2026-08-19T05:24:44Z)
• A Continual Validation, Updating, and Decision-Making Framework for Self-Adaptive Digital Twins via Robust Model Predictive Control: A Case Study in Additive Manufacturing [11.700555168494917]
  Digital Twins rely on surrogate models to mirror physical systems in real time, yet these models can degrade as operating conditions evolve.<n>Existing adaptive frameworks lack principled mechanisms for detecting when updates are needed.<n>We present an adaptive Digital Twin framework that integrates a Fisher score--based multivariate drift detector, Low-Rank Adaptation (LoRA) for parameter-efficient continual learning, and a Mann--Whitney $U$ test for online statistical validation.
  arXiv  Detail & Related papers  (2026-07-20T17:07:41Z)
• ASTRO: Adaptive Spatio-Temporal Reinforcement Optimization for GNN Powered Anomly Detection in Cyber Physical Systems [0.0]
  This article introduces Adaptive Spatio-Temporal Reinforcement Optimization ASTRO (ASTRO), a novel anomaly detection framework.<n>By integrating a Deep Q-Network (DQN) with Graph Neural Networks (GNNs), temporal modelling and a Multi-Head Attention mechanism, ASTRO continuously adapts its decision boundaries to improve detection accuracy.<n>The ASTRO approach is evaluated on two real world industrial benchmarks: the Secure Water Treatment (SWaT) and Water Distribution (WADI) datasets.
  arXiv  Detail & Related papers  (2026-05-24T15:30:14Z)
• When Sensors Fail: Temporal Sequence Models for Robust PPO under Sensor Drift [64.37959940809633]
  We study robustness of Proximal Policy Optimization (PPO) under temporally persistent sensor failures.<n>We show Transformer-based sequence policies substantially outperform, RNN, and SSMs in robustness, maintaining high returns even when large fractions of sensors are unavailable.
  arXiv  Detail & Related papers  (2026-03-04T22:21:54Z)
• Distilling the Thought, Watermarking the Answer: A Principle Semantic Guided Watermark for Large Reasoning Models [46.12198035083885]
  This paper introduces ReasonMark, a novel watermarking framework specifically designed for reasoning-intensive LLMs.<n>Our approach decouples generation into an undisturbed Thinking Phase and a watermarked Answering Phase.<n>Experiments show ReasonMark surpasses state-of-the-art methods by reducing text Perplexity by 0.35, increasing translation BLEU score by 0.164, and raising mathematical accuracy by 0.67 points.
  arXiv  Detail & Related papers  (2026-01-08T17:32:22Z)
• An Efficient Anomaly Detection Framework for Wireless Sensor Networks Using Markov Process [2.5777932046298786]
  A lightweight and interpretable anomaly detection framework based on a first order Markov chain model has been proposed.<n>The proposed framework was validated using the Intel Berkeley Research Lab dataset.
  arXiv  Detail & Related papers  (2025-11-01T10:19:00Z)
• An Ensemble Framework for Unbiased Language Model Watermarking [60.99969104552168]
  We propose ENS, a novel ensemble framework that enhances the detectability and robustness of unbiased watermarks.<n>ENS sequentially composes multiple independent watermark instances, each governed by a distinct key, to amplify the watermark signal.<n> Empirical evaluations show that ENS substantially reduces the number of tokens needed for reliable detection and increases resistance to smoothing and paraphrasing attacks.
  arXiv  Detail & Related papers  (2025-09-28T19:37:44Z)
• PMark: Towards Robust and Distortion-free Semantic-level Watermarking with Channel Constraints [49.2373408329323]
  We introduce a new theoretical framework on watermark-leveling (SWM) for large language models (LLMs)<n>We propose PMark, a simple yet powerful SWM method that estimates the median next sentence dynamically through sampling channels.<n> Experimental results show that PMark consistently outperforms existing SWM baselines in both text quality and paraphrasing.
  arXiv  Detail & Related papers  (2025-09-25T12:08:31Z)
• ViSTR-GP: Online Cyberattack Detection via Vision-to-State Tensor Regression and Gaussian Processes in Automated Robotic Operations [5.95097350945477]
  Connected and automated factories face growing cybersecurity risks that can potentially cause interruptions and damages to physical operations.<n>Data-integrity attacks often involve sophisticated exploitation of vulnerabilities that enable an attacker to access and manipulate the operational data.<n>This paper develops an online detection framework, ViSTR-GP, that cross-checks encoder-reported measurements against a vision-based estimate from an overhead camera outside the controller's authority.
  arXiv  Detail & Related papers  (2025-09-13T19:10:35Z)
• AI-Powered Machine Learning Approaches for Fault Diagnosis in Industrial Pumps [0.0]
  This study presents a practical approach for early fault detection in industrial pump systems using real-world sensor data.<n>The framework is scalable, interpretable, and suitable for real-time industrial deployment.
  arXiv  Detail & Related papers  (2025-08-21T13:33:09Z)
• Improving Real-Time Concept Drift Detection using a Hybrid Transformer-Autoencoder Framework [0.0]
  In applied machine learning, concept drift can significantly reduce model performance.<n>Our study proposes a hybrid framework consisting of Transformers and Autoencoders to model complex temporal dynamics.<n>Our results support that the Transformation-Autoencoder detected drift earlier and with more sensitivity than the autoencoders commonly used in the literature.
  arXiv  Detail & Related papers  (2025-08-09T19:39:33Z)
• Machine Learning-Based Cyberattack Detection and Identification for Automatic Generation Control Systems Considering Nonlinearities [0.6144680854063939]
  AGC systems' reliance on communicated measurements exposes them to false data injection attacks (FDIAs)<n>This paper proposes a machine learning (ML)-based detection framework that identifies FDIAs and determines the compromised measurements.<n>Our results demonstrate the efficacy of the proposed method in detecting FDIAs while maintaining a low false alarm rate, with an F1-score of up to 99.98%, outperforming existing approaches.
  arXiv  Detail & Related papers  (2025-04-12T23:06:59Z)
• Lazy Layers to Make Fine-Tuned Diffusion Models More Traceable [70.77600345240867]
  A novel arbitrary-in-arbitrary-out (AIAO) strategy makes watermarks resilient to fine-tuning-based removal. Unlike the existing methods of designing a backdoor for the input/output space of diffusion models, in our method, we propose to embed the backdoor into the feature space of sampled subpaths. Our empirical studies on the MS-COCO, AFHQ, LSUN, CUB-200, and DreamBooth datasets confirm the robustness of AIAO.
  arXiv  Detail & Related papers  (2024-05-01T12:03:39Z)
• Wavelet Dynamic Selection Network for Inertial Sensor Signal Enhancement [11.793803540713695]
  Inertial sensors are widely used in various portable devices. Wavelet dynamic selection network (WDSNet) intelligently selects appropriate wavelet basis for variable inertial signals. WDSNet, as a weakly-supervised method, achieves the state-of-the-art performance of all the compared fully-supervised methods.
  arXiv  Detail & Related papers  (2023-12-29T07:44:06Z)
• Formal Controller Synthesis for Markov Jump Linear Systems with Uncertain Dynamics [64.72260320446158]
  We propose a method for synthesising controllers for Markov jump linear systems. Our method is based on a finite-state abstraction that captures both the discrete (mode-jumping) and continuous (stochastic linear) behaviour of the MJLS. We apply our method to multiple realistic benchmark problems, in particular, a temperature control and an aerial vehicle delivery problem.
  arXiv  Detail & Related papers  (2022-12-01T17:36:30Z)
• Incremental learning of LSTM framework for sensor fusion in attitude estimation [2.064612766965483]
  This paper presents a novel method for attitude estimation of an object in 3D space by incremental learning of the Long-Short Term Memory (LSTM) network. Inertial sensors data are fed to the LSTM network which are then updated incrementally to incorporate the dynamic changes in motion occurring in the run time. The proposed framework offers a significant improvement in the results compared to the traditional method, even in the case of a highly dynamic environment.
  arXiv  Detail & Related papers  (2021-08-04T09:03:53Z)
• Deep Soft Procrustes for Markerless Volumetric Sensor Alignment [81.13055566952221]
  In this work, we improve markerless data-driven correspondence estimation to achieve more robust multi-sensor spatial alignment. We incorporate geometric constraints in an end-to-end manner into a typical segmentation based model and bridge the intermediate dense classification task with the targeted pose estimation one. Our model is experimentally shown to achieve similar results with marker-based methods and outperform the markerless ones, while also being robust to the pose variations of the calibration structure.
  arXiv  Detail & Related papers  (2020-03-23T10:51:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.