A Proactive Management Scheme for Data Synopses at the Edge

• URL: http://arxiv.org/abs/2107.10558v1
• Date: Thu, 22 Jul 2021 10:22:37 GMT
• Title: A Proactive Management Scheme for Data Synopses at the Edge
• Authors: Kostas Kolomvatsos, Christos Anagnostopoulos
• Abstract summary: The Internet of Things (IoT) with numerous processing nodes present at the Edge Computing ecosystem opens up new pathways to support intelligent applications. Such applications can be provided upon humongous volumes of data collected by IoT devices being transferred to the edge nodes through the network. Various processing activities can be performed on the discussed data and multiple collaborative opportunities between EC nodes can facilitate the execution of the desired tasks. In this paper, we recommend the exchange of data synopses than real data between EC nodes to provide them with the necessary knowledge about peer nodes owning similar data.
• Score: 20.711789781518753
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The combination of the infrastructure provided by the Internet of Things (IoT) with numerous processing nodes present at the Edge Computing (EC) ecosystem opens up new pathways to support intelligent applications. Such applications can be provided upon humongous volumes of data collected by IoT devices being transferred to the edge nodes through the network. Various processing activities can be performed on the discussed data and multiple collaborative opportunities between EC nodes can facilitate the execution of the desired tasks. In order to support an effective interaction between edge nodes, the knowledge about the geographically distributed data should be shared. Obviously, the migration of large amounts of data will harm the stability of the network stability and its performance. In this paper, we recommend the exchange of data synopses than real data between EC nodes to provide them with the necessary knowledge about peer nodes owning similar data. This knowledge can be valuable when considering decisions such as data/service migration and tasks offloading. We describe an continuous reasoning model that builds a temporal similarity map of the available datasets to get nodes understanding the evolution of data in their peers. We support the proposed decision making mechanism through an intelligent similarity extraction scheme based on an unsupervised machine learning model, and, at the same time, combine it with a statistical measure that represents the trend of the so-called discrepancy quantum. Our model can reveal the differences in the exchanged synopses and provide a datasets similarity map which becomes the appropriate knowledge base to support the desired processing activities. We present the problem under consideration and suggest a solution for that, while, at the same time, we reveal its advantages and disadvantages through a large number of experiments.

Related papers

• An Ensemble Scheme for Proactive Dominant Data Migration of Pervasive Tasks at the Edge [5.4327243200369555]
  We propose a scheme to be implemented by autonomous edge nodes concerning their identifications of the appropriate data to be migrated to particular locations within the infrastructure. Our objective is to equip nodes with the capability to comprehend the access patterns relating to offloaded data-driven tasks. It is evident that these tasks depend on the processing of data that is absent from the original hosting nodes. To infer these data intervals, we utilize an ensemble approach that integrates a statistically oriented model and a machine learning framework.
  arXiv  Detail & Related papers  (2024-10-12T19:09:16Z)
• How Much Data are Enough? Investigating Dataset Requirements for Patch-Based Brain MRI Segmentation Tasks [74.21484375019334]
  Training deep neural networks reliably requires access to large-scale datasets. To mitigate both the time and financial costs associated with model development, a clear understanding of the amount of data required to train a satisfactory model is crucial. This paper proposes a strategic framework for estimating the amount of annotated data required to train patch-based segmentation networks.
  arXiv  Detail & Related papers  (2024-04-04T13:55:06Z)
• Collaborative Mean Estimation over Intermittently Connected Networks with Peer-To-Peer Privacy [86.61829236732744]
  This work considers the problem of Distributed Mean Estimation (DME) over networks with intermittent connectivity. The goal is to learn a global statistic over the data samples localized across distributed nodes with the help of a central server. We study the tradeoff between collaborative relaying and privacy leakage due to the additional data sharing among nodes.
  arXiv  Detail & Related papers  (2023-02-28T19:17:03Z)
• Semantic Segmentation of Vegetation in Remote Sensing Imagery Using Deep Learning [77.34726150561087]
  We propose an approach for creating a multi-modal and large-temporal dataset comprised of publicly available Remote Sensing data. We use Convolutional Neural Networks (CNN) models that are capable of separating different classes of vegetation.
  arXiv  Detail & Related papers  (2022-09-28T18:51:59Z)
• Two-Bit Aggregation for Communication Efficient and Differentially Private Federated Learning [79.66767935077925]
  In federated learning (FL), a machine learning model is trained on multiple nodes in a decentralized manner, while keeping the data local and not shared with other nodes. The information sent from the nodes to the server may reveal some details about each node's local data, thus raising privacy concerns. A novel two-bit aggregation algorithm is proposed with guaranteed differential privacy and reduced uplink communication overhead.
  arXiv  Detail & Related papers  (2021-10-06T19:03:58Z)
• A communication efficient distributed learning framework for smart environments [0.4898659895355355]
  This paper proposes a distributed learning framework to move data analytics closer to where data is generated. Using distributed machine learning techniques, it is possible to drastically reduce the network overhead, while obtaining performance comparable to the cloud solution. The analysis also shows when each distributed learning approach is preferable, based on the specific distribution of the data on the nodes.
  arXiv  Detail & Related papers  (2021-09-27T13:44:34Z)
• National Access Points for Intelligent Transport Systems Data: From Conceptualization to Benefits Recognition and Exploitation [55.41644538483948]
  The European Union has proposed the development of a National Access Point (NAP) by each individual Member State. This paper aims to ascertain the role of a NAP within the ITS ecosystem, to investigate methodologies used in designing such platforms, and, through the drafting of an extended use case, showcase a NAP operational process and associate possible benefits with specific steps of it.
  arXiv  Detail & Related papers  (2020-10-14T17:13:00Z)
• Data Synopses Management based on a Deep Learning Model [14.180331276028662]
  We argue on the delivery of data synopses to EC nodes making them capable to take offloading decisions fully aligned with data present at peers. Our approach involves a Deep Learning model for learning the distribution of calculated synopses and estimate future trends.
  arXiv  Detail & Related papers  (2020-08-01T12:04:21Z)
• On the Use of Interpretable Machine Learning for the Management of Data Quality [13.075880857448059]
  We propose the use of interpretable machine learning to deliver the features that are important to be based for any data processing activity. Our aim is to secure data quality, at least, for those features that are detected as significant in the collected datasets.
  arXiv  Detail & Related papers  (2020-07-29T08:49:32Z)
• An Intelligent Scheme for Uncertainty Management of Data Synopses Management in Pervasive Computing Applications [9.289846887298852]
  Internet of Things (IoT) devices collect ambient data transferring them towards the Edge Computing (EC) and Cloud for further processing. EC nodes could become the hosts of distributed datasets where various processing activities take place. A critical issue for concluding this cooperative approach is the exchange of data synopses to have EC nodes informed about the data present in their peers. We propose n uncertainty driven model for the exchange of data synopses.
  arXiv  Detail & Related papers  (2020-07-24T16:58:51Z)
• Mining Implicit Entity Preference from User-Item Interaction Data for Knowledge Graph Completion via Adversarial Learning [82.46332224556257]
  We propose a novel adversarial learning approach by leveraging user interaction data for the Knowledge Graph Completion task. Our generator is isolated from user interaction data, and serves to improve the performance of the discriminator. To discover implicit entity preference of users, we design an elaborate collaborative learning algorithms based on graph neural networks.
  arXiv  Detail & Related papers  (2020-03-28T05:47:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.