A Visual Analytics Framework for Reviewing Streaming Performance Data
- URL: http://arxiv.org/abs/2001.09399v1
- Date: Sun, 26 Jan 2020 04:34:22 GMT
- Title: A Visual Analytics Framework for Reviewing Streaming Performance Data
- Authors: Suraj P. Kesavan, Takanori Fujiwara, Jianping Kelvin Li, Caitlin Ross,
Misbah Mubarak, Christopher D. Carothers, Robert B. Ross, Kwan-Liu Ma
- Abstract summary: We introduce a visual analytic framework comprising of three modules: data management, analysis, and interactive visualization.
In particular, we introduce a set of online and progressive analysis methods for not only controlling the computational costs but also helping analysts better follow the critical aspects of the analysis results.
- Score: 20.61348106852359
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Understanding and tuning the performance of extreme-scale parallel computing
systems demands a streaming approach due to the computational cost of applying
offline algorithms to vast amounts of performance log data. Analyzing large
streaming data is challenging because the rate of receiving data and limited
time to comprehend data make it difficult for the analysts to sufficiently
examine the data without missing important changes or patterns. To support
streaming data analysis, we introduce a visual analytic framework comprising of
three modules: data management, analysis, and interactive visualization. The
data management module collects various computing and communication performance
metrics from the monitored system using streaming data processing techniques
and feeds the data to the other two modules. The analysis module automatically
identifies important changes and patterns at the required latency. In
particular, we introduce a set of online and progressive analysis methods for
not only controlling the computational costs but also helping analysts better
follow the critical aspects of the analysis results. Finally, the interactive
visualization module provides the analysts with a coherent view of the changes
and patterns in the continuously captured performance data. Through a
multi-faceted case study on performance analysis of parallel discrete-event
simulation, we demonstrate the effectiveness of our framework for identifying
bottlenecks and locating outliers.
Related papers
- CityGPT: Towards Urban IoT Learning, Analysis and Interaction with Multi-Agent System [4.612237040042468]
CityGPT employs three agents to accomplish thetemporal analysis of IoT data.
We have agnentized the framework, facilitated by a large language model (LLM), to increase the data comprehensibility.
Our evaluation results on real-world data with different time show that the CityGPT framework can guarantee robust performance in computing.
arXiv Detail & Related papers (2024-05-23T15:27:18Z) - Demonstration of InsightPilot: An LLM-Empowered Automated Data
Exploration System [48.62158108517576]
We introduce InsightPilot, an automated data exploration system designed to simplify the data exploration process.
InsightPilot automatically selects appropriate analysis intents, such as understanding, summarizing, and explaining.
In brief, an IQuery is an abstraction and automation of data analysis operations, which mimics the approach of data analysts.
arXiv Detail & Related papers (2023-04-02T07:27:49Z) - EasyDGL: Encode, Train and Interpret for Continuous-time Dynamic Graph Learning [92.71579608528907]
This paper aims to design an easy-to-use pipeline (termed as EasyDGL) composed of three key modules with both strong ability fitting and interpretability.
EasyDGL can effectively quantify the predictive power of frequency content that a model learn from the evolving graph data.
arXiv Detail & Related papers (2023-03-22T06:35:08Z) - Reinforced Approximate Exploratory Data Analysis [7.974685452145769]
We are first to consider the impact of sampling in interactive data exploration settings as they introduce approximation errors.
We propose a Deep Reinforcement Learning (DRL) based framework which can optimize the sample selection in order to keep the analysis and insight generation flow intact.
arXiv Detail & Related papers (2022-12-12T20:20:22Z) - Distributed intelligence on the Edge-to-Cloud Continuum: A systematic
literature review [62.997667081978825]
This review aims at providing a comprehensive vision of the main state-of-the-art libraries and frameworks for machine learning and data analytics available today.
The main simulation, emulation, deployment systems, and testbeds for experimental research on the Edge-to-Cloud Continuum available today are also surveyed.
arXiv Detail & Related papers (2022-04-29T08:06:05Z) - DANCE: DAta-Network Co-optimization for Efficient Segmentation Model
Training and Inference [85.02494022662505]
DANCE is an automated simultaneous data-network co-optimization for efficient segmentation model training and inference.
It integrates automated data slimming which adaptively downsamples/drops input images and controls their corresponding contribution to the training loss guided by the images' spatial complexity.
Experiments and ablating studies demonstrate that DANCE can achieve "all-win" towards efficient segmentation.
arXiv Detail & Related papers (2021-07-16T04:58:58Z) - FENXI: Deep-learning Traffic Analytics at the Edge [69.34903175081284]
We present FENXI, a system to run complex analytics by leveraging TPU.
FENXI decouples operations and traffic analytics which operates at different granularities.
Our analysis shows that FENXI can sustain forwarding line rate traffic processing requiring only limited resources.
arXiv Detail & Related papers (2021-05-25T08:02:44Z) - Graph signal processing for machine learning: A review and new
perspectives [57.285378618394624]
We review a few important contributions made by GSP concepts and tools, such as graph filters and transforms, to the development of novel machine learning algorithms.
We discuss exploiting data structure and relational priors, improving data and computational efficiency, and enhancing model interpretability.
We provide new perspectives on future development of GSP techniques that may serve as a bridge between applied mathematics and signal processing on one side, and machine learning and network science on the other.
arXiv Detail & Related papers (2020-07-31T13:21:33Z) - Informative Scene Decomposition for Crowd Analysis, Comparison and
Simulation Guidance [10.000622844914272]
Crowd simulation is a central topic in several fields including graphics.
With the fast-growing volume of crowd data, such a bottleneck needs to be addressed.
We propose a new framework which comprehensively tackles this problem.
arXiv Detail & Related papers (2020-04-29T12:03:32Z) - Interpretable Feature Learning in Multivariate Big Data Analysis for
Network Monitoring [0.4342241136871849]
We present an extension of the Multivariate Big Data Analysis (MBDA) methodology, a recently proposed interpretable data analysis tool.
We propose a solution to the automatic derivation of features, a cornerstone step for the application of MBDA when the amount of data is massive.
We apply the extended MBDA to two case studies: UGR'16, a benchmark flow-based real-traffic dataset for anomaly detection, and Dartmouth'18, the longest and largest Wi-Fi trace known to date.
arXiv Detail & Related papers (2019-07-05T04:51:49Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.