Efficient PMU Data Analysis through High Performance Data Management Platform

Transcription

1 NASPI WG meeting Data & Network Management Task Team Efficient PMU Data Analysis through High Performance Data Management Platform 10/14/2015 Bo Lucy Yang, Jun Yamazaki, Norifumi Nishikawa, Hsiu-Khuern Tang, Alex Wang, Anshuman Sahu Big Data Research Laboratory Hitachi America Ltd.

2 Contents 1. Platform Architecture 2. High Performance Data Management 3. Integrated PMU applications 4. Visualization 1

3 Number Motivation Background PMUs have become increasingly popular in North America Many PMU data analysis tools have been developed Grid dynamics monitoring Oscillation detection Model validation Challenges Increasing PMU data size requires; Fast loading of historical data (Hundreds of TB/year) Efficient data cleansing against missing and bad data Effective data analysis To accelerate integrated online/offline analysis fully utilizing every PMU data into grid operation PMU installation in North America ~2,000 * *1: Silverstein, A. May NERC Board of Trustees Meeting, May 5, 2015, "An Update on the North American SynchroPhasor Initiative" 2

4 1-1. Platform Architecture High performance data management platform for PMU data apps High speed database engine Integrated PMU applications Visualizations Hitachi Visualization Oscillation Analysis Contingency Analysis Other Tools Hitachi Data Analysis Package Common Framework For Third Party Tools Hitachi Machine Learning Hitachi Database 3

5 1-2. Platform Architecture Total integration from fast data acquisition to various applications and effective visualization Case.1 DSA CA Hitachi Visualization Online PMU Historical PMU SE SCADA Topology MV Dynamic Model Oscillation Analysis Contingency Analysis Hitachi Data Analysis Package Case.2 PAA PAA VSA OD ED Online PMU DSA ML VSA OD ED Historical PMU Common Framework For Third Party Tools Hitachi Machine Learning Hitachi Database 4

6 Address (sector) Address (sector) 2-1. Hitachi database technology Fully utilizes the hardware (server, storage) resources SQL processing for DB search is automatically divided and executed with a high degree of parallelism Storage Access Trace in Conventional Approach [Conventional Approach] Sequential Execution Server Search Performance(μs) Storage Periodic I/O Processing(ms) Processing time (sec) [New Approach] Out-of-Order Execution Principle *1 Server more than thousands of tasks Storage Access Trace in New Approach Fully utilize storage s performance Storage Reduce the processing time dramatically Processing time (sec) Task Allocation Search Processing I/O Wait Disk I/O Application of the outcome of Development of the fastest database engine for the era of very large database, and Experiment and evaluation of strategic social services enabled by the database engine project (Principle Investigator: Prof. Masaru Kitsuregawa, University of Tokyo and also Director-General, National Institute of Informatics), supported by the Japanese Cabinet Office s FIRST Program (Funding Program for World-Leading Innovative R&D on Science and Technology). *1 A new principle invented by Professor Kitsuregawa and Project Associate Professor Goda (The University of Tokyo). 5

7 2-2. Performance evaluation Query #1 Time series Trend Search 10 minutes, 4 PMU Query #2 Snapshot Trend Search 5 snapshots, 500 PMU Conv. DB HADB x 55 x Query #1 Query #2 HADB: Hitachi Advanced Database 6

8 3-1. Integrated PMU applications Unsupervised data cleansing Outlier detection without domain knowledge Noise reduction based on data correlation Unsupervised, scalable solution 7

9 3-2. Integrated PMU applications Automatic abnormality event detection Online detection of grid events with measurement-based method Historical data mining for similar events Updating detection rules from stored historical event data Start Compute Feature Vector for PMUs Event Decision FFT amplitude Residue from fitted complex exponentials Spectral density Max-min change Update thresholds End 8

10 4. Visualization Visualization of various applications Coordination of information derived from heterogeneous data Intuitive display and operator support 9

11 Conclusion Conclusion To utilize both online and historical PMU data for operation Fast DB for acceleration and better efficiency of analysis tasks Coordination of various power apps for comprehensive analysis Visualization for intuitive awareness with heterogeneous information Future Plan Performance evaluation tests are ongoing on every layers; Database, Applications, and Visualization Integration of third party analysis tools to be tested Evaluation with real grid data will be planned 10

12 END Efficient PMU Data Analysis through High Performance Data Management Platform 10/14/2015 Bo Lucy Yang, Jun Yamazaki, Norifumi Nishikawa, Hsiu-Khuern Tang, Alex Wang, Anshuman Sahu Big Data Research Laboratory Hitachi America Ltd. 11

13