Enabling the Smart Grid through Big Data

Transcription

1 Enabling the Smart Grid through Big Data Paul A. Navrá;l, Ph.D. Manager Scalable Visualiza;on Technologies Texas Advanced Compu;ng Center TACC SC12 November 14, 2012

2 The Age of Big Data Records FiPeen sectors in US have more data stored per company than US Library of Congress (235 TB per company) As of 2012, six billion mobile phones worldwide RFID tag sales projected to grow from 12 million in 2011 to 209 billion in 2021 (17,500x) Smart meter growth projected from 130 million in 2011 to 340 million in 2015 (2.6x) Sources: ITU, McKinsey Global Ins;tute analysis, Pikes Research

3 The Age of Big Data Communica;on On Facebook in June 2012 now averages one trillion page views per month On Twi`er ;me to reach first billion tweets: 3 years, 2 months, 1 day ;me to reach one billion new tweets in 2011: 7 days ;me to reach one billion new tweets in 2012: 3 days Average tweets per day March million March million March million (as of June 12 now 400M +) On Reddit Feb 2011 reaches 1 billion page views per month Dec 2011 reaches 2 billion page views per month Oct 2012 now at 3.8 billion page views per month h`p://bit.ly/rmcl3j h`p://bit.ly/rmb7i1 h`p://bit.ly/wqp8am h`p://bit.ly/xrwv1p h`p://bit.ly/ags01w

4 Pecan Street Ini;a;ve Smart grid program in Aus;n, Texas 400+ homes wired for energy, water and gas consump;on at 15 second intervals In 12 months, 3 billion records, 200 GB on disk In 15 months, 5.6 billion records, 862 GB on disk now over 7 billion records, over 1 TB on disk Transi;on from tradi;onal row- store to column- store star schema for efficient queries Project expansion homes across Texas (interested in other states too) Finer- grained readings: 15 s - > 5 s - > 1 s - > 60 Hz More circuits: AC, PV, EV, etc.

5 Granularity Ma`ers August 10, 2011: 1- hour Consump;on Data 12! 8! 4! 0! 04:00! 08:00! 12:00! 16:00! 20:00!

6 Granularity Ma`ers August 10, 2011: 15- minute Consump;on Data 12! 8! 4! 0! 04:00! 08:00! 12:00! 16:00! 20:00!

7 Granularity Ma`ers August 10, 2011: 1- minute Consump;on Data 12! 8! 4! 0! 04:00! 08:00! 12:00! 16:00! 20:00!

8 Pecan Street on TACC Systems Data hos;ng on Corral Intel Xeon Westmere processing 12 data and service nodes with 2x 6- core Westmere hos;ng 4.8 PB SATA disk and 300 TB SAS disk from Data Direct Networks Visualiza;on and data analysis on Longhorn Intel Xeon Nehalem processing across 256 nodes, each with 2x 4- core Nehalem E5540 Visualiza;on and display on Stallion Intel Xeon Sandy Bridge processing across 20 nodes, each with 2x 6- core Sandy Bridge E head node has 2x 8- core Sandy Bridge E5-2687W will use Stampede Xeon E and Xeon Phi SE10P too

9 Intel- Based Hadoop Tes;ng Configura;on 4 node cluster, per node: 2x Intel Xeon 2.70GHz w/hyperthreading, 32 total processing sites 64 GB RAM 3x Seagate ST NS, 1TB 7200RPM, 64MB Cache 128 total processing sites 258 GB distributed RAM 12 TB total hadoop disk space.

10 Intel- Based Data Improvements Data currently in star- schema within a PostGreSQL database Intel op;miza;ons for Apache Hadoop make hadoop a viable alterna;ve Focusing on performance, management and security Be`er match for TACC distributed architecture ETL and query improvements Example: query ten minutes of data for one house Test set: 17.5M rows, 0.15 TB, 47 ms response Scale set 1: 209M rows, 1.8 TB, 53 ms response Scale set 2: 418M rows, 3.6 TB, 53 ms response 12x and 24x more data, only 6 ms longer

11

12

13

14 Energy Readings: AC, PV, EV and net usage

15 Energy Readings: AC and EV

16 Pecan Street Data Portal

17 Web- Based Visualiza;on

18 House Map

19 Geospa;al Visualiza;on Average home energy usage at 7 am, August Average home energy usage at 7 pm, August Mueller neighborhood data grouped by census block Visualized in Google Earth

20

21 Web- Based Geospa;al Visualiza;on

22 Average Wa`s Air Condi;oning Inside Mueller 4,000 2,000 04:00 08:00 12:00 16:00 20:00

23 August average Whole-home electricity usage (kw) 5! Consumption 2.5! 0! 4:00 AM! 8:00 AM! 12:00 PM! 4:00 PM! 8:00 PM! Pecan Street Consortium 23

24 August average Whole-home electricity usage and generation (kw) 5! Consumption and PV 2.5! 0! 4:00 AM! 8:00 AM! 12:00 PM! 4:00 PM! 8:00 PM! Pecan Street Consortium 24

25 August average Whole-home usage and modeled generation(kw) 5! Consumption, South PV, West PV 2.5! 0! 4:00 AM! 8:00 AM! 12:00 PM! 4:00 PM! 8:00 PM! Pecan Street Consortium 25

26 Average marginal price of wholesale energy - Aug PM $/megawatt hour $60

27 August average Peak consumption from grid (kw) 5! Consumption, South PV,, West PV 15.5 kwh 2.5! 7.5 kwh (48.6%) 4.7 kwh (30.1%) 0! 3:00 PM! 4:30 PM! 5:45 PM! 7:00 PM! Pecan Street Consortium 27

28 Select Enabled Research Projects Incen;ves for differing solar system configura;ons Pecan St., funded University of Texas researchers Feeder- Level Storage and Grid Impacts from EV, PV, storage University of Texas IGERT students, Center for Electromechanics Electric, Gas, and Water Disaggrega;on and Behavior Info funded University of Texas researchers, Pecan St. Consumer Behavior and Appliance Use Pa`erns funded University of Texas researchers, Pecan St. Impacts of System Accuracy and Data funded University of Texas researchers, Pecan St. AC system sizing, Savings from Energy Efficiency Measures funded University of Texas researchers Emissions Reduc;ons Environmental Defense Fund, Pecan St.

29 Thank you! Paul A.