CANARIE: Providing Essential Digital Infrastructure for Canada Mark Wolff; CTO April 16, 2014
A Transformation of the Science Paradigm thousands of years ago last few hundred years last few decades today tomorrow empirical theoretical computational DATA volume and complexity data exploration (e-science) 2
3
The Digital Transformation of Research 350 EB / year: Data produced by the Square Kilometre Array Yottabyte 1,000,000,000,000,000,000,000,000 25 PB / year: Data generated by the Large Hadron Collider Zettabyte 1,000,000,000,000,000,000,000 1 PB of MP3 songs would take 2000 years to play Exabyte 1,000,000,000,000,000,000 Giga 1,000,000,000,000 Terabyte 1,000,000,000,000 Petabyte 1,000,000,000,000,000 not to scale. 4
The e-science Data Deluge Volume Terabytes + Variety Structured, unstructured Velocity Real-time, requires immediate response Value Complex, predictive analysis 5
The Challenge for Digital Infrastructure Advanced Computing Advanced High-Speed Networks Unified Digital Infrastructure Applications & Software Tools Expertise Distributed Data Storage Digital infrastructure is more than advanced networks. 6
Building Tomorrow s Digital Infrastructure Today Network & Network s Cloud s for Entrepreneurs Software for Research 7
Why? To accelerate Canadian innovation through research in both the public and private sectors To drive the adoption of transformative technologies among our stakeholder communities To ensure Canada continues to develop a vibrant knowledge economy that benefits all Canadians 8
Network & Network s 9
Network & Network s 10
CANARIE Connects to the World Acknowledgements - The Global Lambda Integrated Facility (GLIF) Map 2011 visualization was created by Robert Patterson of the Advanced Visualization Laboratory (AVL) at the National Center for Supercomputing Applications (NCSA) at the University of Illinois at Urbana-Champaign (UIUC), using an Earth image provided by NASA with texture retouching by Jeff Carpenter, NCSA. Data was compiled by Maxine D. Brown of the Electronic Visualization Laboratory (EVL) at the University of Illinois at Chicago (UIC). Support was provided by GLIF, NCSA/UIUC, the State of Illinois, and US N 11
Network & Network s 12
Network & Network s Canadian Access Federation Ensuring a trusted, unified, digital identity for Canada s research and education communities. WiFi access worldwide using your home institution credentials Single Sign On : Use one credential to access multiple services s access worldwide using your home institution credentials 13
But Networks Alone Do Not Drive the Adoption of Digital Infrastructure Commercial toolsets require costprohibitive development Minimal shared infrastructure among disciplines Barrier to adoption & leverage of powerful infrastructure 14
The Missing Piece: Software for Researchers Facilitates use of CANARIE network Funds the development of tools and software that help researchers focus on research rather than technologies and equipment Focused on big data tools and platforms 15
Middleware for Canadian Research: Network-Enabled Platforms (NEPs) and Research Platform Interfaces (RPIs) NEP = a software platform optimized to leverage digital infrastructure for big data analysis in support of research, discovery and innovation RPI = a toolkit of reusable software services that accelerate the development of domain-specific research platforms A more efficient way to turn research data into knowledge Specifically designed by researchers for researchers 16
The Power of Research Software Platforms Existing CBRAIN NEP (soon to be an RPI contributor) best illustrates a networkenabled platform https://brainbrowser.cbrain.mcgill.ca/surface-viewer# 17
Why RPI? Excellent capabilities (services) are being built within research platforms But services are locked inside (most) platforms Platforms were/are not designed to use or provide external services John Gress/Reuters 18
NEP Program - Platform Architecture Platform 1 1 Platform 2 1 Platform 3 1 Platform 4 1 2 2 2 2 3 3 3 4 4 5 19
Platform Architecture after RPI Platform 1 1 Platform 2 1 Platform 3 1 Platform 4 1 2 2 2 2 3 3 A B 4 C 20
Platform Architecture after NEP-RPI Platform 1 1 Platform 2 1 Platform 3 1 Platform 4 1 2 2 2 A B C D E F G H I 21
Use Case: Ocean Networks Canada Observatory Platform ONC makes their data OGC compliant and releases the data as a (an RPI). Example: water pressure Chilean Earthquake and Tsunami of 27 Feb 2010
Disaster Recovery Platform - Sendai Japan Earthquake & Tsunami, March 11, 2011 The Disaster Recovery Platform adds OCG data compliance, and connects to the ONC Observatory RPI for realtime data.
The result is a highly advanced tsunami detector
Efficient Reuse of Software Elements from Platforms Previous Mandate: 21 Platforms Funded New RPIs created 21 15 Software Platforms (NEP) Software Elements (RPI) leveraged by new Platforms 12 spinoff additional RPIs 47 25
A New Paradigm for Research Software Development Unlocks functional elements built within research Software Platforms Supports the culture of collaboration and reuse within scientific research community Maximizes efficiency by eliminating duplication of effort and investment Minimizes time to discovery: 1. Browse / search software element repository 2. Reuse existing software elements to build customized platform for research requirements 3. Share new, discreet software elements with the global community 4. Contribute to innovative software ecosystem CANARIE s toolkit of software elements is available to the research community at no cost: https://science.canarie.ca Interested in providing a software element? support@science.canarie.ca 26
The Cloud Transforms the Provisioning and Delivery of Software and s Web-based access to centralized computing resources & software applications On-demand, pay-per-use access Pool of shared, configurable resources (servers, storage, network) Rapid provisioning with minimal management effort or service provider interaction 27
Digital Technology Megatrends Mobile Devices 18x: size of mobile data traffic in 2013 vs global internet traffic in 2000 81%: global mobile data traffic growth in 2013 500M: mobile devices added in 2013 alone Social Media 100: hours of video uploaded to YouTube each minute 2: # of new Facebook users each second 93%: adoption of social media by marketers 30+ Petabytes: volume of user generated data stored, accessed & analyzed by Facebook Big Data 2.5 Exabytes: volume of data created daily, doubling each month 44x: forecasted increase in data volume 2009-2020 1 week: time to process decoding of human genome today vs 10 years initially Internet of Things 2008: year in which # of things connected to Internet surpassed world population 50B: forecasted # of things connected to the Internet by 2020 Source: Cisco Visual Networking Index, IBM, IDC, The Economist, Huffington Post, Wikibon, YouTube, Pew Research, Business Insider, Global Web Index 28
All Enabled By Cloud Computing!!! 29
What is Cloud Computing? A network of remote servers on the Internet to store, manage, and process data instead of on local servers or personal computers. 30
Cloud Computing is Already Ubiquitous 31
Digital Accelerator for Innovation & Research (DAIR) Application development accelerator provisioned on the CANARIE network Powerful test-bed for SMEs to develop, test, & demonstrate new cloud applications Access to cloud computing, network & storage infrastructure (IaaS -- Infrastructure as a ) A kick-start to new cloud service offerings Mobilizes new cloud ICT businesses in Canada 32
Digital Accelerator for Innovation & Research (DAIR) 1 st users on DAIR June 2011, among first IaaS clouds in Canada Edmonton and Sherbrooke regions operated by Compute Canada with cloud software support from Cybera, using OpenStack Close to 200 companies have used DAIR to speed their time to market. 33
Summary: CANARIE Mandate Network Operations: Continue to operate the CANARIE Network as essential research infrastructure; Technology Innovation: Develop, demonstrate, and implement next-generation technologies to advance the CANARIE Network as a leading-edge research network; and Private Sector Innovation: Leverage the CANARIE Network to assist firms operating in Canada, and Canadian universities, to advance innovation and commercialization of products and services to bolster Canada s technology innovation capabilities. 34
35