Demystifying Cloud Data Warehousing

Transcription

1 YOUR DATA, NO LIMITS Demystifying Cloud Data Warehousing Nicolas Baret Director of Pre-Sales TDWI Helsinki, October

2 What is a Cloud Data Warehouse and what should we expect? 2

3 What is a Cloud Data Warehouse? DW- Data Warehouse Relational database Uses standard SQL Optimized for fast loads and analytic queries aas As a Service Like SaaS (e.g. SalesForce.com) No infrastructure set up Minimal to no administration Managed for you by the vendor Pay as you go, for what you use 3

4 What to Expect from a Cloud DW (1/2) It should support standard SQL (natively) Support standard ETL & BI tools ODBC or JDBC connectivity It should be infinitely scalable (cloud) Handle huge amounts of data Handle large number of concurrent queries without performance degradation It should handle flexible schema data types No sharding or ETL required 4

5 What to Expect from a Cloud DW (2/2) It should be secure Built in encryption? It shoud be stable Resiliancy and high availability by design It should be easy to configure and manage It should provide a lower TCO Cloud scale pricing 5

6 Common customer scenarios nosql replacement Replace use of nosql system (e.g. Hadoop) for transformation and SQL analytics of multistructured data Data warehouse for SaaS offerings Use Cloud DW as backend data warehouse supporting data-driven SaaS products Data warehouse modernization Consolidate legacy datamarts and support new projects 6

7 Enabling key use cases Datamart & data silo consolidation Consolidate legacy datamarts to eliminate silos and support new projects Integrated data analytics Directly load structured + semi-structured data into a single platform for reporting & analytics Exploratory & ad hoc analytics Direct access to data for SQL analysts to explore data, identify correlations, build & test models 7

8 Data challenges today 8

9 The world of data has changed Gartner 2016 Predictions: By 2018, six billion connected devices will be requesting support. Connecting applications, devices, and things Reaching employees, business partners, and consumers Anytime, anywhere mobility On demand, unlimited scale Understanding behavior, generating, retaining, and analyzing data 9

10 40 Zettabytes by 2020 Web 3 rd party apps Mobile Enterprise apps ERP IoT 10

11 It s not the data itself Web 3 rd party apps Mobile Enterprise apps ERP IoT it s how you take full advantage of the insight it provides 11

12 Most firms don t consistently turn data into action 73% All Possible data All Possible Action of firms aspire to be data-driven. 29% of firms are good at turning data into action. Source: Forrester 12

13 Symptoms of fundamental challenges Data silos Cost Performance Inflexibility Complexity Data locked into separate databases, big data systems, and applications Painful upfront costs and overprovisioned capacity Contention for limited resources resulting in latency and delays Slow, cumbersome scaling and limited support for diverse data Multiple systems to integrate and manage requiring specialized skills and tools 13

14 The evolution of data platforms Relational database Oracle, DB2, SQL Server Data warehouse appliance Teradata Data warehouse & platform software Vertica, Greenplum, Paraccel, Hadoop Cloud DWaaS Snowflake 1980s 1990s 2000s 2010s 14

15 Introducing Snowflake 15

16 About Snowflake 2012 Founded by industry veterans with over 120 database patents First data warehouse built for the cloud Over 800 customers since GA Vision: A world with no limits on data Experienced, accomplished leadership team 16

17 Snowflake: 1st Data Warehouse Built for the Cloud Data Warehousing for Everyone Data scientists SQL users & tools SQL relational database Optimized storage & processing Standard connectivity BI, ETL, Existing SQL skills and tools Load and go ease of use Cloud-based elasticity to fit any scale 17

18 What customers like about Snowflake Performance Concurrency Simplicity 200x We can now run an entire set of monthly reports that used to take 2-3 days in just 45 minutes. Ken Wood, Data Team Manager Consistency of performance and concurrency has been a huge win for us. Erika Bakse, BI Solution Architect That we don t need to do any configuration or tuning is great: we can focus on analyzing data instead of managing and tuning a data warehouse. Craig Lancaster, CTO 18

19 Data warehouse built for the cloud 100% Cloud Complete SQL Database Zero Management All of your Data All of your Users Pay only for what you use

20 Top 10 Cool Things about Snowflake 20

21 #10 - Persistent Result Sets No setup In Query History By Query ID 24 Hours No re-execution No Cost for Compute 21

22 #9 - Connect w/jdbc & ODBC to the cloud Data Sources Data Modeling, Management & Transformation >_ Custom & Packaged Applications Java JDBC Scripting Reporting & Analytics 22

23 #8 - UNDROP UNDROP TABLE <table name> UNDROP SCHEMA <schema name> UNDROP DATABASE <db name> Part of Time Travel feature: AWESOME! 23

24 #7 - Zero-Copy Clone Instant copy of table, schema, or database: CREATE OR REPLACE TABLE MyTable_V2 CLONE MyTable; With Time Travel: CREATE SCHEMA mytestschema_clone_restore CLONE testschema BEFORE (TIMESTAMP => TO_TIMESTAMP(40*365*86400)); PROD PUBLIC Table A Table B Table C DEV PUBLIC Table A Table B Table C INT PUBLIC Table A Table B Table C 24

25 #6 - JSON Support with SQL Structured data (e.g. CSV) Apple FIH-2316 Pear IHO-6912 Orange WHQ-6090 Semi-structured data (e.g. JSON, Avro, XML) { "firstname": "John", "lastname": "Smith", "height_cm": , "address": { "streetaddress": "21 2nd Street", "city": "New York", "state": "NY", "postalcode": " " }, "phonenumbers": [ { "type": "home", "number": " " }, { "type": "office", "number": " " } ] } All Your Data! Optimized storage Flexible schema - Native Relational processing select v:lastname::string as last_name from json_demo; 25

26 #5 - Standard SQL w/analytic Functions select Nation, Customer, Total from (select n.n_name Nation, c.c_name Customer, sum(o.o_totalprice) Total, rank() over (partition by n.n_name order by sum(o.o_totalprice) desc) customer_rank from orders o, customer c, nation n where o.o_custkey = c.c_custkey and c.c_nationkey = n.n_nationkey group by 1, 2) where customer_rank <= 3 order by 1, customer_rank SQL Complete SQL database Data definition language (DDLs) Query (SELECT) Updates, inserts and deletes (DML) Role based security Multi-statement transactions 26

27 #4 - Separation of Storage & Compute Snowflake s multi-cluster, shared data architecture Centralized storage Service Compute Storage Instant, automatic scalability & elasticity 27

28 #3 - Support Multiple Workloads Deliver faster analytics at any scale Accelerate the data pipeline Run loading & analytics at any time, concurrently, to get data to users faster Scale compute to support any workload Marketing Scale processing horsepower up and down on-the-fly, with zero downtime or disruption Loading Finance Scale concurrency without performance impact Multi-cluster virtual warehouse architecture scales concurrent users & workloads without contention 28

29 #2 - Secure by Design with Automatic Encryption of Data! Authentication Access control Data encryption External validation Embedded multi-factor authentication Role-based access control model All data encrypted, always, end-to-end Certified against enterpriseclass requirements Federated authentication available Granular privileges on all objects & actions Encryption keys managed automatically NEW: Tri-secret security HIPPA Certified! PCI Certified! 29

30 #1 - Automatic Query Optimization Zero Management Fully managed with no knobs or tuning required No indexes, distribution keys, partitioning, vacuuming, Zero infrastructure costs Zero admin costs 30

31 New #1 - Data Sharing (The Data Sharehouse ) Data Providers Data Consumers No data movement Share with unlimited number of consumers Live access Data consumers immediately see all updates Ready to use Consumers can immediately start querying 31

32 Snowflake in Action Today 32

33 What customers are doing with Snowflake DATA SOURCES REPORTING, ANALYTICS & APPLICATIONS STAGING DATA WAREHOUSE DATA LAKE DATA MARTS & EXTRACTS Gaming company replaced Hadoop + SQL database with Snowflake Consumer retailer modernizing DW by replacing legacy appliance with Snowflake Market research company consolidated data marts to reduce costs and data silos Mobile analytics company shares live data with clients 33

34 Delivering compelling results Simpler data pipeline Replace nosql database with Snowflake for storing & transforming JSON event data Snowflake: 1.5 minutes nosql data base: 8 hours to prepare data Faster analytics Replace on-premise data warehouse with Snowflake for analytics workload Snowflake: 45 minutes Data warehouse appliance: 20+ hours Significantly lower cost Improved performance while adding new workloads - at a fraction of the cost Snowflake: added 2 new workloads for $50K Data warehouse appliance: > $5M to expand 34

35 Ranked #1 Cloud Data Warehouse! Snowflake Hits All the Marks Gigaom Gigaom Disruption Vectors Cloud Analytics Database Robustness of SQL 15% Built-in Optimization 15% On-the-fly Elasticity 25% Dynamic Environment Adaption 20% Separation of Compute from storage 15% Support for Diverse data 10% 4,85 4,50 4,45 3,75 3,75 3,35 Snowflake Google Big Query Teradata DashDB (IBM) Vertica Azure Data Warehouse You can tell the data warehouse pedigree from the development With superior performance and the most hands-off model of ownership, Snowflake is the epitome of data warehouse as a service. The model, cost, features and scalability have already caused some to postpone Hadoop adoption. 3,20 3,15 SAP HANA Cloud Platform Oracle Database Exdata Cloud Service William McKnight 2,60 AWS Redshift Distruption Vectors Gigaom Analyst Report: Sector Roadmap: Cloud Analytic Databases 2017 Score Read the full report on snowflake.net 35

36 Thank You! Want to follow-up? YOUR DATA, NO LIMITS 36