SESSION TITLE GOES HERE Second Cosmos for Line the Goes Business Here Intelligence Professional

Size: px

Start display at page:

Download "SESSION TITLE GOES HERE Second Cosmos for Line the Goes Business Here Intelligence Professional"

Stuart Heath
5 years ago
Views:

1 Azure Cosmos DB with Power BI SESSION TITLE GOES HERE Second Cosmos for Line the Goes Business Here Intelligence Professional Cosmos for the Business Intelligence Professional Speaker Name Speaker Title Company Name Speaker Name Speaker Title Company Name

Jen Stirrup Boutique Consultancy Owner of Data

com http://bit.ly/jenstirruprd http://bit.

2 Jen Stirrup Boutique Consultancy Owner of Data Relish Postgraduate degrees in Artificial Intelligence and Cognitive Science Twenty year career in industry Author JenStirrup.com DataRelish.com

3 Tipping Point to NoSQL Large Data Sets Structured / Unstructured Data New Paradigm Scalability Social Media

4 CAP Theorem Consistency, Availability and Partition Tolerance. A shared-data system can have at most two of those three.

5 Atomicity: All or Nothing Consistency: Any transaction should result in valid tables Isolation: separate transactions Durability: Database will survive system failures

6 Basically Available - system seems to work all the time Soft State - it doesn't have to be consistent all the time Eventually Consistent - becomes consistent at some later time

7 Different Types of Data need Different Solutions Data Structured Semi-structured Unstructured Transactional Documents Relational DW XML JSON Scientific Data Video

8 Different Types of Data need Different Solutions Data Structured Semi-structured Unstructured Transactional Documents Relational DW XML JSON Scientific Data Video

9 Data Sources Prior to NoSQL Data Sources including NoSQL Relational Key-Value Key Key Value Value Graph Column - Family Document OLAP 6

10 Relational Tabular format SQL concepts Tables Joins Rows / Columns SQL Language Rigid Data Modelling

11 Key-Value Stores Key Key Value Value Keys are used to access blobs of data Video Images.. A key uniquely identifies each record. Dictionaries have records that are stored and retrieved using a key. If it fast because the key uniquely identifies each record. Data is a single opaque collection

12 Locker Analogy Keys are used to access blobs of data Video Images.. A key uniquely identifies each record. Dictionaries have records that are stored and retrieved using a key. The Value is simply an object.

13 Relational Queries return a set Rows for large data sets must be indexed Column values must all have the same data type Key-Value Store Model Queries return a single item No indexes on values Values can be any data type Result Result

14 Types of Key-Value Stores Eventually consistent key value Hierarchical key value stores Key-Value stores in RAM Key-Value stores on disk High availability key-value store Ordered key value stores store Values that allow simple list operations 25

15 Graph Store Data is stored in nodes, which have properties They are connected by critical relationships

16 Documents Data stored in nested hierarchies Logical data remains stored together as a unit Any item in the document can be queried Pros: No object-relational mapping layer, ideal for search Cons: Complex to implement, incompatible with SQL

17 RDBMS BI/DW Web Crawlers Open Linked Data Transactional Log Files JSON Read/Write Read Mostly BinaryGraph XML Structured Unstructured Documents

18 JSON Documents

19 Azure Cosmos DB ODBC driver Connects Azure Cosmos DB to analytical tools such as: SSIS Power BI Desktop Tableau Excel

20 JSON Documents { } "Volcano Name": "Rainier", "Country": "United States", "Region": "US-Washington", "Location": { "type": "Point", "coordinates": [ , ] }, "Elevation": 4392, "Type": "Stratovolcano", "Status": "Dendrochronology", "Last Known Eruption": "Last known eruption from , inclusive"

21 Objective Retrieve the volcano data from the Azure Cosmos DB account Visualize data in an interactive Power BI report like the following report

22 Tools Power BI Desktop Azure CosmosDB Connector DAX

Sequence Open Power BI Connect to Azure CosmosDB using required keys Content goes here. Content goes here. Content goes here. Content goes here. Content goes here. Content goes here. Content goes here. Content goes here. Content goes here. Content goes here. Content goes here. Content goes here. Content goes here. Content goes here. Content goes here. Content goes here. Content goes here. Content goes here. Content goes here. Content goes here. Format data in Power BI Create Power BI Visualization Publish to PowerBI.

23 Sequence Open Power BI Connect to Azure CosmosDB using required keys Content goes here. Content goes here. Content goes here. Content goes here. Content goes here. Content goes here. Content goes here. Content goes here. Content goes here. Content goes here. Content goes here. Content goes here. Content goes here. Content goes here. Content goes here. Content goes here. Content goes here. Content goes here. Content goes here. Content goes here. Format data in Power BI Create Power BI Visualization Publish to PowerBI.com

24 References Apache Spark as an intermediary cosmosdb-spark/wiki/configuring-power-bi-direct-query-to-azure- Cosmos-DB-via-Apache-Spark-(HDI)

25 Boutique Consultancy Owner of Data Relish Jen Stirrup Postgraduate degrees in Artificial Intelligence and Cognitive Science Twenty year career in industry Author

26 What is blockchain?

27 What is Blockchain? The blockchain is an incorruptible digital ledger of economic transactions that can be programmed to record not just financial transactions but virtually everything of value. Don & Alex Tapscott, authors Blockchain Revolution (2016)

28 It can be perceived as a database problem Can databases and blockchain live happily ever after? By allowing digital information to be distributed but not copied, blockchain technology created the backbone of a new type of internet. Bitcoin has been called digital gold, To date, the total value of the currency is close to $9 billion US.

29 Blockchain was introduced by Bitcoin Response by technical community to Wall Street crash Decentralized Control Nobody owns the network Immutability Tamper Resistance Create and transfer assets with no reliance on a central entity

31 Applications Decentralized blockchain computing platforms Decentralized processing and storage Smart Contracts Cryptographic primitives, consensus protocols and algorithms

34 Public Verifiability Uncorrupted and unaltered data Transparency

35 Throughput is a few transactions a second Latency before a confirmed write is ten minutes Capacity is a few Gb. Adding nodes causes more issues Double the nodes produces quadruple the network traffic No improvement in throughput, latency or capacity Oh, and you can t query it very well

36 Blockchain as a Database is good Consortium Database Decentralized / shared control blockchains allow enemies to work together for a common benefit R3 for banks Open Music Initiative Immutability and audit trail Assets and exchanges

38 User can add more data in the form of blocks Read Operations: these query and retrieve data from the blockchain Write Operations: these add more data onto the blockchain

39 Permissioned and Private: R3 Corda Hyperledger Public Smart Contract Ethereum

40 Cryptokitties

42 Q&A

Introduction to NoSQL Databases

Introduction to NoSQL Databases Roman Kern KTI, TU Graz 2017-10-16 Roman Kern (KTI, TU Graz) Dbase2 2017-10-16 1 / 31 Introduction Intro Why NoSQL? Roman Kern (KTI, TU Graz) Dbase2 2017-10-16 2 / 31 Introduction