Data sources. Gartner, The State of Data Warehousing in 2012

Size: px
Start display at page:

Download "Data sources. Gartner, The State of Data Warehousing in 2012"

Transcription

1

2

3 data warehousing has reached the most significant tipping point since its inception. The biggest, possibly most elaborate data management system in IT is changing. Gartner, The State of Data Warehousing in 2012 Data sources

4 5 2 Real-time data 1 Increasing data volumes New data sources & types Data sources Non-Relational Data 3 4 Cloud-born data

5 Extract Transform Load Original Data ETL Tool (SSIS, etc) Transformed Data EDW (SQL Svr, Teradata, etc) BI Tools Data Marts Data Lake(s) Dashboards Apps

6 Extract Transform Load Original Data ETL Tool (SSIS, etc) Transformed Data EDW (SQL Svr, Teradata, etc) BI Tools Data Marts Data Lake(s) Dashboards Ingest (EL) Original Data Apps

7 Extract Transform Load Original Data ETL Tool (SSIS, etc) Transformed Data EDW (SQL Svr, Teradata, etc) BI Tools Data Marts Data Lake(s) Ingest (EL) Original Data Scale-out Storage & Compute (HDFS, Blob Storage, etc) Dashboards Apps Streaming data Transform & Load

8 Extract Transform Load Original Data ETL Tool (SSIS, etc) Transformed Data EDW (SQL Svr, Teradata, etc) BI Tools Data Marts Data Lake(s) Ingest (EL) Original Data Scale-out Storage & Compute (HDFS, Blob Storage, etc) Dashboards Apps Streaming data Transform & Load

9 Data Sources (Import From) Ingest Data Hub (Storage & Compute) BI Tools Data Marts Move data among Hubs Data Lake(s) Data Sources (Import From) Ingest Data Hub (Storage & Compute) Move to data mart, etc Dashboards Apps Information Production: Connect & Collect Transform & Enrich Publish

10 Data Sources (Import From) Data Connector: Import from source to Hub Data Hub (Storage & Compute) Coordination & Scheduling Monitoring & Mgmt Data Lineage BI Tools Data Connector: Import/Export among Hubs Data Marts Data Lake(s) Data Sources (Import From) Data Connector: Import from source to Hub Data Hub (Storage & Compute) Data Connector: Export from Hub to data store Dashboards Apps Information Production: Connect & Collect Transform & Enrich Publish

11

12

13 Example Scenario: Data warehouse sales to Azure pipeline

14 Raw sales (Custom view on top of DW tables) Sales by category by day Hive processing Qty Unit OrderDate Company Category Sales Order Ordered Price 6/1/2004 Action Bicycle Specialists Accessories SO /1/2004 Action Bicycle Specialists Bikes SO /1/2004 Action Bicycle Specialists Clothing SO /1/2004 Action Bicycle Specialists Components SO /1/2004 Aerobic Exercise Company Components SO /1/2004Action Bicycle Specialists Accessories SO71938

15 Data Factory Walkthrough

16 New-AzureDataFactory -Name DW-Demo2 -Location West-US New-AzureDataFactory -Name HaloTelemetry -Location West-US

17 Azure Data Factory New User View On Premises SQL Server Azure Blob Storage

18 View Of Azure Data Factory New Sales Aggregated sales AdventureWorksLTDW2014 On Premises SQL Server Azure Blob Storage

19 View Of View Of Azure Data Factory Pipeline New Sales Copy NewSales to Blob Storage Cloud New Sales New User Activity New User View On Premises SQL Server Azure Blob Storage

20 View Of Azure Data Factory Pipeline New Sales Copy New Sales to Blob Storage Cloud New Sales Pipeline OnPrem SSIS package Aggregated Sales Cloud New Sales Aggregate AggregatedSales HDInsight New User View On Premises SQL Server Azure Blob Storage

21 "availability": { "frequency": "Day", interval": 6 } Activity: (e.g. Hive): Hourly AggregatesSales

22 Hourly Sales From DW Daily Monday Daily Sales Dataset3 Hive Activity Tuesday Daily other source Dataset2 Wednesday Monday Tuesday Wednesday

23 Is my data successfully getting produced? Is it produced on time? Am I alerted quickly of failures? What about troubleshooting information? Are there any policy warnings or errors?

24

25

26 Easily move data to my existing data marts for consumption by my existing BI tools Azure DB SQL Server on premises Oracle Files Azure Blob content

27 Coordination: Rich scheduling Complex dependencies Incremental rerun Authoring: JSON & Powershell/C# Management: Lineage Data production policies (late data, rerun, latency, etc) Hub: Azure Hub (HDInsight + Blob storage) Activities: Hive, Pig, C# Data Connectors: Blobs, Tables, Azure DB, On Prem SQL Server, Oracle

28 Contact me:

29

Data sources. Gartner, The State of Data Warehousing in 2012

Data sources. Gartner, The State of Data Warehousing in 2012 data warehousing has reached the most significant tipping point since its inception. The biggest, possibly most elaborate data management system in IT is changing. Gartner, The State of Data Warehousing

More information

Azure Data Factory VS. SSIS. Reza Rad, Consultant, RADACAD

Azure Data Factory VS. SSIS. Reza Rad, Consultant, RADACAD Azure Data Factory VS. SSIS Reza Rad, Consultant, RADACAD 2 Please silence cell phones Explore Everything PASS Has to Offer FREE ONLINE WEBINAR EVENTS FREE 1-DAY LOCAL TRAINING EVENTS VOLUNTEERING OPPORTUNITIES

More information

Oskari Heikkinen. New capabilities of Azure Data Factory v2

Oskari Heikkinen. New capabilities of Azure Data Factory v2 Oskari Heikkinen New capabilities of Azure Data Factory v2 Oskari Heikkinen Lead Cloud Architect at BIGDATAPUMP Microsoft P-TSP Azure Advisors Numerous projects on Azure Worked with Microsoft Data Platform

More information

BIG DATA COURSE CONTENT

BIG DATA COURSE CONTENT BIG DATA COURSE CONTENT [I] Get Started with Big Data Microsoft Professional Orientation: Big Data Duration: 12 hrs Course Content: Introduction Course Introduction Data Fundamentals Introduction to Data

More information

Alexander Klein. #SQLSatDenmark. ETL meets Azure

Alexander Klein. #SQLSatDenmark. ETL meets Azure Alexander Klein ETL meets Azure BIG Thanks to SQLSat Denmark sponsors Save the date for exiting upcoming events PASS Camp 2017 Main Camp 05.12. 07.12.2017 (04.12. Kick-Off abends) Lufthansa Training &

More information

Microsoft Analytics Platform System (APS)

Microsoft Analytics Platform System (APS) Microsoft Analytics Platform System (APS) The turnkey modern data warehouse appliance Matt Usher, Senior Program Manager @ Microsoft About.me @two_under Senior Program Manager 9 years at Microsoft Visual

More information

Data Architectures in Azure for Analytics & Big Data

Data Architectures in Azure for Analytics & Big Data Data Architectures in for Analytics & Big Data October 20, 2018 Melissa Coates Solution Architect, BlueGranite Microsoft Data Platform MVP Blog: www.sqlchick.com Twitter: @sqlchick Data Architecture A

More information

Duration: 5 Days. EZY Intellect Pte. Ltd.,

Duration: 5 Days. EZY Intellect Pte. Ltd., Implementing a SQL Data Warehouse Duration: 5 Days Course Code: 20767A Course review About this course This 5-day instructor led course describes how to implement a data warehouse platform to support a

More information

Increase Value from Big Data with Real-Time Data Integration and Streaming Analytics

Increase Value from Big Data with Real-Time Data Integration and Streaming Analytics Increase Value from Big Data with Real-Time Data Integration and Streaming Analytics Cy Erbay Senior Director Striim Executive Summary Striim is Uniquely Qualified to Solve the Challenges of Real-Time

More information

Microsoft Developer Day

Microsoft Developer Day Microsoft Developer Day Pradeep Menon Microsoft Developer Day Solutions Architect Agenda Microsoft Developer Day Traditional Business Intelligence Architecture Structured Sources Extract Transform Structurize

More information

Modern Data Warehouse The New Approach to Azure BI

Modern Data Warehouse The New Approach to Azure BI Modern Data Warehouse The New Approach to Azure BI History On-Premise SQL Server Big Data Solutions Technical Barriers Modern Analytics Platform On-Premise SQL Server Big Data Solutions Modern Analytics

More information

Stages of Data Processing

Stages of Data Processing Data processing can be understood as the conversion of raw data into a meaningful and desired form. Basically, producing information that can be understood by the end user. So then, the question arises,

More information

Azure Data Factory. Data Integration in the Cloud

Azure Data Factory. Data Integration in the Cloud Azure Data Factory Data Integration in the Cloud 2018 Microsoft Corporation. All rights reserved. This document is provided "as-is." Information and views expressed in this document, including URL and

More information

Ian Choy. Technology Solutions Professional

Ian Choy. Technology Solutions Professional Ian Choy Technology Solutions Professional XML KPIs SQL Server 2000 Management Studio Mirroring SQL Server 2005 Compression Policy-Based Mgmt Programmability SQL Server 2008 PowerPivot SharePoint Integration

More information

Overview of Data Services and Streaming Data Solution with Azure

Overview of Data Services and Streaming Data Solution with Azure Overview of Data Services and Streaming Data Solution with Azure Tara Mason Senior Consultant tmason@impactmakers.com Platform as a Service Offerings SQL Server On Premises vs. Azure SQL Server SQL Server

More information

USERS CONFERENCE Copyright 2016 OSIsoft, LLC

USERS CONFERENCE Copyright 2016 OSIsoft, LLC Bridge IT and OT with a process data warehouse Presented by Matt Ziegler, OSIsoft Complexity Problem Complexity Drives the Need for Integrators Disparate assets or interacting one-by-one Monitoring Real-time

More information

Updating your Business Intelligence Skills to Microsoft SQL Server 2012

Updating your Business Intelligence Skills to Microsoft SQL Server 2012 Course 40009A: Updating your Business Intelligence Skills to Microsoft SQL Server 2012 Course Details Course Outline Module 1: Introduction to SQL Server 2012 for Business Intelligence This module provides

More information

Oliver Engels & Tillmann Eitelberg. Big Data! Big Quality?

Oliver Engels & Tillmann Eitelberg. Big Data! Big Quality? Oliver Engels & Tillmann Eitelberg Big Data! Big Quality? Like to visit Germany? PASS Camp 2017 Main Camp 5.12 7.12.2017 (4.12 Kick Off Evening) Lufthansa Training & Conference Center, Seeheim SQL Konferenz

More information

What is Gluent? The Gluent Data Platform

What is Gluent? The Gluent Data Platform What is Gluent? The Gluent Data Platform The Gluent Data Platform provides a transparent data virtualization layer between traditional databases and modern data storage platforms, such as Hadoop, in the

More information

Ayush Ganeriwal Senior Principal Product Manager, Oracle. Benjamin Perez-Goytia Principal Solution Architect A-Team, Oracle

Ayush Ganeriwal Senior Principal Product Manager, Oracle. Benjamin Perez-Goytia Principal Solution Architect A-Team, Oracle Oracle Data Integration Platform A Cornerstone for Big Data Ayush Ganeriwal Senior Principal Product Manager, Oracle Benjamin Perez-Goytia Principal Solution Architect A-Team, Oracle Pencho Tzonev Head

More information

Alexander Klein. ETL in the Cloud

Alexander Klein. ETL in the Cloud Alexander Klein ETL in the Cloud Sponsors help us to run this event! THX! You Rock! Sponsor Gold Sponsor Silver Sponsor Bronze Sponsor You Rock! Sponsor Session 13:45 Track 1 Das super nerdige Solisyon

More information

Modeling. Preparation. Operationalization. Profile Explore. Model Testing & Validation. Feature & Algorithm Selection. Transform Cleanse Denormalize

Modeling. Preparation. Operationalization. Profile Explore. Model Testing & Validation. Feature & Algorithm Selection. Transform Cleanse Denormalize Preparation Modeling Ingest Transform Cleanse Denormalize Profile Explore Visualize Feature & Algorithm Selection Model Testing & Validation Operationalization Models Visualizations Deploy Apps, Services

More information

Data 101 Which DB, When. Joe Yong Azure SQL Data Warehouse, Program Management Microsoft Corp.

Data 101 Which DB, When. Joe Yong Azure SQL Data Warehouse, Program Management Microsoft Corp. Data 101 Which DB, When Joe Yong (joeyong@microsoft.com) Azure SQL Data Warehouse, Program Management Microsoft Corp. The world is changing AI increased by 300% in 2017 Data will grow to 44 ZB in 2020

More information

Implementing a SQL Data Warehouse

Implementing a SQL Data Warehouse Implementing a SQL Data Warehouse 20767B; 5 days, Instructor-led Course Description This 4-day instructor led course describes how to implement a data warehouse platform to support a BI solution. Students

More information

Implementing a SQL Data Warehouse

Implementing a SQL Data Warehouse Implementing a SQL Data Warehouse Course 20767B 5 Days Instructor-led, Hands on Course Information This five-day instructor-led course provides students with the knowledge and skills to provision a Microsoft

More information

microsoft

microsoft 70-775.microsoft Number: 70-775 Passing Score: 800 Time Limit: 120 min Exam A QUESTION 1 Note: This question is part of a series of questions that present the same scenario. Each question in the series

More information

Transitioning From SSIS to Azure Data Factory. Meagan Longoria, Solution Architect, BlueGranite

Transitioning From SSIS to Azure Data Factory. Meagan Longoria, Solution Architect, BlueGranite Transitioning From SSIS to Azure Data Factory Meagan Longoria, Solution Architect, BlueGranite Microsoft Data Platform MVP I enjoy contributing to and learning from the Microsoft data community. Blogger

More information

Take P, R or U. and solve your data quality problems Oliver Engels & Tillmann Eitelberg, OH22

Take P, R or U. and solve your data quality problems Oliver Engels & Tillmann Eitelberg, OH22 Take P, R or U and solve your data quality problems Oliver Engels & Tillmann Eitelberg, OH22 Oliver Engels CEO, oh22data AG @oengels Datamonster from Germany MS Data Platform MVP President of PASS Germany

More information

White Paper / Azure Data Platform: Ingest

White Paper / Azure Data Platform: Ingest White Paper / Azure Data Platform: Ingest Contents White Paper / Azure Data Platform: Ingest... 1 Versioning... 2 Meta Data... 2 Foreword... 3 Prerequisites... 3 Azure Data Platform... 4 Flowchart Guidance...

More information

Updating your Business Intelligence Skills to Microsoft SQL Server 2012 Course 40009A; 3 Days, Instructor-led

Updating your Business Intelligence Skills to Microsoft SQL Server 2012 Course 40009A; 3 Days, Instructor-led Updating your Business Intelligence Skills to Microsoft SQL Server 2012 Course 40009A; 3 Days, Instructor-led Course Description This three-day instructor-led course provides existing SQL Server Business

More information

Microsoft Implementing a SQL Data Warehouse

Microsoft Implementing a SQL Data Warehouse 1800 ULEARN (853 276) www.ddls.com.au Microsoft 20767 - Implementing a SQL Data Warehouse Length 5 days Price $4290.00 (inc GST) Version C Overview This five-day instructor-led course provides students

More information

Data 101 Which DB, When Joe Yong Sr. Program Manager Microsoft Corp.

Data 101 Which DB, When Joe Yong Sr. Program Manager Microsoft Corp. 17-18 March, 2018 Beijing Data 101 Which DB, When Joe Yong Sr. Program Manager Microsoft Corp. The world is changing AI increased by 300% in 2017 Data will grow to 44 ZB in 2020 Today, 80% of organizations

More information

Welcome! Power BI User Group (PUG) Copenhagen

Welcome! Power BI User Group (PUG) Copenhagen Welcome! Power BI User Group (PUG) Copenhagen Connect to Data in Power BI Desktop Just Thorning Blindbæk Consultant, Trainer and Speaker Connect to Data in Power BI Desktop Basic introduction to data connectivity

More information

Exam /Course 20767B: Implementing a SQL Data Warehouse

Exam /Course 20767B: Implementing a SQL Data Warehouse Exam 70-767/Course 20767B: Implementing a SQL Data Warehouse Course Outline Module 1: Introduction to Data Warehousing This module describes data warehouse concepts and architecture consideration. Overview

More information

Oliver Engels & Tillmann Eitelberg. Big Data! Big Quality?

Oliver Engels & Tillmann Eitelberg. Big Data! Big Quality? Oliver Engels & Tillmann Eitelberg Big Data! Big Quality? Sponsors help us to run this event! THX! You Rock! Sponsor Gold Sponsor Silver Sponsor Bronze Sponsor You Rock! Sponsor Session 13:45 Track 1 Das

More information

28 February 1 March 2018, Trafo Baden. #techsummitch

28 February 1 March 2018, Trafo Baden. #techsummitch #techsummitch 28 February 1 March 2018, Trafo Baden #techsummitch Transform your data estate with cloud, data and AI #techsummitch The world is changing Data will grow to 44 ZB in 2020 Today, 80% of organizations

More information

Microsoft Exam

Microsoft Exam Volume: 42 Questions Case Study: 1 Relecloud General Overview Relecloud is a social media company that processes hundreds of millions of social media posts per day and sells advertisements to several hundred

More information

Asanka Padmakumara. ETL 2.0: Data Engineering with Azure Databricks

Asanka Padmakumara. ETL 2.0: Data Engineering with Azure Databricks Asanka Padmakumara ETL 2.0: Data Engineering with Azure Databricks Who am I? Asanka Padmakumara Business Intelligence Consultant, More than 8 years in BI and Data Warehousing A regular speaker in data

More information

HDInsight > Hadoop. October 12, 2017

HDInsight > Hadoop. October 12, 2017 HDInsight > Hadoop October 12, 2017 2 Introduction Mark Hudson >20 years mixing technology with data >10 years with CapTech Microsoft Certified IT Professional Business Intelligence Member of the Richmond

More information

@Pentaho #BigDataWebSeries

@Pentaho #BigDataWebSeries Enterprise Data Warehouse Optimization with Hadoop Big Data @Pentaho #BigDataWebSeries Your Hosts Today Dave Henry SVP Enterprise Solutions Davy Nys VP EMEA & APAC 2 Source/copyright: The Human Face of

More information

Processing Unstructured Data. Dinesh Priyankara Founder/Principal Architect dinesql Pvt Ltd.

Processing Unstructured Data. Dinesh Priyankara Founder/Principal Architect dinesql Pvt Ltd. Processing Unstructured Data Dinesh Priyankara Founder/Principal Architect dinesql Pvt Ltd. http://dinesql.com / Dinesh Priyankara @dinesh_priya Founder/Principal Architect dinesql Pvt Ltd. Microsoft Most

More information

Implementing a SQL Data Warehouse (20767)

Implementing a SQL Data Warehouse (20767) Implementing a SQL Data Warehouse (20767) Formato do curso: Presencial e Live Training Localidade: Lisboa Data: 11 Set. 2017 a 22 Set. 2017 Preço: 1630 Horário: Pós-laboral - das 18:30 às 21:30 Duração:

More information

From Single Purpose to Multi Purpose Data Lakes. Thomas Niewel Technical Sales Director DACH Denodo Technologies March, 2019

From Single Purpose to Multi Purpose Data Lakes. Thomas Niewel Technical Sales Director DACH Denodo Technologies March, 2019 From Single Purpose to Multi Purpose Data Lakes Thomas Niewel Technical Sales Director DACH Denodo Technologies March, 2019 Agenda Data Lakes Multiple Purpose Data Lakes Customer Example Demo Takeaways

More information

Things I Learned The Hard Way About Azure Data Platform Services So You Don t Have To -Meagan Longoria

Things I Learned The Hard Way About Azure Data Platform Services So You Don t Have To -Meagan Longoria Things I Learned The Hard Way About Azure Data Platform Services So You Don t Have To -Meagan Longoria 2 University of Nebraska at Omaha Special thanks to UNO and the College of Business Administration

More information

20767B: IMPLEMENTING A SQL DATA WAREHOUSE

20767B: IMPLEMENTING A SQL DATA WAREHOUSE ABOUT THIS COURSE This 5-day instructor led course describes how to implement a data warehouse platform to support a BI solution. Students will learn how to create a data warehouse with Microsoft SQL Server

More information

Přehled novinek v SQL Server 2016

Přehled novinek v SQL Server 2016 Přehled novinek v SQL Server 2016 Martin Rys, BI Competency Leader martin.rys@adastragrp.com https://www.linkedin.com/in/martinrys 20.4.2016 1 BI Competency development 2 Trends, modern data warehousing

More information

Modern ETL Tools for Cloud and Big Data. Ken Beutler, Principal Product Manager, Progress Michael Rainey, Technical Advisor, Gluent Inc.

Modern ETL Tools for Cloud and Big Data. Ken Beutler, Principal Product Manager, Progress Michael Rainey, Technical Advisor, Gluent Inc. Modern ETL Tools for Cloud and Big Data Ken Beutler, Principal Product Manager, Progress Michael Rainey, Technical Advisor, Gluent Inc. Agenda Landscape Cloud ETL Tools Big Data ETL Tools Best Practices

More information

20767: Implementing a SQL Data Warehouse

20767: Implementing a SQL Data Warehouse Let s Reach For Excellence! TAN DUC INFORMATION TECHNOLOGY SCHOOL JSC Address: 103 Pasteur, Dist.1, HCMC Tel: 08 38245819; 38239761 Email: traincert@tdt-tanduc.com Website: www.tdt-tanduc.com; www.tanducits.com

More information

17/05/2017. What we ll cover. Who is Greg? Why PaaS and SaaS? What we re not discussing: IaaS

17/05/2017. What we ll cover. Who is Greg? Why PaaS and SaaS? What we re not discussing: IaaS What are all those Azure* and Power* services and why do I want them? Dr Greg Low SQL Down Under greg@sqldownunder.com Who is Greg? CEO and Principal Mentor at SDU Data Platform MVP Microsoft Regional

More information

Cortana Intelligence Suite; Where the Magic Happens

Cortana Intelligence Suite; Where the Magic Happens Cortana Intelligence Suite; Where the Magic Happens Reza Rad, Leila Etaati #509 Brisbane 2016 About Us Reza Rad Leila Etaati MVP BI Consultant and Trainer Author of Books Speaker in conferences; PASS Summit,

More information

Compact Solutions Connector FAQ

Compact Solutions Connector FAQ Compact Solutions Connector FAQ We Solve Problems Others Can t Experts for over 15 years providing solutions in the data transformation and management fields Passion for cutting-edge technology and the

More information

Big Data Technology Ecosystem. Mark Burnette Pentaho Director Sales Engineering, Hitachi Vantara

Big Data Technology Ecosystem. Mark Burnette Pentaho Director Sales Engineering, Hitachi Vantara Big Data Technology Ecosystem Mark Burnette Pentaho Director Sales Engineering, Hitachi Vantara Agenda End-to-End Data Delivery Platform Ecosystem of Data Technologies Mapping an End-to-End Solution Case

More information

Modernizing Business Intelligence and Analytics

Modernizing Business Intelligence and Analytics Modernizing Business Intelligence and Analytics Justin Erickson Senior Director, Product Management 1 Agenda What benefits can I achieve from modernizing my analytic DB? When and how do I migrate from

More information

Implementing a SQL Data Warehouse

Implementing a SQL Data Warehouse Course 20767B: Implementing a SQL Data Warehouse Page 1 of 7 Implementing a SQL Data Warehouse Course 20767B: 4 days; Instructor-Led Introduction This 4-day instructor led course describes how to implement

More information

Streaming Integration and Intelligence For Automating Time Sensitive Events

Streaming Integration and Intelligence For Automating Time Sensitive Events Streaming Integration and Intelligence For Automating Time Sensitive Events Ted Fish Director Sales, Midwest ted@striim.com 312-330-4929 Striim Executive Summary Delivering Data for Time Sensitive Processes

More information

Making Data Integration Easy For Multiplatform Data Architectures With Diyotta 4.0. WEBINAR MAY 15 th, PM EST 10AM PST

Making Data Integration Easy For Multiplatform Data Architectures With Diyotta 4.0. WEBINAR MAY 15 th, PM EST 10AM PST Making Data Integration Easy For Multiplatform Data Architectures With Diyotta 4.0 WEBINAR MAY 15 th, 2018 1PM EST 10AM PST Welcome and Logistics If you have problems with the sound on your computer, switch

More information

Developing in Power BI. with Streaming Datasets and Real-time Dashboards

Developing in Power BI. with Streaming Datasets and Real-time Dashboards Developing in Power BI with Streaming Datasets and Real-time Dashboards Code and Slides for this Session https://github.com/criticalpathtraining/realtimedashboards Critical Path Training https://www.criticalpathtrainig.com

More information

Best practices for building a Hadoop Data Lake Solution CHARLOTTE HADOOP USER GROUP

Best practices for building a Hadoop Data Lake Solution CHARLOTTE HADOOP USER GROUP Best practices for building a Hadoop Data Lake Solution CHARLOTTE HADOOP USER GROUP 07.29.2015 LANDING STAGING DW Let s start with something basic Is Data Lake a new concept? What is the closest we can

More information

ETL Best Practices and Techniques. Marc Beacom, Managing Partner, Datalere

ETL Best Practices and Techniques. Marc Beacom, Managing Partner, Datalere ETL Best Practices and Techniques Marc Beacom, Managing Partner, Datalere Thank you Sponsors Experience 10 years DW/BI Consultant 20 Years overall experience Marc Beacom Managing Partner, Datalere Current

More information

Azure SQL Data Warehouse. Andrija Marcic Microsoft

Azure SQL Data Warehouse. Andrija Marcic Microsoft Azure SQL Data Warehouse Andrija Marcic Microsoft End to end platform built for the cloud Hadoop SQL Azure SQL Data Warehouse Azure SQL Database App Service Intelligent App Azure Machine Learning Power

More information

IT directors, CIO s, IT Managers, BI Managers, data warehousing professionals, data scientists, enterprise architects, data architects

IT directors, CIO s, IT Managers, BI Managers, data warehousing professionals, data scientists, enterprise architects, data architects Organised by: www.unicom.co.uk OVERVIEW This two day workshop is aimed at getting Data Scientists, Data Warehousing and BI professionals up to scratch on Big Data, Hadoop, other NoSQL DBMSs and Multi-Platform

More information

VOLTDB + HP VERTICA. page

VOLTDB + HP VERTICA. page VOLTDB + HP VERTICA ARCHITECTURE FOR FAST AND BIG DATA ARCHITECTURE FOR FAST + BIG DATA FAST DATA Fast Serve Analytics BIG DATA BI Reporting Fast Operational Database Streaming Analytics Columnar Analytics

More information

SQL Server SQL Server 2008 and 2008 R2. SQL Server SQL Server 2014 Currently supporting all versions July 9, 2019 July 9, 2024

SQL Server SQL Server 2008 and 2008 R2. SQL Server SQL Server 2014 Currently supporting all versions July 9, 2019 July 9, 2024 Current support level End Mainstream End Extended SQL Server 2005 SQL Server 2008 and 2008 R2 SQL Server 2012 SQL Server 2005 SP4 is in extended support, which ends on April 12, 2016 SQL Server 2008 and

More information

Implementing a Data Warehouse with Microsoft SQL Server 2014 (20463D)

Implementing a Data Warehouse with Microsoft SQL Server 2014 (20463D) Implementing a Data Warehouse with Microsoft SQL Server 2014 (20463D) Overview This course describes how to implement a data warehouse platform to support a BI solution. Students will learn how to create

More information

BI ENVIRONMENT PLANNING GUIDE

BI ENVIRONMENT PLANNING GUIDE BI ENVIRONMENT PLANNING GUIDE Business Intelligence can involve a number of technologies and foster many opportunities for improving your business. This document serves as a guideline for planning strategies

More information

Introduction to SSIS. Or you want to take some data, change it, and put it somewhere else? Then boy do I have THE tool for you!

Introduction to SSIS. Or you want to take some data, change it, and put it somewhere else? Then boy do I have THE tool for you! Introduction to SSIS Or you want to take some data, change it, and put it somewhere else? Then boy do I have THE tool for you! Who am I? Ed Watson Data Services Consultant or Ambassador of Mayhem Twitter:

More information

The Cortana Intelligence Suite

The Cortana Intelligence Suite Slide 1 The Cortana Intelligence Suite Foundations Data Discovery and Ingestion Microsoft Machine Learning and Data Science Team CortanaIntelligence.com Main page: http://cortanaanalytics.com To begin

More information

Copyright 2016 Datalynx Pty Ltd. All rights reserved. Datalynx Enterprise Data Management Solution Catalogue

Copyright 2016 Datalynx Pty Ltd. All rights reserved. Datalynx Enterprise Data Management Solution Catalogue Datalynx Enterprise Data Management Solution Catalogue About Datalynx Vendor of the world s most versatile Enterprise Data Management software Licence our software to clients & partners Partner-based sales

More information

COURSE 10977A: UPDATING YOUR SQL SERVER SKILLS TO MICROSOFT SQL SERVER 2014

COURSE 10977A: UPDATING YOUR SQL SERVER SKILLS TO MICROSOFT SQL SERVER 2014 ABOUT THIS COURSE This five-day instructor-led course teaches students how to use the enhancements and new features that have been added to SQL Server and the Microsoft data platform since the release

More information

Franck Mercier. Technical Solution Professional Data + AI Azure Databricks

Franck Mercier. Technical Solution Professional Data + AI Azure Databricks Franck Mercier Technical Solution Professional Data + AI http://aka.ms/franck @FranmerMS Azure Databricks Thanks to our sponsors Global Gold Silver Bronze Microsoft JetBrains Rubrik Delphix Solution OMD

More information

Training 24x7 DBA Support Staffing. MCSA:SQL 2016 Business Intelligence Development. Implementing an SQL Data Warehouse. (40 Hours) Exam

Training 24x7 DBA Support Staffing. MCSA:SQL 2016 Business Intelligence Development. Implementing an SQL Data Warehouse. (40 Hours) Exam MCSA:SQL 2016 Business Intelligence Development Implementing an SQL Data Warehouse (40 Hours) Exam 70-767 Prerequisites At least 2 years experience of working with relational databases, including: Designing

More information

Microsoft. Perform Data Engineering on Microsoft Azure HDInsight Version: Demo. Web: [ Total Questions: 10]

Microsoft. Perform Data Engineering on Microsoft Azure HDInsight Version: Demo. Web:   [ Total Questions: 10] Microsoft 70-775 Perform Data Engineering on Microsoft Azure HDInsight Web: www.marks4sure.com Email: support@marks4sure.com Version: Demo [ Total Questions: 10] IMPORTANT NOTICE Feedback We have developed

More information

Microsoft Perform Data Engineering on Microsoft Azure HDInsight.

Microsoft Perform Data Engineering on Microsoft Azure HDInsight. Microsoft 70-775 Perform Data Engineering on Microsoft Azure HDInsight http://killexams.com/pass4sure/exam-detail/70-775 QUESTION: 30 You are building a security tracking solution in Apache Kafka to parse

More information

Index. Scott Klein 2017 S. Klein, IoT Solutions in Microsoft s Azure IoT Suite, DOI /

Index. Scott Klein 2017 S. Klein, IoT Solutions in Microsoft s Azure IoT Suite, DOI / Index A Advanced Message Queueing Protocol (AMQP), 44 Analytics, 9 Apache Ambari project, 209 210 API key, 244 Application data, 4 Azure Active Directory (AAD), 91, 257 Azure Blob Storage, 191 Azure data

More information

Capture Business Opportunities from Systems of Record and Systems of Innovation

Capture Business Opportunities from Systems of Record and Systems of Innovation Capture Business Opportunities from Systems of Record and Systems of Innovation Amit Satoor, SAP March Hartz, SAP PUBLIC Big Data transformation powers digital innovation system Relevant nuggets of information

More information

exam. Microsoft Perform Data Engineering on Microsoft Azure HDInsight. Version 1.0

exam.   Microsoft Perform Data Engineering on Microsoft Azure HDInsight. Version 1.0 70-775.exam Number: 70-775 Passing Score: 800 Time Limit: 120 min File Version: 1.0 Microsoft 70-775 Perform Data Engineering on Microsoft Azure HDInsight Version 1.0 Exam A QUESTION 1 You use YARN to

More information

Leverage the Oracle Data Integration Platform Inside Azure and Amazon Cloud

Leverage the Oracle Data Integration Platform Inside Azure and Amazon Cloud Leverage the Oracle Data Integration Platform Inside Azure and Amazon Cloud WHITE PAPER / AUGUST 8, 2018 DISCLAIMER The following is intended to outline our general product direction. It is intended for

More information

Implementing a Data Warehouse with Microsoft SQL Server 2012

Implementing a Data Warehouse with Microsoft SQL Server 2012 10777 - Implementing a Data Warehouse with Microsoft SQL Server 2012 Duration: 5 days Course Price: $2,695 Software Assurance Eligible Course Description 10777 - Implementing a Data Warehouse with Microsoft

More information

Neues Dream Team Azure Data Factory v2 und SSIS

Neues Dream Team Azure Data Factory v2 und SSIS Neues Dream Team Azure Data Factory v2 und SSIS PASS Regionalgruppe Hamburg Stefan Kirner 8.2.2018 Agenda 1. Target Scenarios 2. Current State 3. Intro Data Factory v2 4. Triggers 5. Control Flow 6. Integration

More information

Migrating Enterprise BI to Azure

Migrating Enterprise BI to Azure Migrating Enterprise BI to Azure Best Practices Wlodek Bielski SQLSat Kyiv Team Yevhen Nedashkivskyi Mykola Pobyivovk Denis Reznik Eugene Polonichko Oksana Borysenko Oksana Tkach Sponsors Session will

More information

Open Source Tools as a platform for research on Microsoft Azure

Open Source Tools as a platform for research on Microsoft Azure Open Source Tools as a platform for research on Microsoft Azure Alessandro Jannuzi Open Source Lead Microsoft Brasil Jaime Puente Director Microsoft Research Azure, Microsoft Cloud Platform 24 Regions

More information

##SQLSatMadrid. Project [Vélib by Cortana]

##SQLSatMadrid. Project [Vélib by Cortana] Project [Vélib by Cortana] BIG Thanks to SQLSatMadrid Sponsors Speakers Agenda Presentation of the Project Cortana Intelligent Suite Creation of the architecture Purpose of the Project Get a descriptive

More information

Azure Learning Circles

Azure Learning Circles Azure Learning Circles Azure Management Session 1: Logs, Diagnostics & Metrics Presented By: Shane Creamer shanec@microsoft.com Typical Customer Narratives Most customers know how to operate on-premises,

More information

Data Warehouse Design Decisions

Data Warehouse Design Decisions Data Warehouse Design Decisions August 2015 Colleen Barnitz Director, IT Development MVT Services Colleen Barnitz over 20 Years in IT worked with SQL Server since version 6.5 developer and an architect

More information

SQL Server Pre Lanzamiento. Federico Marty. Mariano Kovo. Especialista en Plataforma de Aplicaciones Microsoft Argentina & Uruguay

SQL Server Pre Lanzamiento. Federico Marty. Mariano Kovo. Especialista en Plataforma de Aplicaciones Microsoft Argentina & Uruguay 2016 Pre Lanzamiento Federico Marty Especialista en Plataforma de Aplicaciones Microsoft Argentina & Uruguay Mariano Kovo Especialista en Precision IT 2016: Everything built-in built-in built-in built-in

More information

NYC Cloud Machine Learning Meetup. Introduction to Cortana Analytics

NYC Cloud Machine Learning Meetup. Introduction to Cortana Analytics NYC Cloud Machine Learning Meetup Dec 1, 2015 Introduction to Cortana Analytics note: deck will be posted to the meetup site more->files tab. Topics What is Cortana Analytics What are the core components

More information

Activator Library. Focus on maximizing the value of your data, gain business insights, increase your team s productivity, and achieve success.

Activator Library. Focus on maximizing the value of your data, gain business insights, increase your team s productivity, and achieve success. Focus on maximizing the value of your data, gain business insights, increase your team s productivity, and achieve success. ACTIVATORS Designed to give your team assistance when you need it most without

More information

Accelerate Your Data Pipeline for Data Lake, Streaming and Cloud Architectures

Accelerate Your Data Pipeline for Data Lake, Streaming and Cloud Architectures WHITE PAPER : REPLICATE Accelerate Your Data Pipeline for Data Lake, Streaming and Cloud Architectures INTRODUCTION Analysis of a wide variety of data is becoming essential in nearly all industries to

More information

Data-Intensive Distributed Computing

Data-Intensive Distributed Computing Data-Intensive Distributed Computing CS 451/651 431/631 (Winter 2018) Part 5: Analyzing Relational Data (1/3) February 8, 2018 Jimmy Lin David R. Cheriton School of Computer Science University of Waterloo

More information

WHITEPAPER. MemSQL Enterprise Feature List

WHITEPAPER. MemSQL Enterprise Feature List WHITEPAPER MemSQL Enterprise Feature List 2017 MemSQL Enterprise Feature List DEPLOYMENT Provision and deploy MemSQL anywhere according to your desired cluster configuration. On-Premises: Maximize infrastructure

More information

Implementing a Data Warehouse with Microsoft SQL Server 2014

Implementing a Data Warehouse with Microsoft SQL Server 2014 Course 20463D: Implementing a Data Warehouse with Microsoft SQL Server 2014 Page 1 of 5 Implementing a Data Warehouse with Microsoft SQL Server 2014 Course 20463D: 4 days; Instructor-Led Introduction This

More information

The Enterprise Data Marketplace

The Enterprise Data Marketplace The Enterprise Data Marketplace Rick F. van der Lans Industry analyst Email rick@r20.nl Twitter @rick_vanderlans www.r20.nl Copyright 2018 R20/Consultancy B.V., The Netherlands. All rights reserved. No

More information

Implementing a Data Warehouse with Microsoft SQL Server 2012

Implementing a Data Warehouse with Microsoft SQL Server 2012 Implementing a Data Warehouse with Microsoft SQL Server 2012 Course 10777A 5 Days Instructor-led, Hands-on Introduction Data warehousing is a solution organizations use to centralize business data for

More information

Microsoft Big Data and Hadoop

Microsoft Big Data and Hadoop Microsoft Big Data and Hadoop Lara Rubbelke @sqlgal Cindy Gross @sqlcindy 2 The world of data is changing The 4Vs of Big Data http://nosql.mypopescu.com/post/9621746531/a-definition-of-big-data 3 Common

More information

Understanding the latent value in all content

Understanding the latent value in all content Understanding the latent value in all content John F. Kennedy (JFK) November 22, 1963 INGEST ENRICH EXPLORE Cognitive skills Data in any format, any Azure store Search Annotations Data Cloud Intelligence

More information

Evolution of Big Data Facebook. Architecture Summit, Shenzhen, August 2012 Ashish Thusoo

Evolution of Big Data Facebook. Architecture Summit, Shenzhen, August 2012 Ashish Thusoo Evolution of Big Data Architectures@ Facebook Architecture Summit, Shenzhen, August 2012 Ashish Thusoo About Me Currently Co-founder/CEO of Qubole Ran the Data Infrastructure Team at Facebook till 2011

More information

Cloud has become the New Normal

Cloud has become the New Normal Yip In Tsoi Cloud Cloud has become the New Normal By 2020 the distinction between public and private cloud disappears as self-built private clouds become extinct #idcgrac @craw Crawford Del Prete EVP,

More information

DURATION : 03 DAYS. same along with BI tools.

DURATION : 03 DAYS. same along with BI tools. AWS REDSHIFT TRAINING MILDAIN DURATION : 03 DAYS To benefit from this Amazon Redshift Training course from mildain, you will need to have basic IT application development and deployment concepts, and good

More information

Bridge the cloud divide with hybrid business intelligence in SharePoint 2016 and Office 365

Bridge the cloud divide with hybrid business intelligence in SharePoint 2016 and Office 365 Bridge the cloud divide with hybrid business intelligence in SharePoint 2016 and Office 365 Chris McNulty, Microsoft John White, MVP John White CTO/Co-Founder of UnlimitedViz MSc. MVP SQL Server v-ts

More information

MCSA SQL SERVER 2012

MCSA SQL SERVER 2012 MCSA SQL SERVER 2012 1. Course 10774A: Querying Microsoft SQL Server 2012 Course Outline Module 1: Introduction to Microsoft SQL Server 2012 Introducing Microsoft SQL Server 2012 Getting Started with SQL

More information