Open Source Tools as a platform for research on Microsoft Azure

Size: px

Start display at page:

Download "Open Source Tools as a platform for research on Microsoft Azure"

Kathryn O’Connor’
5 years ago
Views:

1 Open Source Tools as a platform for research on Microsoft Azure Alessandro Jannuzi Open Source Lead Microsoft Brasil Jaime Puente Director Microsoft Research

Azure, Microsoft Cloud Platform 24 Regions Worldwide, 19 ONLINE huge capacity around the world growing every year West US California

Quebec City East US Virginia East US 2 Virginia North Europe Ireland West Europe Netherlands India West Mumbai India Central Pune

Singapore Australia East New South Wales Brazil South Sao Paulo 100+ datacenters Top 3 networks in the world 2x AWS, 6x Google DC

2 Azure, Microsoft Cloud Platform 24 Regions Worldwide, 19 ONLINE huge capacity around the world growing every year West US California US Gov Iowa Central US Iowa South Central US Texas North Central US Illinois Canada Central Toronto US Gov Virginia Canada East Quebec City East US Virginia East US 2 Virginia North Europe Ireland West Europe Netherlands India West Mumbai India Central Pune China South * Shanghai India South Chennai China North * Beijing East Asia Hong Kong Japan East Saitama Japan West Osaka SE Asia Singapore Australia East New South Wales Brazil South Sao Paulo 100+ datacenters Top 3 networks in the world 2x AWS, 6x Google DC Regions G Series Largest VM in World, 32 cores, 448GB Ram, SSD Operational Announced/Not Operational * Operated by 21Vianet Australia South East Victoria

3 Platform Services Security & Management Portal Active Directory Cloud Services Batch Service Fabric Remote App Web Apps Mobile Apps API Apps Logic Apps API Management Notification Hubs Visual Studio Team Project Azure SDK Application Insights Hybrid Operations Azure AD Connect Health AD Privileged Identity Management Multi-Factor Authentication Backup Automation Storage Queues Biztalk Services HDInsight Machine Learning SQL Database SQL Data Warehouse Operational Insights Key Vault Hybrid Connections Service Bus Data Factory Event Hubs Redis Cache Search Import/Export Store / Marketplace VM Image Gallery & VM Depot Media Services Content Delivery Network (CDN) Stream Analytics Mobile Engagement DocumentDB Tables Site Recovery StorSimple Infrastructure Services

4 Old and new trends? Machine Learning Internet of Things Móvel Social corporativo Big data Nuvem Next-Gen Architectures DevOps

Flat files, raw text, system logs, images, audio, video Parse, cleanse, calculate, aggregate Real time and Batch

Databases Performance Management Reporting, Analysis Files, XLS External feed Data Marts Geospatial visualizations

Summaries Complex analytical and statistical processing Derived insights High end Business analytics, actionable

Discovery Risk analytics Fraud detection, prevention Logs Social media, Social network analysis Business strategy,

5 Flat files, raw text, system logs, images, audio, video Parse, cleanse, calculate, aggregate Real time and Batch Data load ASCII Files, DB feeds, XML, Extract, Cleanse, Integrate, Load Real time and Batch Data load Metadata/Semantic Layer Data! Structured or not, always welcome Traditional Data sources ERP EDW PLATFORMS CRM Custom App Operations DWH Databases Performance Management Reporting, Analysis Files, XLS External feed Data Marts Geospatial visualizations Business analytics Enterprise transactional data Consolidated EDW data feeds BI access for Big Data Derived Summaries Complex analytical and statistical processing Derived insights High end Business analytics, actionable insight New Data sources Sensors Streams BIG DATA PLATFORMS High volume Complex aggregations Rule / Pattern Discovery Risk analytics Fraud detection, prevention Logs Social media, Social network analysis Business strategy, evaluation Apps Bots Crawlers High performance clusters, Scale out/ NoSQL databases, MapReduce platforms Statistical modeling Data mining, analytics and the like Campaign modeling, analytics Profitability, churn analytics and more possibilities

6 Where to host it and how to handle it? Non-relational Relational HIVE Pig R Python Power Pivot Power Query Manage Streaming Process M/R Explore Excel Visualize Power Map Blobs Tables Data Exchange Workflow Azure Machine Learning 3rd party libraries Power BI Custom App Collect Integrate Predict Share

7 Machine Learning as a Service Input Dataset Feature Selection Algorithm Training set Train Model Testing set Age Workclass Education Occupation Sex Hours-per-week Income 39 State-gov Bachelors Adm-clerical Male 40 <=50K 50 Self-emp-not-inc Bachelors Exec-managerial Male 13 <=50K Score Model 38 Private HS-grad Handlers-cleaners Male 40 <=50K 53 Private 11th Handlers-cleaners Male 40 <=50K 28 Private Bachelors Prof-specialty Female 40 <=50K Evaluate Model 37 Private Masters Exec-managerial Female 40 <=50K Publish Model 49 Private 9th Other-service Female 16 <=50K

9 Microsoft + Open Source Momentum Tweet Cnet, Q&A Tweet Industry Leaders The Seattle Times

10 The Microsoft Open Approach For your journey to the cloud Empowering Customers By Enabling Choice To Provide a Trusted Cloud Freedom to Choose Freedom to Change Optimal Value Vibrant Local IT Economy X-Platform Open Standards Interoperability Open Source Ecosystem Engagement Secure Private Control Transparent

11 Microsoft Azure is an Open Cloud MS Integrated Ecosystem Provided Languages, Dev Tools & App Containers CMS & Apps Devices Databases Management Operating systems

12 Microsoft + Open Source Momentum

14 Academic partnerships Azure as a platform for research Engagement with Microsoft Labs and Researchers Fellowships for students Examples: UFMG, PUC-Rio, UFRGS and others

15 Cases UFMG Traffic jam prediction PUC-Rio Buses in Rio, IoMT Framework, others UFRGS Weather Forecast on Azure

17 Global datacenter footprint 100+ Datacenters in over 40 countries

Azure Everywhere. Brandon Murray, Cami Williams, David Haver, Kevin Carter, Russ Henderson

Azure Everywhere. Brandon Murray, Cami Williams, David Haver, Kevin Carter, Russ Henderson Azure Everywhere Brandon Murray, Cami Williams, David Haver, Kevin Carter, Russ Henderson Agenda Azure Everywhere Workshop Brief Overview of Azure Azure Infrastructure Azure DevOps SQL in Azure SharePoint