Open Source Tools as a platform for research on Microsoft Azure Alessandro Jannuzi Open Source Lead Microsoft Brasil Jaime Puente Director Microsoft Research
Azure, Microsoft Cloud Platform 24 Regions Worldwide, 19 ONLINE huge capacity around the world growing every year West US California US Gov Iowa Central US Iowa South Central US Texas North Central US Illinois Canada Central Toronto US Gov Virginia Canada East Quebec City East US Virginia East US 2 Virginia North Europe Ireland West Europe Netherlands India West Mumbai India Central Pune China South * Shanghai India South Chennai China North * Beijing East Asia Hong Kong Japan East Saitama Japan West Osaka SE Asia Singapore Australia East New South Wales Brazil South Sao Paulo 100+ datacenters Top 3 networks in the world 2x AWS, 6x Google DC Regions G Series Largest VM in World, 32 cores, 448GB Ram, SSD Operational Announced/Not Operational * Operated by 21Vianet Australia South East Victoria
Platform Services Security & Management Portal Active Directory Cloud Services Batch Service Fabric Remote App Web Apps Mobile Apps API Apps Logic Apps API Management Notification Hubs Visual Studio Team Project Azure SDK Application Insights Hybrid Operations Azure AD Connect Health AD Privileged Identity Management Multi-Factor Authentication Backup Automation Storage Queues Biztalk Services HDInsight Machine Learning SQL Database SQL Data Warehouse Operational Insights Key Vault Hybrid Connections Service Bus Data Factory Event Hubs Redis Cache Search Import/Export Store / Marketplace VM Image Gallery & VM Depot Media Services Content Delivery Network (CDN) Stream Analytics Mobile Engagement DocumentDB Tables Site Recovery StorSimple Infrastructure Services
Old and new trends? Machine Learning Internet of Things Móvel Social corporativo Big data Nuvem Next-Gen Architectures DevOps
Flat files, raw text, system logs, images, audio, video Parse, cleanse, calculate, aggregate Real time and Batch Data load ASCII Files, DB feeds, XML, Extract, Cleanse, Integrate, Load Real time and Batch Data load Metadata/Semantic Layer Data! Structured or not, always welcome Traditional Data sources ERP EDW PLATFORMS CRM Custom App Operations DWH Databases Performance Management Reporting, Analysis Files, XLS External feed Data Marts Geospatial visualizations Business analytics Enterprise transactional data Consolidated EDW data feeds BI access for Big Data Derived Summaries Complex analytical and statistical processing Derived insights High end Business analytics, actionable insight New Data sources Sensors Streams BIG DATA PLATFORMS High volume Complex aggregations Rule / Pattern Discovery Risk analytics Fraud detection, prevention Logs Social media, Social network analysis Business strategy, evaluation Apps Bots Crawlers High performance clusters, Scale out/ NoSQL databases, MapReduce platforms Statistical modeling Data mining, analytics and the like Campaign modeling, analytics Profitability, churn analytics and more possibilities
Where to host it and how to handle it? Non-relational Relational HIVE Pig R Python Power Pivot Power Query Manage Streaming Process M/R Explore Excel Visualize Power Map Blobs Tables Data Exchange Workflow Azure Machine Learning 3rd party libraries Power BI Custom App Collect Integrate Predict Share
Machine Learning as a Service Input Dataset Feature Selection Algorithm Training set Train Model Testing set Age Workclass Education Occupation Sex Hours-per-week Income 39 State-gov Bachelors Adm-clerical Male 40 <=50K 50 Self-emp-not-inc Bachelors Exec-managerial Male 13 <=50K Score Model 38 Private HS-grad Handlers-cleaners Male 40 <=50K 53 Private 11th Handlers-cleaners Male 40 <=50K 28 Private Bachelors Prof-specialty Female 40 <=50K Evaluate Model 37 Private Masters Exec-managerial Female 40 <=50K Publish Model 49 Private 9th Other-service Female 16 <=50K
Microsoft + Open Source Momentum Tweet Cnet, Q&A Tweet Industry Leaders The Seattle Times
The Microsoft Open Approach For your journey to the cloud Empowering Customers By Enabling Choice To Provide a Trusted Cloud Freedom to Choose Freedom to Change Optimal Value Vibrant Local IT Economy X-Platform Open Standards Interoperability Open Source Ecosystem Engagement Secure Private Control Transparent
Microsoft Azure is an Open Cloud MS Integrated Ecosystem Provided Languages, Dev Tools & App Containers CMS & Apps Devices Databases Management Operating systems
Microsoft + Open Source Momentum
Academic partnerships Azure as a platform for research Engagement with Microsoft Labs and Researchers Fellowships for students Examples: UFMG, PUC-Rio, UFRGS and others
Cases UFMG Traffic jam prediction PUC-Rio Buses in Rio, IoMT Framework, others UFRGS Weather Forecast on Azure
Global datacenter footprint 100+ Datacenters in over 40 countries