OPENSTACK BEIJING CONFERENCE. by: Steven Hallett Head of Cloud Infrastructure Engineering and Operations

Transcription

1 OPENSTACK BEIJING CONFERENCE by: Steven Hallett Head of Cloud Infrastructure Engineering and Operations August 10, 2012

2 Agenda X.commerce, an Introduction Platform Vision Current Status The Opportunity Before Us 2

3 ebay Inc. Enables Commerce Enable consumers to buy anything, anywhere, any how Enable merchants to succeed in new commerce environment X.commerce Goal: Build a platform that offers world-class, flexible and scalable commerce capabilities for businesses of all sizes 3

4 X.commerce: Largest Ecosystem of Commerce Capabilities Consumer Products Merchant Products Merchant Products Other Storefronts Platform X.Commerce Fabric and Open Commerce Language Commerce Capabilitie s Community 8/16/2012 OVER 1.2M DEVELOPERS 4 4

5 E-commerce Commerce Accelerating pace of innovation Blurring of online and offline commerce Today Tomorrow Global Mobile Local Social Online 6% Webinfluenced offline Offline 37% Online + Offline Service & delivery Digital 5

6 Merchants New Competitive Forces Consumer engagement & loyalty Multi-channel & devices Global reach and scale Assortment, Price, Service Optimization Innovation & data New competition Amazon, e-tailers 6

7 The Emerging Commerce Reality Promotions & coupons Store location Mobile commerce Self-scanning & self-checkout Payments Mobile shopping lists In-store research Social Commerce Loyalty 7

8 Each Consumer Touchpoint Must Interoperate Promotions & Coupons Store location Online commerce Mobile commerce Self-scanning & self-checkout Offline commerce Mobile shopping lists Payments In-store research Mail order Social Commerce Loyalty 8

9 Unprecedented Flexibility to Address Merchant Needs Technology integrated to ebay assets and an ecosystem of innovation Marketplace + Apps X.commerce Fabric Inc. Assets Partners X.commerce capabilities Developers 9

10 X.commerce Fabric and the Open Commerce Language Point-to-point integrations Reuse model Fabric and Open Commerce Language Merchants: easy adoption of new capabilities, low switching cost Developers: low barrier to entry, easy onboarding of customers 10

11 Platform Guiding Principles Composability Evolvability Developer usability Operability Extensibility Data resiliency Security 11

12 The Cloud, a Journey in progress Prepare for anything to fail Capacity modeling Automate relentlessly Force simplicity Uniform application architecture Storage, integration, scaling Ephemeral environments CI/CD Hybrid - on/off premise 12

13 Why OpenStack? OpenStack Openness aligns with our need for multi-vendor-ness OpenStack We have lots of moving parts... flexibility is key Momentum you re gonna be around tomorrow, right? 13

14 Vision: X.Cloud Self Healing Application Self Service Customers (Capability Developers + Merchants) Manage Data Runtime SLA Driven CI/CD XOCL XFabric Product Taxonomy Middleware O/S Virtualization We Manage Multi-Tenant Secure Commerce Capabilities Servers Storage Networking PaaS Automated 14

15 Guiding Principles: An Open Source Philosophy Cloud Foundry (PaaS) Open Stack (IaaS) Redis (In-Memory Store) Chef (Config. Mgmt) Rabbit MQ (Messaging System) Mongo DB (Persistence Store) Hadoop (Persistence Store) Collectd (Metrics Collector) Flume (Log Collection/Transport) BOSH (Cloud Mgmt. System) Open TSDB (Time Series Database) Do not reinvent the wheel & grow with the community 15

16 X.Cloud Stack An Overview CI/CD Self-Service Portal Analytics Cloud Platform Services Cloud Foundry Apigee Cloud Infrastructur e Management System Rabbit MQ Mongo DB Hadoop Flume Logs & Metrics Platform Monitoring & Alerting System Redis IaaS - Open Stack Compute Storage Network 16

17 Development Cloud Developer Commit Propagate GIT/SVN Build My Func My Integ 1 My L&P Stage Promote Push Button Deployment to Prod Nexus Pipeline Builder/Monitor Cloud Platform Services Consistent Management Tools Basic Monitoring, Logging, Cloud Mgmt System, etc Essex, 2:1 Over Subscribed, Tenants/Quotas By Engineering Groups Physically Isolated from Sandbox/Production Logical Zones Dev Tools (Nexus, SVN, GIT, Jira, etc), Continuous Integration, Cloud Engg. Test 17

18 Spirit of CI / CD 1 Developer Environment Type Aware, Pipeline/Stage Aware, SCM Aware, Artifact Aware, Application Stack Aware, Application Dependency Aware, Test Type Aware 2 Stage Characterized By Test Type Automated Test Execution Automated Environment Setup Automated Deployment Automated Propagation Automated Promotion n: Named Environments ~ n:pipeline Stages 3 Provides isolation for logical environments based on users or micro-cf installations Supports Java, node.js, Python, PHP, Ruby runtimes Supports all atomic PaaS operations (i.e. deploy, start, stop, restart, scale, etc) Manages environment variables Manages Data Service (i.e. Mongo DB, Rabbit MQ, etc) Bindings Access to logs, files, etc Manages routing to Application Access Points 18

19 Production Cloud Onboard New App Stack Upgrade App Logs/Msg. Trace Metrics Operational Health Alerts Scale App Self-Service Deployment Analytics Dashboard Alerts Console, Monitoring Dashboard Access Point Configurations Cloud Platform Services Self-Healing Cloud Management System Cloud Foundry Logs & Metrics Platform Monitoring & Alerting System Scale Data Services Provision Shared Infra. Environment Type (COS) = Environment Name Logical Environments = Sandbox & Production Availability Zones = 3 per Logical Environment Backup Data 19

20 Cloud Management System The Problem We Are Working Try to Solve Complex Deployment Orchestration Cloud Foundry Hadoop with HBASE Apigee Mongo DB, Rabbit MQ Scale Up Upgrade From: 2.0 To: 3.0 Configuration Management Replace Instances X From: To: Manage Instance Sizes/Types Manage Packages Versioning Environment Variables Security Settings Current State: Custom Orchestration, Chef based Configuration M Future State: BOSH 20

21 Logs, Metrics, Monitoring External Monitoring Alarms Config Mgmt Update Cloud Manager Pager Duty Alerts Config. Confi g Agent Monito r Agent Provision / Repair Enroll Metrics CEP Alarms Monitorin g System Alarms Metrics Open TSDB HBASE Logs/Metrics Console Flume Agent VM Enroll Logs Master Logs Hadoop Collector Flume 21

22 The X.commerce Technology Stack Built greenfield Designed for the cloud Assume failure Model capacity Automate relentlessly Force simplicity Environments are ephemeral Expect hybridization Completely virtualized operational environment Offers a high-level PaaS for commerce applications Tooled for continuous delivery CI/CD DevOps MongoDB for all data 22

23 Current Status 18+ months of work with OpenStack X.commerce Production Cloud is running on Diablo Essex upgrade is underway Go-live for Essex production cluster is less than 3 weeks away X.commerce Development Cloud and Developer Sandbox is now running on Essex and growing from X.commerce to ebay ebay has just gone live with Nicira 23

24 THE OPPORTUNITY BEFORE US

25 Focus for the future 25

26 The Open Cloud: a Call to Action Opportunities We see significant opportunities for the OpenStack Community Big Data, Signal Detection Automation Self-healing Open, Commoditized, Public-cloud Bursting Challenges We see significant challenges for the OpenStack Community Fragmentation through variability or splintering at the Kernel Speed of Innovation 26

27 THANK YOU