Disclaimer This presentation may contain product features that are currently under development. This overview of new technology represents no commitme

Size: px

Start display at page:

Download "Disclaimer This presentation may contain product features that are currently under development. This overview of new technology represents no commitme"

Myra Taylor
6 years ago
Views:

1 VIRT1381BE Turkey's Biggest Bank s Data Center Migration to a New Data Center (ZVM) Volkan Işıkhan, Serdar Sunar, Mehmet Can Önal #vmworld #VIRT1381BE

2 Disclaimer This presentation may contain product features that are currently under development. This overview of new technology represents no commitment from VMware to deliver these features in any generally available product. Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind. Technical feasibility and market demand will affect final delivery. Pricing and packaging for any new technologies or features discussed or presented have not been determined. 2

3 Content General Information About Ziraat Financial Group (ZFG) and Datacenter Migration Program Ziraat Financial Group Ziraat Datacenter Program Ziraat Datacenters Migrating Applications Distributed Systems Introduction Migration Story Centralized Systems Introduction Migration Story 3

4 Ziraat Financial Group 4

5 Ziraat Bank «More Than a Bank» Leading Banking Sector in Turkey Celebrating 154 years with pride Ziraat Bank now has an operational presence at 97 locations in 18 countries with over 2,000 branches Over 25,000 employees We are always at our Customer s side with our alternative channels: ATM Internet & Mobile Banking Phone Banking SMS Banking 5

6 Why a New Datacenter is Needed? Improve infrastructure performance Ensure continuous delivery and improvement Become agile Manage risks Set up a private cloud Embrace new technologies in a different way 6

7 Ziraat Datacenter Transformation Program Building the datacenter facility Migrating applications 7

8 Building The Datacenter Facility

9 9

10 Migrating Applications

Migrating Applications Wave - 3,750 VM & Physical Servers - 500 Applications - 380 Databases - 154 VLANs - 10 Migration Waves - 35 Application Bundles

22/10-26/3-20/8 27/36/4-9/4-9/4 Wave 6 Wave Wave 7 6 17/5-21/5 Wave 9 1 Dec, 2016 1 Jan, 2017 1 Feb, 2017 1 Mar, 2017 1 Apr, 2017 1 May, 2017 1 Jun,

11 Migrating Applications Wave - 3,750 VM & Physical Servers Applications Databases VLANs - 10 Migration Waves - 35 Application Bundles Project Plan 24/10-20/11 21/11-27/11 5/12-17/12 11/12-18/12 Wave 1 Wave 2 Wave Wave Nov, /1-3/2 Wave 4 31/1-27/2 Wave 5 6/3 22/10-26/3-20/8 27/36/4-9/4-9/4 Wave 6 Wave Wave /5-21/5 Wave 9 1 Dec, Jan, Feb, Mar, Apr, May, Jun, Jul, Aug, Oct, Aug, /4-7/5 Wave 8 25/5-20/8 Wave 10 TODAY Wave Pass Failback In Progr. Postpone Re-Planned Pending

12 Distributed Systems

13 Distributed Systems Services VMware Virtualization Services MS Active Directory Messaging Systems Bulk Messaging Systems Instant Messaging Systems Intranet Portal Sites Internet Web Sites E-Fax Systems Service Manager System Test and Development Servers, File Servers Consulting Services to Foreign Branches & Subsidiaries 13

14 VMware Virtualization Environment 120 ESXi hosts in primary and secondary datacenters 7 vsphere clusters Over 1,250 virtual machines, Over 250 applications 100% virtualized, x86 Messaging systems +32,000 mailbox, +12,000 online activesync sessions 0,3 Pbyte (Petabyte) db, +300M archieved items Daily 1.2M receive, 200K send, Monthly +20M bulk send Instant Messaging systems +20k online instant messaging sessions E-fax Messaging Systems Monthly +60k receive, +60k send 14

15 Network Topology Söğütözü - Ankara 10G (DWDM) ZVM Esenyurt - İstanbul 10G (DWDM) 10G (DWDM) 10G (DWDM Redundant ISP) Maslak - İstanbul DRC KKB - Ankara 15

16 Analysis and Planning Phase Assessment Current State Analysis Conceptual Design Application Dependency Mapping vadp (VMware Application Dependency Planner) Enterprise Architecture Knowledge Plan for Migration Create a plan for migration bundles and waves Pre-migration tests 16

17 Alternative Methods for Migration Long Distance vmotion across datacenters vsphere Replication Array Based Replication SRM (Site Recovery Manager) 17

18 Pre-Migration Steps New Server hardware installation and configuration for new datacenters Increasing speed and capacity of migration process 10 GigE network adapter installation for servers in old datacenters vsphere upgrade to 6.0 U2 Design and Install vcenter Server and Platform Services Controller vsphere Standard Switches to vsphere Distributed Switches migration in new datacenters 18

19 Virtualization Platform Topology 19

20 Experience The Execution Intranet portal migration Test/development/UAT/Reference servers migration Various application servers, terminal servers, system management servers migration Internet web sites migration E-fax, service manager servers migration Active directory service migration Instant messaging service migration messaging service migration 20

21 Lessons Learned LDvMotion Issue with Enhanced vmotion Compatibility - Upgrade the Compatibility Level for Virtual Machines - Disable EVC mode in old datacenter vsphere clusters and set EVC mode baseline as Haswell in new datacenter vsphere clusters Current datacenter vss, new datacenter dvs infrastructure 980 Mbit/s bandwidth limitation for migrating VM (PR) Backup network isolation Private Cloud LDvMotion SRM for Disaster Recovery 21

22 or Centralized Systems distribution

23 Centralized Systems Infrastructure Management Team Responsible for Servers Virtualization Platforms SAN & Storage Management Backup Systems OS & Middlewares (Application Platforms) Principals Maksimum Performance High Avaliability Right Sizing and Utilization (Resource Management) Adapting new Technologies 23

24 Services Core Banking Service ~2,000 Branches, 25,000 users Internet and Mobile Banking 5,000,000+ clients ATM 10,000 ATMs Credit Cards, POS Services Telephony Banking Services Ziraat Insurance & Retirement Company Ziraat Investment Company Others 24

25 Infrastructure Summary Servers 150 physical servers (4 sockets Intel E7-4850v4, 512GB RAM/2 sockets Intel E5-2698v4, 512GB RAM) Dedicated to VMware Platforms 15 vsphere clusters on production 2500 VMs 60 physcial servers (2 sockets Intel E5-2690v4 512GB RAM) MS SQL clusters Linux servers for specific apps Media agents for backup systems 25

26 Infrasturcture Summary (cont d) SAN & Storage Systems 6 PB Storage Capacity 4xEMC-VMAX 400K 2xEMC-VMAX 100K 3 Storage Virtualization Clusters 2xEMC-VPLEX-VS6 (4 Engines) 1xEMC-VPLEX-VS6 (2 Engines) Replication Clusters 64 RecoverPoint Appliances distributed across 6 clusters SAN Environment 16Gbps/port Total 3,000 ports hold by 6 Cisco SAN Directors Backup Systems EMC 2xDataDomain Virtual Tape Libraries (1.5PB capacity) Quantum Physical Tape library (70 Drives capacity) 26

27 Virtualization Platform Topology VMworld 2017 Content: Not for Esenyurt Maslak Söğütözü Maslak vcenter Server (Source I) 30 ESXi Hosts 100 pcpu 13TB Memory 670TB Datastore 800 VMs Esenyurt vcenter Server (Target) 95 ESXi Hosts 344 pcpu 48TB Memory 1,450TB Datastore 2,250 VMs Söğütözü vcenter Server (Source II) 56 ESXi Hosts 202 pcpu 22TB Memory 530TB Datastore publication 1450 VMs 27

28 Getting Ready/Preparation VMware Application Dependency Planner (vadp) Discovery Dependency Matrices Relational Diagrams SOP/TOP documentation and impact analysis APM & NPM (AppInternals & AppResponse & AppDynamics) Application behaviour and pattern detection Response Analysis Metric collection Impact Analysis & Migration Strategy Migration Method Tests Determination of needs Method limitations Re-designing strategy 28

29 Ready, Steady and Go Long Distance vmotion Why? Live migration No downtime Single instance Apps and DBs App with sessions and states Why not? Takes time Cost limits SvMotion (4 costs ) - vmotion (1 cost) 2 LDvMotion/host (8 costs) Network Bandwidth Difficult to roll back Golden copy operations (storage clones, not snapshots!!!) VM Cloning Consumes Space Takes extra time 29

30 Services Migrated with LDvMotion ATMs 10,000 ATMs online Mobile Authentication App Paloma App Group 30

31 Ready, Steady and Go (cont d) vsphere Replication Why? Large (crowded) server farms behind Load Balancer Enabling/Disabling operations with Load Balancer No downtime (again!!!) Faster than array based replication Easier operation Not having disk signatures (fancy) No Rescanning HBA processes and snap named volumes Easy and Smooth roll back Scenarios Do more migration at one shot Why not? Needs downtime Huge capacity DB servers (initial sync) Changes on source (RPO violation) Takes a few more steps (comparing with LDvMotion) 31

32 Services Migrated with vsphere Replication Finart Core Banking App 500 VMs Internet Banking App 200VMs VMworld 2017 Card Payment Systems 400VMs Mobile Banking App 200VMs Subsidaries 400VMs Content: Not for publication Other Apps 600VMs 32

33 Ready, Steady and Go (cont d) What about physical environment? Inventory to migrate MSSQL Database Servers Specific Apps (Anti-Money Laundering, Bosnia Card Payment Systems) Physical Dependencies (HSMs, dongles) Difficulties Alignment of servers Testing new infrastructure (hardwares, firmwares, drivers ) Upgrade needs Storage Replication (changing vendors) 33

34 What We Gain? Modern (Tier III) Datacenters Powerful Infrastructure Improved High Avaliability Fresh Technologies Storage Virtualization Network Virtualization Modernized Disaster Strategies 34

Disclaimer This presentation may contain product features that are currently under development. This overview of new technology represents no commitme

STO1297BE Stretched Clusters or VMware Site Recovery Manager? We Say Both! Jeff Hunter, VMware, @jhuntervmware GS Khalsa, VMware, @gurusimran #VMworld Disclaimer This presentation may contain product features