Data Center Interconnection

Transcription

1 Dubrovnik, Croatia, South East Europe May, 2013 Data Center Interconnection Network Service placements Yves Louis TSA Data Center Cisco and/or its affiliates. All rights reserved. Cisco Connect 1

2 Agenda Objectives Feedback from the Field Experiences with State-full Devices placements and their impact within DCI environment Understand the Data Workflow with State-full Devices across multiple sites Discuss about the evolution of the Act/Act Firewall with the ASA Clustering Agenda Review generic State-full devices roles and related workflow inside the DC Traditional solutions ASA clustering Discuss about state-full devices placement and roles across Multiple Sites Impact on the Workflow LISP Path optimization integration with State-full devices

3 Security and Network Services inside the DC State-full devices deployment inside the DC Multiple types of State-full services Firewalls Load Balancers Inspection Prevention/Detection Systems SSL Off-loader WAAS State-full implies one-way symmetrical establishments State-full Devices HA and Scalability: Active-Standby mode for statefull convergences & recovery Active Active mode for Redundancy and Scalability WAAS FW SSL Offload SLB Outside VLAN Inside VLAN IPS Front-End VLAN DC Core layer Aggregation Layer Service layer FW Access Layer Application Layer Back-End VLAN Access and Application layer

4 Security and Network Service inside the DC Nominal workflow with State-full devices deployment inside the PoD Network Services in active-standby context mode (boxes run A/A) Multiple Models of deployment o o o o SLB facing the server farm FW facing the server farm SLB In Line versus One Arm One-arm: PBR or Src-NAT VRF L3 segmentation Multiple modes of forwarding o o o Transparent Routed Mixed Session and State synchronization Usually enable Interface Tracking to force a failover for other services Simplest use case deployment for the purpose of this session 1 context of FW and SLB A/S 1 tier Application (front-end) SLB in One-arm with src-nat Dynamic FW NAT for Security VIP Source-NAT traffic flow control Web serverfarm Outside LAN FW FT & Synchro Inside LAN SLB FT Front-End VLAN DC-1 DC Core layer Aggregation layer Service layer Sub-Aggregation layer Access and Application layer

5 Security and Network Service inside the DC Ping-Pong effect with A/S State-full devices inside the PoD Network Services in active-standby context mode (boxes run A/A) Multiple Models of deployment o o o o SLB facing the server farm FW facing the server farm SLB In Line versus One Arm One-arm: PBR or Src-NAT VRF L3 segmentation Multiple modes of forwarding o o o Transparent Routed Mixed Session and State synchronization Usually enable Interface Tracking to force a failover for other services Simplest use case deployment for the purpose of this session 1 context of FW and SLB A/S 1 tier Application (front-end) SLB in One-arm with src-nat Dynamic FW NAT for Security VIP Source-NAT traffic flow control Web serverfarm Outside LAN FW FT & Synchro Inside LAN SLB FT Front-End VLAN DC-1 Ping-Pong workflow exists inside the DC, but has not impact in term of perfs, nor latency. Some devices do not support preemption making the troubleshooting/analysis a bit more challenging. DC Core layer Aggregation layer Service layer Sub-Aggregation layer Access and Application layer

6 One-armed SLB and Source NAT App 1? Outside World App1= VLAN VLAN150 server-farm nat-pool VIP Client hits VIP SLB the request to one of the real servers and source-nats the client to So that responses from the real server is sent to SLB The advantage of the one-arm configuration is that it is very easy to bypass the load-balancer when necessary. If certain clients have a need to communicate with a real server directly, it is very straightforward: the router takes care of forwarding the packet directly to its destination without involving the SLB. If on the other hand the intervention of SLB is desired, hitting the VIP will do the job. The return-traffic (server back to client) must be sent back to the SLB. There are two ways to address the problem: 1. Policy-based routing on the router 2. Source-NAT the clients IP addresses Src-NAT offers very granular matching and L4-L7-based decisions (versus Direct Srv Return) This particular scenario explores option number 2.

7 Firewall Load Balancing Symmetrical flow using Source NAT * this doesn t apply to the ASA clustering FWLB addresses scalability and redundancy by distributing traffic over parallel FW devices Each FW requires a one-way symmetrical establishment. Multiple Models of deployment o o o FWLB in sandwich between SLB Source-NAT for return flows for one-way symmetrical establishment ASA Clustering (new) No Session synchronization * o All FW are active and autonomous Simplest use case deployment for the purpose of this session 3 A/A FW 1 tier Application (front-end) * this doesn t apply to the ASA clustering Predictor Hash SLB FT Outside VLAN Inside VLAN Client VLAN (VIP) VIP Front-End VLAN Web serverfarm DC-1 DC Core layer Aggregation layer FW NAT for security Source-NAT for symmetric flow control Service layer Sub-Aggregation layer Access and Application layer

8 Firewall Load Balancing (cont) Symmetrical flow using SLB in sandwich mode FWLB in sandwich mode between 2 SLB engines Each FW requires a one-way symmetrical establishment. FW are configured in Routed mode MAC sticky from the inside SLB performs persistence for the return traffic to the original FW device. Predictor used to LB is IP Src and Dst hash (maintaining each TCP session through the same FW) IPS can be enabled on each ASA No Session synchronization o All FW are active and autonomous Src+Dst Hash MAC sticky SLB FT Client VLAN (VIP) DC Core layer Service layer Sub-Aggregation layer Front-End VLAN Web serverfarm DC-1 Access and Application layer

9 ASA Clustering (9.0) Or what s new that may help deploying security state-full devices in a DCI Interfaces in a ASA cluster can be configured in either L2 or L3 mode Interface L2 mode: All ASA share a single IP and MAC Interface L3 mode: Each ASA uses its own IP and MAC (per interface) Fully Distributed Data-Path State sharing between units (Identity, authenticate, HA etc..) Stateless Load Balancing by: External switch using ECLB for L2 mode or Router (ECMP, PBR) for L3 mode Connection Load balancing within cluster over Cluster Control Protocol State-full firewall inspection No single point of failure Centralized management and monitoring One unit is designated as the master, all other are slaves LACP (L2) or ECMP (L3) Data Traffic Port-Channel Master slave slave slave clacp for L2 mode only Cluster Control Link (CCL) LACP or ECMP

10 ASA Clustering (9.0) Connection setup when traffic is Symmetric Cluster Control Link (CCL) Director 1) State Update Owner SYN SYN/ACK SYN SYN/ACK Client Server Outside Network Inside Network State replication from Owner to Director, also serves as failover msg to provide redundancy should owner fail Director is selected per connection using consistent hashing algorithm.

11 ASA Clustering (9.0) TCP SYN cookies with Asymmetrical Traffic workflows Cluster Control Link (CCL) Director 1) State Update 1) Encodes the owner information into SYN cookies 2) forwards SYN packet encoded with the cookie toward the server Owner SYN SYN/ACK SYN/ACK SYN SYN/ACK Client Outside Network Inside Network Server 3) SYN/ACK arrives at non-owner unit 4) decodes the owner information from the SYN cookie 5) forward packet to the owner unit over CCL It is possible that the SYN/ACK from the server arrives at a non-owner unit before the connection is built at the director. As the owner unit processes the TCP SYN, it encodes within the Sequence # which unit in the cluster is the owner Other units can decode that information and forward the SYN/ACK directly to the owner without having to query the director

12 5) Here is the Owner ID 4) Owner Query? ASA Clustering (9.0) UDP sessions with Asymmetric Traffic workflows Cluster Control Link (CCL) Director 1) Owner Query 2) Not Found 3) State Update Owner Client Server Outside Network Inside Network When a unit receives a UDP packet for a flow that it does not own, it queries the director to find the owner Thereafter, it maintains a forwarding flow. It can punt packets directly to the owner, bypassing the query to the director Short-lived flows (eg. DNS, ICMP) do not have forwarding flows

13 1) Owner Query? 1) Owner Query? ASA Clustering (9.0) ASA Failover Session Recovery Cluster Control Link (CCL) Director Client Owner ASA X 3) You are onwer 4) The Owner is ASA X Server Outside Network ASA Y Inside Network Director unit maintains backup stub flow Redirects units towards the flow owner In case owner unit fails, director unit elects the owner Receives connection updates, so that they are up to date in case of owner failure

14 ASA Clustering (9.0) ASA Failover Session Recovery Cluster Control Link (CCL) Director Owner Client Outside Network ASA Y Packet M+1 Packet M Inside Network Server Director unit maintains backup stub flow Redirects units towards the flow owner In case owner unit fails, director unit elects the owner Receives connection updates, so that they are up to date in case of owner failure

15 How does State-full devices policies apply to DCI 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 15

16 Network Service placement for Metro Distances A/S state-full devices stretched across 2 locations nominal workflow Network Services are usually active on primary DC Distributed pair of Act/Sby FW & SLB on each location Additional VLAN Extended for state synchronization between peers Source NAT for SLB VIP Nota: With traditional pair cluster this scenario is limited to 2 sites Historically this has been well accepted for most of Metro Virtual DC (Twin-DC) Almost 80% of Twin-DC follows this model Outside VLAN FW FT and session synch Inside VLAN VIP Src-NAT VIP VLAN SLB session synch Front-end VLAN Back-end VLAN Primary DC-1 Secondary DC-2

17 Network Service placement for Metro Distances Ingress/Egress flows: Ping-Pong impact with A/S state-full devices stretched across 2 locations - FW failover to remote site - Source NAT for SLB VIP - Consider +/- 1 ms for each round trip for 100 km - For Secured multi-tier software architecture, it is usual to see + 10 round-trips from the client request up to the result. - Interface tracking optionally enabled to maintain active security and network services on the same site Historically limited to Network services and HA clusters offering state-full failover & fast convergences It is accepted to work in degraded mode with predictable mobility of Network Services Outside VLAN Inside VLAN VIP Src-NAT VIP VLAN Front-end VLAN Back-end VLAN Primary DC km +/- 1 ms per round trip Secondary DC-2

18 Network Service placement for Metro Distances Ingress/Egress flows: Additional Ping-Pong impact with IP mobility between 2 locations - FW failover to remote site - Front-end server farm moves to remote site - Source NAT for SLB VIP Network team is not necessarily aware of the Application/VM mobility Uncontrolled degraded mode with unpredictable mobility of Network Services Outside VLAN Inside VLAN VIP Src-NAT VIP VLAN Front-end VLAN Back-end VLAN Primary DC km +/- 1 ms per round trip Secondary DC-2

19 Network Service placement for Metro Distances State-full Devices and Trombone effect for IP Mobility between 2 locations - Migrate the whole multi-tier framework and enable HSRP filtering to reduce the trombone effect - FHRP filtering is ON on the Front-end & Back-end side gateways - Source NAT for SLB VIP maintains the return path thru the Active SLB Limited relation between server team (VM mobility) and Network Team (HSRP Filtering) and Service Team (FW, SLB, IPS..) Ping-Pong effect with active services placement may impact the performances Outside VLAN Src-NAT Inside VLAN VIP VLAN HSRP Filter Front-end VLAN Back-end VLAN Primary DC km +/- 1 ms per round trip Secondary DC-2

20 Network Service placement for Metro Distances Intelligent placement of Network Services based on IP Mobility localization - Move the FW Context associated to the application of interests - Interface Tracking to maintain the state-full devices in same location when possible - Return traffic keeps symmetrical via the state-full devices - Intra-DC Path Optimization almost achieved, however Ingress Path Optimization may be required - Sillo ed organisations - Server/app - Network/hsrp filter - service & security - Storage Outside VLAN Improving relations between sillo ed organizations increases workflow efficiency Reduce trombon ing with active services placement VIP Src-NAT Inside VLAN VIP VLAN HSRP Filter VIP Src-NAT Front-end VLAN Back-end VLAN Primary DC km +/- 1 ms per round trip Secondary DC-2

21 Network Service placement for long distances Active/Standby Network Services per Site with Extended LAN (State-full Live migration) Subnet Replication is possible using NAT or LISP Ingress Path Optimization can be initiated to reduce trombone effect due to active services placement Src NAT on each FW is mandatory Extend the VLAN of interests FW and SLB maintain state-full session per DC. No real limit in term of number of DC Granular migration is possible only using LISP or RHI (if the Enterprise owns the L3 core) Localization IP routed mode Src NAT routed mode Src NAT routed mode Src NAT Inside VLAN Inside VLAN Front-End VLAN Front-End VLAN Back-End VLAN DC-1 Move the whole framework (Front-End and Back-End) Back-End VLAN HSRP Filter DC-2 DC-3

22 Network Service placement for long distances Active/Standby Network Services per Site across Subnets (Cold migration) FW and SLB maintain state-full session per DC. No Limit in term of number of DC Granular migration is possible with LISP or RHI (if the Enterprise owns the L3 core) Implies Cold migration (stateless) LAN Extension is not required for Cold migration Subnet Replication is possible using NAT or LISP Ingress Path Optimization is needed to improve RTO Localization IP That is likely not going to happen with Cold migration Subnet C DC-1 Move the whole framework (Front-End and Back-End) DC-2 DC-3

23 Can ASA Clustering improve this? 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 23

24 Clarification As of today, the ASA clustering stretched across multiple location has not been validated yet (We are currently working on multiple scenarios to build the test plan ). However our first series of tests in our lab and in conjunction with OTV and LISP are showing great results. Stay tuned for a Cisco Validated Design 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 24

25 Single ASA Cluster stretched across multiple sites ASA Clustering Data Plane Load Distribution in Layer 2 mode - Only 1 port-channel from the ASA clustering - clacp dictated that the same port channel must exist across the same cluster - Therefore the same vpc Domain ID must be replicated on each vpc peer IP ASA Cluster Control Link IP ASA Cluster Control Link IP IP clacp ASA Po 10 IP clacp ASA Po 10 IP clacp ASA Po 10 DC-1 DC-2 DC-3

26 Single ASA Clustering stretched across multiple DC ASA Clustering with VLAN Extension (State-full Live Migration) - State-full Live Migration supported All ASA are Active Certainly good to deploy for Metro Distances using fibers Ingress traffic can be optimized using LISP (or RHI) Theoretically FW Cluster spread over up to 8 DC (more likely 4 DC with 2 Act/Act ASA on each DC) Localization IP TCP SYNCookie? Director ASA Cluster Control Link ASA Cluster Control Link Owner HRSP Filter HRSP Filter Front-End VLAN Back-End VLAN Back-end VLAN HRSP Filter DC-1 HRSP Filter Front-End VLAN HRSP Filter DC-2 HRSP Filter DC-3

27 Multiple ASA Clustering distributed on each DC ASA Clustering with Layer 3 routing between sites (Cold migration) - Certainly the best choice for Cold Migration Provide flexibility in the Operational choice Subnet replication is possible using NAT or LISP ASA can run L2 or L3 mode (mode must be the same inside the DC) Ingress Path redirection to improve RTO Localization IP Owner Director Owner Director CCL DC-1 Director CCL Owner CCL Subnet C DC-2 DC-3

28 Network Services Placement with LISP IP Mobility 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 28

29 LISP values for these scenarios Ingress Path Optimization Reduce the latency between users and application Avoid asymmetric routing In conjunction with FHRP localization IP Mobility Generic Deployment LISP using Extended Subnet Mode (LAN-Extension) LISP using Across Subnet Mode (L3 inter-site connection)

30 LISP Deployment for Ingress Path Optimization LISP w/ LAN Ext. and A/S Network Services per site (high level workflow) 1- VIP is active on DC-1 ITR redirects to DC-1 2- Move action: the whole Server-farm migrates 3- VIP becomes active in DC-2 4- VIP sends packet out (i.e. RHI) through FW 5- ETR notices and updates the Mapping DB accordingly 6- MS updates original ETR 7- ITR redirects to DC-2 1 M-DB User ITR Independent FW & SLB cluster in each location LAN extensions using OTV New state created after moves 7 No state synchronization ETR 6 5 ETR 4 HSRP Filter VIP LAN Extension 3 HSRP Filter LAN Extension DC-1 2 DC-2

31 Traditional ASA deployment across Multiple DC LISP Extend. Subnet Mode with State-Full Device in Act/standby mode (Hot Migration) 1 - End-user sends Request to App 2 - ITR intercepts the Req and check the localization 3 - MS replies location for being ETR DC ITR encaps the packet and sends it to RLOC ETR-DC-1 M-DB ITR Update your Table 4 LISP Multi-hop informs ETR on DC-2 about theismove of App Source NAT required for 5 Meanwhile ETR DC-2 informs MSone about new location of App way symmetric 6 MS updates ETR DC-1 establishment 7 ETR DC-1 updates its table (App:Null0) Stateful migration is not 8 ITR sends traffic to ETR DC-1 achieved 9 ETR DC-1 replies with a Solicit Map Req from a TCP flow point view. 8 ITR sends a Map Req and redirects the of Req to ETR DC-2 However the HTTP session is kept alive ETR App is located in ETR-DC-2 ETR routed mode Src NAT App has moved ETR routed mode Src NAT Inside VLAN Inside VLAN HRSP Filter HRSP Filter HRSP Filter Front-End VLAN Front-End VLAN Back-End VLAN Back-end VLAN HRSP Filter DC-1 routed mode Src NAT HRSP Filter DC-2 HRSP Filter DC-3

32 App has moved Traditional ASA deployment across Multiple DC LISP Across Subnet Mode with State-Full devices in Act / Standby mode (Cold migration) M-DB Update your Table 1 - End-user sends Request to App 2 - ITR intercepts the Req and check the localization 3 - MS replies location for being ETR DC ITR encaps the packet and sends it to RLOC ETR-DC-1 ITR 4 LISP Multi-hop informs ETR on DC-2 about the move of App 5 ETR DC-2 informs MS about new One location way of symmetric App 6 MS updates ETR DC-1 establishment can not be 7 ETR DC-1 updates its table (App:Null0) achieved without VLAN 8 ITR sends traffic to ETR DC-1 extension between DC 9 ETR DC-1 replies Solicit Map Req Cold Migration implies the 8 ITR sends a Map Req and redirects Server the Req to restart to ETR DC-2 ETR ETR ETR Subnet B Subnet C DC-1 DC-2 DC-3

33 Single ASA Clustering stretched across Multiple DC LISP Extended Subnet Mode with ASA Clustering (Stateful Live migration) 1 - End-user sends Request to App 2 - ITR intercepts the Req and check the localization 3 - MS replies location for being ETR DC ITR encaps the packet and sends it to RLOC ETR-DC-1 M-DB ITR Update your Table 4 LISP Multi-hop informs ETR on DC-2 about the move of App Way Symmetric 5 Meanwhile ETR DC-2 informs MSOne about new location of App Establishment is achieved via 6 MS updates ETR DC-1 7 ETR DC-1 updates its table (App:Null0) the CCL 8 ITR sends traffic to ETR DC-1 Current active sessions are 9 ETR DC-1 replies with a Solicit Map Req maintained stateful 8 ITR sends a Map Req and redirects the Req to Ingress flowsetr for DC-2 new Sessions are optimized ETR App is located in ETR-DC-2 ETR ASA Cluster Control Link App has moved ETR Director ASA Cluster Control Link Owner HRSP Filter HRSP Filter Front-End VLAN Front-End VLAN HRSP Filter Back-End VLAN Back-end VLAN HRSP Filter DC-1 HRSP Filter DC-2 HRSP Filter DC-3

34 Single ASA Clustering stretched across Multiple DC LISP Across Subnet Mode with ASA Clustering (Cold migration) 1 - End-user sends Request to App 2 - ITR intercepts the Req and check the localization 3 - MS replies location for being ETR DC ITR encaps the packet and sends it to RLOC ETR-DC-1 M-DB ITR Update your Table 4 LISP Multi-hop informs ETR on DC-2 about the move of App Cold Migration 5 ETR DC-2 informs MS about new location of App implies the Server to restart 6 MS updates ETR DC-1 7 ETR DC-1 updates its table (App:Null0) There is no added value to 8 ITR sends traffic to ETR DC-1 stretch the ASA clustering 9 ETR DC-1 replies Solicit Map Reqacross the sites for Cold 8 ITR sends a Map Req and redirects the flow to ETR DC-2 Migration ETR ETR ASA Cluster Control Link App has moved ETR Director ASA Cluster Control Link Owner Subnet C Subnet B DC-1 DC-2 DC-3

35 ASA Clustering per DC across Multiple sites LISP Across Subnet Mode with ASA Clustering (Cold migration) 1 - End-user sends Request to App 2 - ITR intercepts the Req and check the localization 3 - MS replies location for being ETR DC ITR encaps the packet and sends it to RLOC ETR-DC-1 M-DB ITR Update your Table 4 LISP Multi-hop informs ETR on DC-2 about the move of App Solution designed for Cold 5 ETR DC-2 informs MS about new location of App migration only 6 MS updates ETR DC-1 7 ETR DC-1 updates its table (App:Null0) Preferred choice for Cold 8 ITR sends traffic to ETR DC-1 migration 9 ETR DC-1 replies Solicit Map Req Cold Migration implies the 8 ITR sends a Map Req and redirects the Req to ETR DC-2 Server to restart ETR ETR Director App has moved Owner ETR Owner Director CCL Owner Director CCL CCL Owner Subnet C Subnet B DC-1 DC-2 DC-3

36 State-full devices placement with DCI Key Takeaways Ping-Pong effect might have a bad impact in term of perf with long distances: Greedy bandwidth Latency For Metro Virtual DC, It is commonly accepted to distribute traditional A/S state-full devices between 2 Twin DC (for short Metro Distances (+/- 10km max) Keep transparency and easy to operate limited to 2 Active DC Only 1 FW is Active at a time Preferred method is to deploy Stretch ASA clustering for Metro VDC Easy to operate with all ASA active Not limited to 2 Active DC LISP is the good choice for Ingress Path Optimization GSLB (DNS and KAP-AP) can help to redirect the traffic accordingly, but may face some caveats with proxy DNS and client caching RHI can help but offers App based granularity only for Intranet core (Enterprise owns the L3 core) The recommended choice is ASA clustering in conjunction with the traditional DNS and LISP Mobility. Stretched across multiple DC with LAN extension for Hot Migration Confined inside each DC without LAN extension for Cold Migration ASA Clustering stretch across multiple sites is not yet supported, as not fully tested. Stay tune

37 Thank you Cisco and/or its affiliates. All rights reserved. Cisco Connect 37