Firewalling Avid ISIS in a Cisco environment

Transcription

1 Firewalling Avid ISIS in a Cisco environment Interoperability testing between Cisco ASA and ISIS results Francesca Martucci Consulting System Engineer for Security - Cisco David Shephard - Senior Network Solutions Architect Avid Page 1 of 27

2 Abstract Deploying ISIS and firewalls has always been a challenging proposition, due to the particularity of the ISIS traffic more specifically for its fragmentation and strong latency requirements. We have recently performed a set of tests using the Cisco ASA , and we were able to demonstrate that the firewall did not have problem in handling such a fragmented traffic, at the same time the latency introduced was not noticeable from the usability point of view. We can then validate the possibility of adding a Cisco ASA firewall between a client and ISIS Storage server. This document is going over the details of those tests, which were performed at the Bedfont Lakes CPOC, showing the results that were obtained. Page 2 of 27

3 Contents Abstract... 2 Overview... 5 Understanding ISIS traffic... 5 Next understand Latency... 5 Firewall challenges with Legacy solutions... 6 What ports are used?... 7 Lab setup... 7 Firewall... 7 ISIS System... 8 Clients... 8 Test results... 9 Test 1: no firewall... 9 Test 2: Adding the ASA with Permit any any rule TEST 2a TEST 2b Test 3: Firewall with specific rules allowing only ISIS traffic TEST 3a TEST 3b TEST 3c Conclusions APPENDIX A: firewall configuration with no rules APPENDIX B: firewall configuration with ACLs applied APPENDIX C: Firewall Services Module testing Page 3 of 27

4 Table of Figures Figure 1: Lab Topology... 7 Figure 2: No Firewall unlimited test at low resolution... 9 Figure 3: 48MB/s test Figure 4: one single client at 48MB/s Figure 5: 4 clients at 48MB/s Figure 6: 4 clients at 48MB/s Figure 7: Ping delay for the test Figure 8: 4 clients running at 48MB/s at medium res Figure 9: 4 clients running 48MB/s at medium res Figure 10: 64MB/s with 5 clients Figure 11: 5 clients at 64MB/s Figure 12: Screen capture of the ASA CPU load balancing during the test Figure 13: statistics with 5 clients at 64MB/s Figure 14: 5 clients at 64MB/s Figure 15: FWSM lab topology Page 4 of 27

5 Overview Before going into the details of the test that were run, let s have a quick look at the particularity of the ISIS traffic, in order to understand the challenges that could be found sending such traffic through a firewall. Understanding ISIS traffic By default ISIS 1.x sends 256KB packages as 5 UDP datagrams to the client. Because the MTU size on an IP network (without Jumbo Frames) is 1500 Bytes, each datagram this gets broken down by the IP stack on the ISB into approximately 36 fragments which are sent to the receiving client, which must re-assemble the datagram and then send up the IP stack to the application. Hence the reason why we need a Network Interface Card with lots of descriptors (1500 Byte buffers). In ISIS 2.x with ISS2000 a 512KB chunk size can be used which results in 9 datagrams of approximately 40 fragments each. NOTE: Why ISIS doesn t we use jumbo frames? Well this is generally used for short haul TCP based server server communications, and this is not server to server type traffic but real time video to editing clients Also using jumbo frames across a wider network diameter brings significant administrative overhead. The ISIS client resolution and the video resolution decide how the [editing] application will ask for the data. An ISIS 1.x client set for Low Resolution and DV25 will request a transfer size of 1024KB, i.e. 4 x 256KB, each 256KB chunk will come from a single ISIS Server Blade (ISB) sequentially. This would result in 20 fragmented datagram (a total of 712 x 1500 byte packets) taking 0.02 seconds then pause for 0.23 seconds until the next 1024KB I/O cycle. When set for Medium Resolution and DV50 the receiving device it may request a transfer size larger than 1024KB, and this will be requested from two ISBs concurrently. The I/O size is not fixed and will vary with editor release, ISIS release and video resolution. For HD based resolutions the editor will typically use a 4 MB transfer size, but it is not limited to that and it will exceed 4 MB when using some HD based resolutions. Later versions of editors typically request a set number of frames so I/O size will vary with video resolutions. Audio requests may use a different I/O size and frequency to video requests. Next understand Latency ISIS client applications are latency sensitive. An editing application needs to be responsive; ISIS was designed for high speed LAN environment. When latency gets to 5ms it becomes noticeable, at 10ms it becomes intrusive, and at 20ms it is unpleasant to use. Some testing with NewsCutter has been done previously as part of a different products but this was based on Gigabit Ethernet MAN connection. Latency applied 0ms 5ms 10ms Result System performs on test network as if locally attached Noticeable degradation in scrubbing performance, slight delay in play function (minimal) Particularly noticeable delay in scrubbing, 1s delay from pressing play Page 5 of 27

6 20ms 50ms 100ms to material playing, may not be suitable for editors More noticeable delay in scrubbing, 2.5s delay from pressing play to material playing this would most likely be unsuitable for editors Unusable delay from pressing play, buffer ran out after 4-5 seconds and then started dropping frames system will not mount ISIS workspaces, reports network errors *Given that the speed of light constant in a vacuum, 'c' is exactly 299,792,458 metres per second, the figure of 1 millisecond per 300km might be an accurate estimate for the purpose of latency calculation over distance However, propagation speed in media is significantly lower than c, for glass roughly 1/2-2/3 of light speed in vacuum, depending on the refraction index of the media., so a figure of 1 millisecond per 200km is more appropriate. Based on the tests performed with a NewsCutter editing client,5ms is an acceptable latency; this translates to a distance of a connection of approx km* where it would be acceptable to the operator. Firewall challenges with Legacy solutions Typically when a legacy firewall encounters a fragmented packet it wants to re-assemble all the fragments into a complete datagram and inspect the content from the inbound interface, once it analyzes the packet and verifies its validity, it will then send the content via the outbound interface. The first challenge for the firewall is to assemble the datagram which will be 256KB in size, it then has to process it, and, if satisfied, it has to re-fragment it and send it on its way. The second challenge for the firewall is to re-fragment datagram in exactly the same way, which it should do under normal circumstances. Add to this the quantity of 256KB bursts per second per client, which is dependent on video resolution. DV 25 = approx 4MB/S do that is 16 x 256KB bursts per second DV 50 = approx 8MB/S do that is 32 x 256KB Burst per second MPEG II Browse uncompressed audio is approx 0.5 MB/S do that is 2 x 256KB bursts per second. Note while the burst size used in ISIS 2.0 HW defaults to 512KB, the transfer rate is resolution dependant and the I/O size similar. An I/O size for 1024KB with DV25 resolution using 256KB chunks will result in four bursts of 178 Packets, or 712 Packets, totalling 2824 packets per second as 80 fragments An I/O size for 1024KB with DV25 resolution using 512KB chunks will result in two bursts of 355 Packets, or 710 Packets, totalling 2840 packets per second as 72 fragments Then multiply that by the number of clients, so 10 clients need 2.5MB of high speed memory available to the firewall, remember to process at high speed this need to be executed in hardware, not software which would add huge amounts of latency. The newer generation of firewalls are much more powerful in terms of processing power, and therefore are able to perform reassembly and disassembly at very high speed, thus not impacting the traffic in a sensitive manner. The latency introduced is well below the 5ms limit allowed by the ISIS traffic Page 6 of 27

7 What ports are used? The TCP and UDP ports used but ISIS, Interplay and WG4 are available from Another source of port usage information which may be more up to date then the document above ids the Avid Products and Network Site Preparation Guide Lab setup In Figure 1: Lab Topology is shown the lab environment that was used for the tests. Figure 1: Lab Topology Firewall For the purpose of the test we used the Cisco ASA 5580 Series which is offered at two performance levels: the Cisco ASA with 5 Gbps of real-world firewall performance, and the high-end Cisco ASA with 10 Gbps of real-world firewall performance. Their multicore, multiprocessor architecture delivers radical scalability for the most demanding network security and VPN concentration applications. Real-time applications can be transparently secured thanks to the extremely low latency, high session concurrency, and connection setup rates. The firewall was deployed in single context and routed mode. Software version was Page 7 of 27

8 In all the tests the firewall was setup to allow up to 1000 fragments per packet, and up to 2000 fragments in the interface queue. Those number could be set higher is needed. For reference to the full firewall configuration, please refer to the appendix ISIS System The ISIS system used for the testing was a single engine, using ISS1000 switch blades (Version 1 hardware) and running ISIS software. This single engine is rate to deliver up to 300MB/S* of video Bandwidth, hence this was the upper limit for this test. * ISIS V1 rating per engine is 240MB/S in a HD configuration (transfer windows above 2048 KB) and 300 MB/S in an SD configuration (transfer windows below 2048 KB). Most of the testing was performed with an HD configuration using a transfer window of 4096KB, however by using small 100MB test files, it was possible to reach 300MB/S reliable because the test file data was kept in the cache of the ISIS Storage blades (ISB), not requiring deeper access to the spinning disk which would reduce the available BW by approx 20% By using additional engines and/or newer hardware, significantly more video bandwidth would be available. However 300MB/S is sufficient to for a Proof of Concept and is indicative of external bandwidth in a typical system, and exceeds by a factor of 10 the results obtained with previous testing on legacy platforms. Clients The test workstations were 4 x HP-compaq-DC5750_Microtower device fitted with Intel Pro 1000/PT adapters running Window XP SP3 and the ISIS client version was V The 5 th test workstation was an (Avid loaned) HP XW 8600 Workstation with A Broadcom NetXtreme Gigabit Ethernet interface card installed with the ISIS client V2.2.2 and Media composer 5. This workstation was used as the subjective viewing device. One of the DC5750 had Media Composer 5 installed but the graphics card was not of sufficient quality to support video. Page 8 of 27

9 Test results Test 1: no firewall The first test was done without the Firewall, with clients in zone 2 (connected directly to an Avid supported Gigabit Ethernet switch), in order verify connectivity and characteristic of the traffic. We can see that we are able to write at 80MB/S and read at approx 55MB/S in Low resolution with receive descriptors at 1024 on the Intel NIC. Figure 2: No Firewall unlimited test at low resolution Page 9 of 27

10 Figure 3: 48MB/s test Based on the performance of the test platform, it was decide to baseline the client performance at 48MB/S in Low resolution to ensure that no performance degradations could be the responsibility of the workstation client. A custom PATHDIAG test was created for this purpose and deployed on all clients. Other customs tests for 16MB/S, 32 BM/S and 64MB?S were created but primarily 48MB/S (for low resolution) and 64 MB/S (for medium resolution) were used. During the testing two metrics were used (i) The PATHDIAG trace (ii) The quantity of re-assembly failures shown using the netstat s command at the windows CLI -any significant increase in this value during any of the testes would be considered a fault Test 2: Adding the ASA with Permit any any rule We have then added the in the path, moving the clients in zone 3. The firewall was setup with a permit any any rule allowing all the traffic through. TEST 2a We run first with one single client at 48MB/s, then moved to running 4 clients at the same 48MB/s speed Page 10 of 27

11 Figure 4: one single client at 48MB/s TEST 2b We then moved to 4 clients at 48MB/s, for a total load of 190MB/s Testing was also performed using the MEDIUM resolution setting in ISIS This changes how the ISIS clients interact with the ISIS storage server so that two ISB will send data to the client concurrently (whereas in Low resolution it is a single ISB concurrently). Medium resolution allows a higher bandwidth to be achieved and also places more burden on the Firewalling device. Figure 5: 4 clients at 48MB/s Page 11 of 27

12 Figure 6: 4 clients at 48MB/s Figure 7: Ping delay for the test The NETWORK CONNECTIVITY test within the PATHDIAG application uses a variation of the common PING and produces a very accurate result. The default packet size used is 8192 (to ensure fragmentation and re-assembly is successful) and the packets are time-stamped and given individual indents so a precise round trip time can be calculated to 6 decimal places. Four ISBs (2 in each ISIS VLAN) were used as targets and the average of averages used as the result/benchmark. Test 3: Firewall with specific rules allowing only ISIS traffic After we have validated the path and performances through the firewall, we have added rules permitting only ISIS traffic through the firewall and nothing else. The performances remained constant with the ones seen before the introduction of the rules. Page 12 of 27

13 TEST 3a We have run 4 clients at 48MB/s (providing an average load of 192 MB/S) for 5 minutes with Medium resolution was used exclusively for the subsequent tests, which permitted an increase to 64 MB/S per clients, with the results you can see in the following screen shots. No reassembly issues were noted. Figure 8: 4 clients running at 48MB/s at medium res Figure 9: 4 clients running 48MB/s at medium res TEST 3b All clients pulling up to 64MB/s (at 4096/100) with pathdiag and one client pulling real video was fine, will all 5 clients reaching up the requested speed, and average ping time of 2.4ms. ASA CPU utilization for each of the 8 cores was average of 10%. Page 13 of 27

14 Figure 10: 64MB/s with 5 clients Figure 11: 5 clients at 64MB/s Figure 12: Screen capture of the ASA CPU load balancing during the test Page 14 of 27

15 Figure 13: statistics with 5 clients at 64MB/s TEST 3c Test 4 clients at 64M/4096/100 medium resolution with DH video on XW8600. Figure 14: 5 clients at 64MB/s The total bandwidth reached was 326MB/S which is the limit for the ISIS system that was in use. Page 15 of 27

16 Conclusions In conclusion we can certify that the ASA can be used within an ISIS deployment in a Layer 3 environment without introducing sensitive latency, therefore without adding visible impact to the performances of the system. The 300MB/S tested is sufficient to for a Proof of Concept and is indicative of external bandwidth in a typical system, and exceeds by a factor of 10 the results obtained with previous testing on legacy platforms. We do suggest though running a more complete and exaustive performance test in case that the firewall would need to be implemented in a scenario where a much higher bandwidth would be needed. At the moment of performing the tests the was the most powerful ASA present on the market; just after running the tests more powerful boxes have been released (5585 series), and therefore we do suggest to perform such performance tests using those newer devices. Page 16 of 27

17 APPENDIX A: firewall configuration with no rules : Saved : Written by enable_15 at 08:04: UTC Fri Oct ASA Version 8.3(1) hostname LE03-ASA-11 enable password giwspkjr1zwxnkyt encrypted passwd giwspkjr1zwxnkyt encrypted names interface Management0/0 nameif management security-level 100 ip address management-only interface Management0/1 management-only interface GigabitEthernet3/0 interface GigabitEthernet3/1 interface GigabitEthernet3/2 interface GigabitEthernet3/3 interface GigabitEthernet4/0 Page 17 of 27

18 interface GigabitEthernet4/1 interface GigabitEthernet4/2 interface GigabitEthernet4/3 interface TenGigabitEthernet5/0 nameif inside security-level 50 ip address interface TenGigabitEthernet5/1 nameif outside security-level 55 ip address interface TenGigabitEthernet6/0 interface TenGigabitEthernet6/1 interface TenGigabitEthernet7/0 interface TenGigabitEthernet7/1 *NOTE: this interface used for test *NOTE: this interface used for test Page 18 of 27

19 interface TenGigabitEthernet8/0 interface TenGigabitEthernet8/1 boot system disk0:/asa831-smp-k8.bin ftp mode passive same-security-traffic permit inter-interface access-list inside_access_in extended permit ip any any access-list inside_access_in extended permit icmp any any access-list outside_access_in extended permit ip any any access-list outside_access_in extended permit icmp any any access-list global_access extended permit object-group DM_INLINE_SERVICE_ log disable inactive pager lines 24 logging enable logging asdm informational mtu management 1500 mtu inside 1500 mtu outside 1500 no failover icmp unreachable rate-limit 1 burst-size 1 no asdm history enable arp timeout access-group inside_access_in in interface inside access-group outside_access_in in interface outside access-group global_access global route management route inside route inside route outside timeout xlate 3:00:00 timeout conn 1:00:00 half-closed 0:10:00 udp 0:02:00 icmp 0:00:02 timeout sunrpc 0:10:00 h323 0:05:00 h225 1:00:00 mgcp 0:05:00 mgcp-pat 0:05:00 timeout sip 0:30:00 sip_media 0:02:00 sip-invite 0:03:00 sip-disconnect 0:02:00 timeout sip-provisional-media 0:02:00 uauth 0:05:00 absolute timeout tcp-proxy-reassembly 0:01:00 dynamic-access-policy-record DfltAccessPolicy http server enable http management Page 19 of 27

20 http management http management no snmp-server location no snmp-server contact snmp-server enable traps snmp authentication linkup linkdown coldstart fragment size 2000 inside fragment chain 1000 inside fragment size 2000 outside fragment chain 1000 outside crypto ipsec security-association lifetime seconds crypto ipsec security-association lifetime kilobytes telnet management telnet management telnet timeout 5 ssh management ssh timeout 5 console timeout 0 management-access management tls-proxy maximum-session 1000 threat-detection basic-threat threat-detection statistics access-list no threat-detection statistics tcp-intercept ntp server source management prefer webvpn username admin password 2sSINGQnY3ksKp6b encrypted privilege 15 class-map inspection_default match default-inspection-traffic policy-map type inspect dns preset_dns_map parameters message-length maximum 512 policy-map global_policy class inspection_default inspect dns preset_dns_map inspect ftp inspect h323 h225 inspect h323 ras inspect rsh inspect rtsp inspect esmtp inspect sqlnet inspect skinny inspect sunrpc inspect xdmcp inspect sip inspect netbios inspect tftp Page 20 of 27

21 inspect ip-options service-policy global_policy global prompt hostname context call-home profile CiscoTAC-1 no active destination address http destination address destination transport-method http subscribe-to-alert-group diagnostic subscribe-to-alert-group environment subscribe-to-alert-group inventory periodic monthly subscribe-to-alert-group configuration periodic monthly subscribe-to-alert-group telemetry periodic daily Cryptochecksum:26ca34f2c601fb68b7ab6d898d73531e : end Page 21 of 27

22 APPENDIX B: firewall configuration with ACLs applied : Saved : Written by enable_15 at 08:04: UTC Fri Oct ASA Version 8.3(1) hostname LE03-ASA-11 enable password giwspkjr1zwxnkyt encrypted passwd giwspkjr1zwxnkyt encrypted names interface Management0/0 nameif management security-level 100 ip address management-only interface Management0/1 management-only interface GigabitEthernet3/0 interface GigabitEthernet3/1 interface GigabitEthernet3/2 interface GigabitEthernet3/3 interface GigabitEthernet4/0 Page 22 of 27

23 interface GigabitEthernet4/1 interface GigabitEthernet4/2 interface GigabitEthernet4/3 interface TenGigabitEthernet5/0 nameif inside security-level 50 ip address interface TenGigabitEthernet5/1 nameif outside security-level 55 ip address interface TenGigabitEthernet6/0 interface TenGigabitEthernet6/1 interface TenGigabitEthernet7/0 interface TenGigabitEthernet7/1 *NOTE: this interface used for test *NOTE: this interface used for test Page 23 of 27

24 interface TenGigabitEthernet8/0 interface TenGigabitEthernet8/1 boot system disk0:/asa831-smp-k8.bin ftp mode passive same-security-traffic permit inter-interface object-group service DM_INLINE_SERVICE_1 *NOTE: this the rule set used for the test service-object tcp destination eq 5015 service-object udp destination range service-object tcp destination eq https service-object udp destination range service-object udp destination range service-object udp destination range object-group service DM_INLINE_SERVICE_2 service-object tcp destination eq 5015 service-object udp destination range service-object tcp destination eq https service-object udp destination range service-object udp destination range service-object udp destination range object-group service DM_INLINE_SERVICE_3 service-object tcp destination eq 5015 service-object udp destination range service-object tcp destination eq https service-object udp destination range service-object udp destination range service-object udp destination range object-group network DM_INLINE_NETWORK_1 network-object network-object object-group network DM_INLINE_NETWORK_2 network-object network-object access-list inside_access_in extended permit object-group DM_INLINE_SERVICE_1 object-group DM_INLINE_NETWORK_ log disable access-list inside_access_in extended permit ip any any inactive access-list inside_access_in extended permit icmp any any Page 24 of 27

25 access-list outside_access_in extended permit object-group DM_INLINE_SERVICE_ object-group DM_INLINE_NETWORK_1 log disable access-list outside_access_in extended permit ip any any inactive access-list outside_access_in extended permit icmp any any access-list global_access extended permit object-group DM_INLINE_SERVICE_ log disable inactive access-list 1 extended permit tcp eq 5015 log disable pager lines 24 logging enable logging asdm informational mtu management 1500 mtu inside 1500 mtu outside 1500 no failover icmp unreachable rate-limit 1 burst-size 1 no asdm history enable arp timeout access-group inside_access_in in interface inside access-group outside_access_in in interface outside access-group global_access global route management route inside route inside route outside timeout xlate 3:00:00 timeout conn 1:00:00 half-closed 0:10:00 udp 0:02:00 icmp 0:00:02 timeout sunrpc 0:10:00 h323 0:05:00 h225 1:00:00 mgcp 0:05:00 mgcp-pat 0:05:00 timeout sip 0:30:00 sip_media 0:02:00 sip-invite 0:03:00 sip-disconnect 0:02:00 timeout sip-provisional-media 0:02:00 uauth 0:05:00 absolute timeout tcp-proxy-reassembly 0:01:00 dynamic-access-policy-record DfltAccessPolicy http server enable http management http management http management no snmp-server location no snmp-server contact snmp-server enable traps snmp authentication linkup linkdown coldstart fragment size 2000 inside fragment chain 1000 inside fragment size 2000 outside fragment chain 1000 outside crypto ipsec security-association lifetime seconds crypto ipsec security-association lifetime kilobytes telnet management telnet management telnet timeout 5 ssh management ssh timeout 5 console timeout 0 Page 25 of 27

26 management-access management tls-proxy maximum-session 1000 threat-detection basic-threat threat-detection statistics access-list no threat-detection statistics tcp-intercept ntp server source management prefer webvpn username admin password 2sSINGQnY3ksKp6b encrypted privilege 15 class-map inspection_default match default-inspection-traffic policy-map type inspect dns preset_dns_map parameters message-length maximum 512 policy-map global_policy class inspection_default inspect dns preset_dns_map inspect ftp inspect h323 h225 inspect h323 ras inspect rsh inspect rtsp inspect esmtp inspect sqlnet inspect skinny inspect sunrpc inspect xdmcp inspect sip inspect netbios inspect tftp inspect ip-options service-policy global_policy global prompt hostname context call-home profile CiscoTAC-1 no active destination address http destination address destination transport-method http subscribe-to-alert-group diagnostic subscribe-to-alert-group environment subscribe-to-alert-group inventory periodic monthly subscribe-to-alert-group configuration periodic monthly subscribe-to-alert-group telemetry periodic daily Cryptochecksum:26ca34f2c601fb68b7ab6d898d73531e : end Page 26 of 27

27 APPENDIX C: Firewall Services Module testing The current Catalyst 6500 FWSM (WS-SVC-FWM-1-K9) running 4.1(2) sw version was also tested in the configuration previously documented within the following topology. Figure 15: FWSM lab topology Prior to 4.0(2) code, the first packet of a UDP connection could not be more than 8500 bytes. With the following releases this limitation has been overcome and this product can now successfully pass AVID ISIS traffic. However this is a firewall with an old architecture and processor, therefore not powerful enough to support such an intensive task as reassembly and fragmentation of such large datagrams.. This limited the throughput to just 30MB/S and introduces approx 4ms of latency which makes it unsuitable for large scale deployment. Page 27 of 27