DX Cluster - Specific Configuration and Troubleshooting

DX Cluster - Specific Configuration and Troubleshooting 1. - Topic Default DX cluster configuration works perfectly in front of the vast majority of web applications. But in few cases, some web applications require specific DX tuning. This describes: What are the default DX clusters features enabled (Chapter 3) Known web applications that require specific DX configuration (Chapter 4) Troubleshooting steps when web application doesn't work with DX default configuration (Chapter 5) 2. Important Note DX is simple to integrate in front of any application. It should only require 15-30 minutes. Depending on the customer application and DX license, three different types of Virtual IPs can be configured on DX. SLB-VIP for any TCP/UDP application Mainly used for non web applications (ldap, dns, ftp, ) on DX with any license. Or web applications (http/https) on DX devices with "SLB" license only. Benefits are scalability (target servers load balancing) and availability (target servers healthcheck). Forwarder-VIP for any SSL application Mainly used for SSL termination on non web applications running on 1 TCP port (ldaps, smtps, ) on DX with any license. Clients talk on SSL (e.g.: pop3s) up to the DX and DX talks in clear text (e.g.: pop3) to the servers. Or web applications (https) on DX devices with "SLB" license only. Clients talk on SSL (https) up to the DX and DX talks in clear text (http) to the servers. Benefits are the same as SLB-VIP+ performance (target servers SSL off-load). Cluster-VIP for any web application (http/https) Used for all web applications (http/https) on DX devices with "HTTP Acceleration" or "HTTP Advanced" license. Extra benefits in addition to Forwarder-VIP are too many to be detailed here. They can be summarized in six topics: better performance, better scalability, better availability, better security, better flexibility and better vision/management/reporting. Note: For further technical information on these extra benefits, refer to "DX Evaluation Steps". This document is focused on Cluster-VIP. Page 1 --- DX Cluster Specific Configuration and Troubleshooting August 2006

3. What is default DX Cluster configuration Compression enabled Target server replies to DX with no compression and DX replies to clients with compression. Note: Benefit is less data transfer better response time for end users and less bandwidth utilization. Connection Binding disabled - in other words TCP multiplexing enabled DX terminates all clients TCP sessions and it uses same persistent concurrent sessions to the target servers to send all clients requests. This technology is usually called "target server TCP multiplexing". Note: Benefit is fewer sessions and no more open/close sessions on target servers target servers faster and increase server capacity. Target servers stickiness disabled DX load balances each new client request to the target server that is currently processing the fewest amount of requests (so the least busy). So a specific client can have its different requests sent to different target server. If application requires stickiness (same user has to remain to same target server for the full communication), this must be enabled. For best load sharing we suggest cookie sticky instead of clientip sticky. Note: Benefit is best load distribution on target servers farm better response time for end users. OWA disabled Few applications (for instance Outlook Web Access and Sharepoint from Microsoft ) use specific advanced http methods/options that are only authorized on DX with this OWA option. Note: Benefit is best security security improved. Via and Warning headers enabled DX Cluster is as a reverse proxy. It adds extra "Via" and "Warning" headers to notice clients and servers they don't talk directly to each other but through a Proxy. Client IP transparency disabled To allow DX to provide target servers off load with TCP multiplexing technology, DX acts as a Reverse Proxy. So only IP@ target servers see is DX-IP@. Apprule disabled AppRules allows administrators to modify, deny, or route each request from clients and do the same with server responses based on anything in the HTTP(S) stream. Since 5.1, each DX appliance comes with "nitro.apprule" that provides browser cache optimization, DX caching and signature of threats. This "nitro.apprule" is not applied by default. Note: "DX evaluation steps" details benefits of "nitro.apprule" and how to apply it to a cluster. DX Caching disabled By default DX cache is disabled. Note: "DX evaluation steps" details benefits of it and how to apply it to a cluster. AAA disabled By default Authentication is disabled. Page 2 --- DX Cluster Specific Configuration and Troubleshooting August 2006

4. Known web applications that requires specific DX configuration This section lists all applications known that requires specific DX configuration. All web applications with NTLM authentication (section 4.1) Domino Web Server (section 4.2) Endeca (section 4.2) IBM WebSphere (sections 4.2 and 4.4) inotes6 (section 4.2) MS Project (section 4.5) Outlook Web access (see specific OWA AppNote) Peoplesoft JD Edwards(section 4.4) Sharepoint (see specific Sharepoint AppNote) Siebel (section 4.6) WebLogic (sections 4.2 and 4.3) Page 3 --- DX Cluster Specific Configuration and Troubleshooting August 2006

4.1. Web application with NTLM authentication Why the DX needs a specific configuration NTLM is a proprietary protocol that authenticates connections rather than users or requests. Therefore, multiplexing connections to the target server must be disabled to avoid violating the NTLM authentication scheme. The customer web administrator can confirm if the application is using NTLM. Alternatively, there are two methods to determine if the web application is using NTLM authentication. 1. On IIS server properties, check if Integrated Windows Authentication is enabled under Authentication Methods : 2. Using a browser with HTTPWatch enabled when an application request for user authentication is generated (first server response code 401): Page 4 --- DX Cluster Specific Configuration and Troubleshooting August 2006

What is the DX specific configuration DX needs to disable TCP multiplexing. This is done with Connection Binding enabled. In addition for better performance, some extra settings must be configured also: set cluster <name> factory c uar enabled To compress 4xx response. IIS with NTLM authentication starts first with two 401 response code for each new client TCP session to authenticate session. Compression on 401 response provides great benefits with slow clients and bandwidth reduction. set cluster <name> factory h tc3 disabled To not close TCP session after a 304 response code. By default DX closes TCP session after 304 responses, as several Apache versions don't support session persistency after a 304. But IIS doesn't have this issue. set cluster <name> factory h w disabled set cluster <name> factory h v disabled DX is a reverse proxy. It adds extra "Via" and "Warning" headers to notice clients and servers they don't talk directly to each other but through a proxy. In that case IIS with NTLM authentication will ask for client authentication after each request. Browsers does the re-authentication transparently (there is no extra user prompt), but that's pointless traffic and slowness for end-users. write To apply and save configuration change. Page 5 --- DX Cluster Specific Configuration and Troubleshooting August 2006

4.2. Applications that don't support DX persistent connections Application list: Domino Web Server (persistent connections may be supported on some versions) Endeca IBM WebSphere (persistent connections may be supported on some versions) inotes6 WebLogic Why the DX needs a specific configuration These applications doesn't support (like applications with NTLM authentication on IIS) having different end-users coming in the same TCP sessions. What is the DX specific configuration DX needs to disable TCP multiplexing. This is done with Connection Binding enabled. set cluster <name> factory h w disabled set cluster <name> factory h v disabled write Page 6 --- DX Cluster Specific Configuration and Troubleshooting August 2006

4.3. Applications that need client IP@ I information Application list: WebLogic (only if Weblogic application needs client IP@ information) Why the DX needs a specific configuration Depending on Weblogic utilization, this application may needs client IP@ information. This can be done with 2 different ways: DX configured with ClientIP Transparency option (see ClientIP Transparency AppNote) But this prevent the DX doing TCP multiplexing to target servers and so prevent the complete target server offload Weblogic accepts to get the client IP information in a specific header instead of source-ip These applications doesn't support (like applications with NTLM authentication on IIS) having different end-users coming in the same TCP sessions. What is the DX specific configuration One of the two following options can be chosen. For performance reasons, we suggest the second one. DX configured with ClientIP Transparency option See ClientIP Transparency AppNote. Weblogic accepts to get the client IP information in a specific header instead of source-ip 1. For BEA Weblogic EARLIER than version 6.1: On the DX: In CLI: Dx%set server customiplogheader X-Forwarded-For In WebUI: 2. For BEA WebLogic version 6.1: On the DX: In CLI: Dx%set server customiplogheader WL-Proxy-Client-IP In WebUI: Page 7 --- DX Cluster Specific Configuration and Troubleshooting August 2006

On BEA: In your Admin Console, they need to set the "WeblogicPluginEnabled" field to "true". Page 8 --- DX Cluster Specific Configuration and Troubleshooting August 2006

4.4. Applications that needs apprules if Cluster TCP port is different than server TCP port Application list: IBM WebSphere (only needed if Cluster TCP port is different than server TCP port) Peoplesoft JD Edwards (only needed if Cluster TCP port is different than server TCP port) Why the DX needs a specific configuration These applications run on specific TCP port (for instance 9000) and DX Cluster is running on another one (for instance 80). These applications check request header "Host" and use redirection. Request header "Host" and redirections are with TCP information. This needs to be changed to DX Cluster TCP port and achieved using an Apprule. What is the DX specific configuration 3. In WebUI: Create Apprule that modifies the http "Host" request header and http application redirects Services AppRules Create RuleSet o Type a name (for this example PortMapping ) and click OK Click Request Translation Header tab New Rule o Copy and Paste (change appli.foo.com to the application hostname and TCP ports) RTH: request_header "Host" eq "appli.foo.com:80" then update_request_header "Host" "appli.foo.com:9000" o Click OK Page 9 --- DX Cluster Specific Configuration and Troubleshooting August 2006

o o Copy and Paste (change appli.foo.com to the application hostname and TCP ports) RTH: request_header "Host" eq "appli.foo.com" then update_request_header "Host" "appli.foo.com:9000" Click OK Click Page Translate Header tab New Rule o Copy and Paste (change appli.foo.com to the application hostname and TCP ports) PTH: http_reply_code eq "302" and reply_header "Location" sw "http://appli.foo.com:9000" then replace reply_header "Location" term "http://appli.foo.com:80" o Click OK o o Copy and Paste (change appli.foo.com to the application hostname and TCP ports) PTH: http_reply_code eq "301" and reply_header "Location" sw "http://appli.foo.com:9000" then replace reply_header "Location" term "http://appli.foo.com:80" Click OK Page 10 --- DX Cluster Specific Configuration and Troubleshooting August 2006

4. In WebUI: Services Cluster Groups select existing cluster from the list AppRules o Enabled RuleSet Checked o RuleSet to Run PortMapping Page 11 --- DX Cluster Specific Configuration and Troubleshooting August 2006

4.5. Applications that don't support chunked mode Application list: MS Project Why the DX needs a specific configuration Since HTTP 1.1, web servers can reply to clients requests with or without specifying a response object size. If web servers specify response object size in response, they use response header "Content-Length". If web servers don't specify response object size in response, they use response header "Transfer-Encoding: Chunked". With this mode web servers specify to client the end of data with 2 carriage returns. Both methods are perfectly supported by all browsers. For better performance, by default the DX uses by default chunked mode when it replies compressed content. Some applications (running on the top of the browser) doesn't support chunked mode and DX has to use Content-Length. What is the DX specific configuration DX needs to "ForceContentLength" utilization. set cluster <name> factory a fcl 1 write Page 12 --- DX Cluster Specific Configuration and Troubleshooting August 2006

4.6. Applications that don't support compression Application list: Siebel (only some versions requires this change Siebel 7.7 and higher) Why the DX needs a specific configuration Web compression has been supported since HTTP 1.1 with browsers supporting it for a few years (from IE4.0 and Netscape 4.0). Compression is done via the gzip and deflate standards. Some browsers have some bugs uncompressing content. To avoid compression issues, DX by default doesn't do compression for IE4.0 and Netscape 4.0, as they have too many bugs related to compressed contents. Latest browsers work well with compression (all our customers have compression enabled), but some applications use a browser plug-in or java/activex client (as pdf with Adobe) which may request compressed content but have problems handling it. For example, Siebel uses an ActiveX client to do most of the POST requests where other requests are really done by Internet Explorer. This Siebel client doesn't correctly support deflate compression. What is the DX specific configuration Siebel client presents itself as IE but supports only deflate where IE supports both deflate and gzip. So workaround is to force DX to use gzip only for IE browsers. So DX responses to Siebel clients won't be done with compression and requests done by IE browsers will be compressed by gzip. In WebUI (from release 5.2): In Services Cluster Compression Browser Types Show Detail In CLI: set cluster <name> compression browser ie50 1 set cluster <name> compression browser ie51 1 set cluster <name> compression browser ie55 1 set cluster <name> compression browser ie6 1 set cluster <name> compression browser ie7 1 set cluster <name> compression browser ieother 1 Page 13 --- DX Cluster Specific Configuration and Troubleshooting August 2006

5. Troubleshooting steps when web application doesn't work with DX default cluster configuration If your web application is not listed in previous chapter and default DX Cluster configuration doesn't work, this chapter details steps that can help. 5.1. Do you use same Application name to access application? Some web servers pay attention to the "Host" header to deliver content. So DX access to application must be done via the same "Host" (e.g.: http://sap.foo.com) and not via Cluster-VIP (e.g.: http://10.80.80.252). To test it transparently without any change on application or DNS, change the PC hosts file (c:\windows\system32\driver\etc\hosts) and add the line at the end of the file: 10.80.80.252 sap.foo.com Note: In this example "10.80.80.252" = ClusterVIP@ and "sap.foo.com" = application hostname 5.2. Do you use ActiveN? ActiveN option allows DX devices to work in active/active/active configuration. When using activen, you must define clusters with DSR option enabled on all DX. If you forgot this step, then you have an IP conflict. If it's still not working, test without ActiveN to facilitate troubleshooting and see if issue is related to ActiveN or something else: Keep failover on all DX ("Admin" "Failover") Disable ActiveN on all DX ("Services" "ActiveN" "Default ActiveN Settings") Page 14 --- DX Cluster Specific Configuration and Troubleshooting August 2006

Disable DSR in all Clusters in all DX ("Services" "Clusters" "Cluster Groups") If issue is related is due to ActiveN, contact Juniper JTAC to determine the ActiveN configuration issue. 5.3. Validate application doesn't use NTLM authentication Most applications that require end-user authentication that run on IIS are using NTLM authentication. If that's the case, go to section 4.1 to see DX specific configuration. 5.4. Validate application doesn't require stickiness Most enterprise applications require stickiness (same user has to remain to same target server for the full communication). For best load sharing we suggest cookie sticky versus clientip sticky. Technical Note: Cookie stickiness provides target servers distribution in any environment, where Client IP stickiness can't (e.g.: clients behind Proxy or Mega-Proxies, Firewall/router doing NAT) 5.5. Test without DX persistent connections As explained in section 4.2, few applications don't support having different end-users coming in same TCP sessions. Test with DX specific configuration in section 4.2. Page 15 --- DX Cluster Specific Configuration and Troubleshooting August 2006

5.6. Test with advanced http methods Few applications (for instance Outlook Web Access and Sharepoint from Microsoft ) use specific advanced http methods/options. These advances http methods/options are dropped by default on DX to protect web servers that don't use them. Note: OWA and Sharepoint have their own AppNote if you need to configure DX for these. To authorize advanced http methods, in WebUI: Services Cluster Groups select existing cluster from the list Advanced o Extended Methods Enabled o WebDAV Methods Enabled 5.7. Test without compression We never met an issue where we had to remove completely compression from DX Cluster configuration. But some specific server headers response may meet some browsers bugs (as javascript compression and Cache-Control: No-Cache response header on IE - http://support.microsoft.com/default.aspx?scid=kb;en- US;Q327286) Disabling compression quickly allows to test if issue is related to compression. In WebUI (from release 5.2): In Services Cluster Compression In CLI (from 5.2 release): set cluster <name> compression policy disabled To enable compression: policy = enabled In CLI (in 5.1 release): set cluster <name> compression policy 1 To enable compression: policy = 0 In CLI (before 5.1 release attention that's a global setting): set server factory c p 1 To enable compression: policy = 0 If disabling compression fixed issue, please contact Juniper JTAC. They'll help to find a workaround to keep completely or partially compression and its benefits. Page 16 --- DX Cluster Specific Configuration and Troubleshooting August 2006

5.8. Test with Forwarder orwarder-vip Forwarder on DX is simply a TCP proxy VIP that provides load balancing and target server availability. Replacing Cluster by Forwarder, you simply remove all AFE (Application Front End) features, as compression, TCP multiplexing, sanity check, etc. Any application should work through Forwarder. Once this step has been tested successfully, contact anyway Juniper JTAC. They'll help to see if this application can work through Cluster with advanced tuning and thus bring you extra benefits (compression, server off load, etc). 5.9. Test with SLB-VIP SLB on DX is the same as Forwarder VIP, but doesn't terminate TCP session. Any application should work through SLB. Once this step has been tested successfully, contact anyway Juniper JTAC. They'll help to see if this application can work through Cluster with advanced tuning and thus bring you extra benefits (compression, server off load, etc). Page 17 --- DX Cluster Specific Configuration and Troubleshooting August 2006