iscsi Clusters High-Availability White Paper

Transcription

1 iscsi Clusters High-Availability White Paper

2 Introduction iscsi is a relatively new technology in the mainstream storage arena, yet the functionality of products provided by iscsi vendors like EqualLogic is exceptional. iscsi provides disk connectivity over the network yet presentation to the OS/user is such that the array(s) appear as a locally attached disk and can be manipulated in the way a directly attach disk can be. However, all the disk management is off loaded to the EqualLogic array - RAID functions, snapshots, remote mirroring and much more are all performed by the arrays on board processors which also provide the management interface - a Java applet GUI. Figure 1. iscsi Example Figure 1 shows an example server connecting over the network to an EqualLogic array, which uses RAID locally and, in this example, is configured to mirror the data to another EqualLogic array in another location. The capabilities of iscsi storage generally and specifically the additional functionality provided by EqualLogic is outstanding. There is a down side though - the iscsi protocol is a form of encapsulation and is transmitted over the network. The performance of SCSI or Fibre Channel attached storage will always out perform that of network attached storage. This paper outlines how customers might choose deploy iscsi arrays in a clustered environment. Traditional Model Figure 2 shows the traditional deployment architecture of a complete multi-tier solution; Web servers today are commonly used in large numbers and a load balancer is often deployed to auto select the most appropriate web server to handle a specific client session. The data on the web server will change periodically but is often fairly static. The data could be provided to the web servers by a clustered NFS service but this of course requires management and resources. Native Clients Wireless Clients Browser Clients The applications, in this example, store all the user data and presentation customisation in the database. However all the nodes have their own copy of the application software binaries. The mirror/raid disks connected to the Oracle servers may have hardware RAID features but the mirroring function is usually performed by the OS on the database servers. The distance between the database servers is limited by the distance supported by the physical interconnects to the disks systems, for SCSI this is typically a matter of metres but for fibre the distances can be much larger. However, the cost of using fibre for long distance is often prohibitive and while many organisations would like real time remote copies, they can not afford it. Mid-Tier & Web Server AR System Server Figure 2. Traditional Application Stack Databases can be replicated using third party or proprietary database replication technologies but replicating data outside the database is more complex. User directories, boxes, application configuration and binaries are all examples of disk data that changes and if a clustered solution over distance is to be deployed then a mechanism must be employed to replicate this data. Oracle Servers Clustered Mirror/RAID Disks

3 Adapting the Model with iscsi Figure 3 shows the logical connections between the servers and EqualLogic s PS series iscsi array; Database Servers Figure 3. iscsi Adaption of the Model iscsi Storage The servers are able to concurrently access the array but the data is protected by RSF-1, which controls access to the disks, only mounting a volume when RSF-1 knows it is safe to do so. The application servers and web servers could also connect to the array and all of the application binaries could be stored there. This would greatly ease upgrades and avoid having to run and maintain an NFS service. An example of a complete replicated solution is shown in Figure 4; Native Clients Wireless Clients Browser Clients The replication of the data is completed by the arrays at the block level. iscsi Replication The arrays are configured to only allow specific machines (IP addresses) to access the appropriate partitions and RSF-1 also controls access by mounting the file systems only on the server(s) which are intended to actively run the service. Figure 4. Complete Replicated Example Fronnt End Web Server Application Server Oracle Servers Clustered RSF-1 is configured to have disk heartbeats on each of arrays. In this example two nodes on a main site act as primary and secondary servers for a given application and the backup site acts as a tertiary failover site. Multiple services can be configured and even run in active-active configuration under RSF-1 but for sake of clarity these options are not expanded in this paper. While the replication provided by EqualLogic PS series arrays very innovative it is block level replication and this is not a best practice method of replicating databases. Databases use complex caching and should normally be replicated at the transaction level - usually using a form of roll-back scraper. Database performance is also a consideration and in many cases direct attach will be preferred over NAS/iSCSI connections for this reason. iscsi Storage However, using a combination of database replication and block level replication for other data is an excellent approach to developing highavailability solutions in a distributed environment.

4 Example Installation Process The following is an example of the process to configure a Solaris 10 cluster with RSF-1 and EqualLogic PS series arrays. Configuration of the Array Setup the basic parameters for the array and create a group; Figure 5. EqualLogic Array Web Admin Interface Now create a volume for RSF-1 heartbeats and permit all appropriate machines to have access to the volume. Note that snapshots are not enabled for heartbeat volumes; Figure 6. Adding a Heartbeat Volume For data volumes, snapshots can be enabled but access to these would not normally be provided to the servers.

5 PWR WARN ALARM Configuration of Solaris Figure 7. Example Sun Setup Solaris 10 (1/06) has iscsi support built into the kernel but by default it is not enabled. To start the iscsi drivers enable the service; solaris10 (root)# svcadm enable /network/iscsi_initiator This example uses no authentication and an auto discovery method. However authenticationmethods, like CHAP, are recommended and static definitions of iscsi devices may be preferred by some administrators. In this example the array group has been assigned an IP address of To enable auto discovery of known nodes; solaris10 (root)# iscsiadm modify discovery --static enable --sendtargets enable To connect to the array group ; solaris10 (root)# iscsiadm add discovery-address Access to the defined group may now be verified; solaris10 (root)# iscsiadm list discovery-address -v To create the IO/device nodes in Solaris; solaris10 (root)# devfsadm -Cv format can now be run and should produce output similar to the following; solaris10 (root)# format Searching for disks...done AVAILABLE DISK SELECTIONS: 0. c0t0d0 <SUN72G cyl alt 2 hd 24 sec 424> /pci@1d,700000/scsi@4/sd@0,0 1. c0t1d0 <SUN72G cyl alt 2 hd 24 sec 424> /pci@1d,700000/scsi@4/sd@1,0 2. c0t8d0 <SUN72G cyl alt 2 hd 24 sec 424> /pci@1d,700000/scsi@4/sd@8,0 3. c0t9d0 <SUN72G cyl alt 2 hd 24 sec 424> /pci@1d,700000/scsi@4/sd@9,0 4. c3t5d0 <EQLOGIC-100E cyl alt 2 hd 16 sec 200> /iscsi/disk@0000iqn com.equallogic%3a6-8a0900-e0d331e01-c8d cf-nviron0001,0 5. c3t6d0 <EQLOGIC-100E cyl 28 alt 2 hd 16 sec 63> /iscsi/disk@0000iqn com.equallogic%3a6-8a0900-f0c331e01-5c e44311-hb0001,0 The volume name is embedded in the path and can be used to distinguish between volumes. Disk c3t6d0 can be partitioned with a single partition (s0) and used for sending heartbeats between RSF-1 instances. The c3t5d0 disk should be placed under the control of RSF-1.