Linux Multipathing Solutions: MPP vs. DM-RDAC A Dell Technical White Paper Dell TM PowerVault TM MD Series Storage Arrays
THIS WHITE PAPER IS FOR INFORMATIONAL PURPOSES ONLY, AND MAY CONTAIN TYPOGRAPHICAL ERRORS AND TECHNICAL INACCURACIES. THE CONTENT IS PROVIDED AS IS, WITHOUT EXPRESS OR IMPLIED WARRANTIES OF ANY KIND. 2010-2011 Dell Inc. All rights reserved. Reproduction of this material in any manner whatsoever without the express written permission of Dell Inc. is strictly forbidden. For more information, contact Dell. Dell, the DELL logo, and the DELL badge, and PowerVault are trademarks of Dell Inc. Other trademarks and trade names may be used in this document to refer to either the entities claiming the marks and names or their products. Dell Inc. disclaims any proprietary interest in trademarks and trade names other than its own. January 2011 Page ii
Contents Introduction... 2 MPP/DM RDAC Overview... 2 MPP Driver Overview... 2 DM-RDAC Overview... 2 Similarities of MPP and DM-RDAC... 3 Differences Between MPP and DM-RDAC... 4 Coexistence of MPP and DM-RDAC... 4 Page 1
Introduction Dell PowerVault MD3000 and MD3000i products currently use a proprietary Linux MPP driver for multipath support. The next generation of Dell PowerVault MD3200/i, MD3220/i, MD3600i, and MD3620i family of products will now use the native Linux Device Mapper (DM) multipathing for multipath support. This white paper will provide a comparison between the two multipathing architectures, along with a suggested migration strategy. MPP/DM RDAC Overview MPP Driver Overview The Linux MPP multipathing driver is a proprietary driver designed to handle device multipathing for Dell PowerVault MD3000 and MD3000i arrays. The following is a list of its features and characteristics: The MPP driver is installed as an external driver package and does not integrate with the native OS stream The MPP driver contains two kernel modules mppupper and mppvhba to handle core multipathing functionality The mppvhba driver handles the I/O path tasks and key failover functionalities The mppupper driver handles device aggregation and device masking The MPP driver contains load balancing policies to handle IO load balancing among multiple paths to the array The MPP driver contains a separate user space utility (mpputil) to process tasks such as displaying multipathing topology and hot addition of devices DM-RDAC Overview The Linux DMPP multipathing driver is a community architected and supported driver designed to handle device multipathing. The Dell PowerVault MD3200 and MD3200i series of products will only use DMMP. The following is a list of its features and characteristics: The DMMP multipathing driver lays above the disk layer and performs the following tasks: o It intercepts I/O requests coming down from the application/file system. o Its path selector module chooses a path through which the device will dispatch I/O. o In the event of path failure, it will choose an alternate path to dispatch I/O. If a failover is needed, a failover request will be initiated. The RDAC device handler processes vendor specific tasks including: Page 2
o o o Vendor specific command construction for failover and failback Processing returned status for failover/failback Processing vendor specific sense keys for I/O requests The DM-RDAC pair also contains a user space package which handles the communication between user space and kernel space. It contains: A multipathing daemon that handles tasks such as path aggregation and failed path checking The multipathing disk partition utility kpartx The Dell-provided script rescan_dm_devs to dynamically refresh the multipathing maps. This is very useful for hot addition of new devices and new paths. For more details regarding DM-RDAC architecture, refer to Appendix A DM-RDAC Stack Diagram. Similarities of MPP and DM-RDAC MPP and DM-RDAC have the following similarities: They both aggregate device paths according to the device s unique worldwide name (WWN) Both perform immediate failover if all paths to one controller have failed They both perform regular checks on failed paths to determine if any of them have been restored Upon restoration of a failed path, they both perform failback to restore LUN ownership to the preferred controller They both have load balancing policies to more efficiently use multiple paths to a single controller They both have a user configuration file to enable user space parameter tuning during run time Page 3
Differences Between MPP and DM-RDAC MPP DM-RDAC Dell supported arrays MD3000I and MD3000i MD3200, MD3200i, & MD3600i Series OS support No support from OS vendor Native support from OS vendor OS layer of multipathing component Path aggregation Universal Transport Device Hot addition of new devices and targets Command queuing in all paths failure Detection for failed paths restoration Load Balancing policies Disk partitioning Multipathing service restarting Dynamic user space parameter change Above disk layer One disk device node (/dev/sd<x>) per virtual device Hidden from user space, no disk device node created. Automatically reconfigures the device upon low level device discovery No command queuing functionality. All commands returned error immediately upon all paths failure. Initiated from kernel space Supports round robin and least queue depth. Standard disk utilities such as fdisk works Multipathing service cannot be manually restarted. Host reboot required to reload. No support for run time change. Host reboot is necessary after changes to configuration file. At disk layer One multipathing device node (/dev/mapper/mpath<x>) associated with multiple path nodes (/dev/sd<x>) Presented to user space by /dev/sd<x> Script rescan_dm_devs has to be run upon low level device discovery Command queuing for 150 seconds by default. Tunable via user space parameters. Initiated from user space Currently only supports round robin. Upstream efforts continuing on other policies. User space utility kpartx has to be run after fdisk to make partition visible to multipathing daemon Multipathing service can be restarted by restarting multipathing daemon. Run time change is achieved by restarting multipathing daemon. Coexistence of MPP and DM-RDAC Dell supports coexistence of MPP and DM-RDAC on the same host. The following details are added to the driver for coexistence: 1. For the MPP driver, add module parameter white_list to indicate what devices are allowed to be managed by the MPP driver. It is set to MD3000/i PowerVault devices. 2. For the DM-RDAC driver, add module parameter black_list parameter rdac_blacklist to indicate what devices are blacklisted from the RDAC driver. It is set to MD3000/i PowerVault devices. Page 4
3. When MD3000/i devices and MD3200/i (or MD3600i) devices are connected to the same host, the MPP driver manages only the MD3000/i device stack. MD3200/i (or MD3600i) devices are only managed by the DM-RDAC driver stack. They don t interfere with each other. In this way, coexistence can be supported. Appendix A: DM-RDAC Stack Diagram Multipath deamon 1. Check added path and remap 2. Path failback 3. Resubmit Q ed IOs. User Application ( kpartx) Create partition on Multipath device Rescan _dm _devs script Control interface Block Interface Core device- mapper driver module Mapping/target interface Linear Mirror Snapshot Multipath Core 1. Provides single device node for multipath device 2. Initiate failover on I/O error Path Selector (round robin) Block Layer Scsi middle level RDAC hardware handler SCSI low level HBA Page 5