Adaptec by PMC ASR Gb/s PCIe RAID Adapter Review Raid Testing With 8 HGST 12Gbps SSDs

Transcription

1 2013 Adaptec by PMC ASR Gb/s PCIe RAID Adapter Review Raid Testing With 8 HGST 12Gbps SSDs Ben Chase The SSD Review 9/30/2013

2 When it comes to high performance RAID add-in cards, there are really only two big players in the market: Adaptec and LSI. These two have been duking it out for years, each one constantly trying to one up the other, with LSI often coming out on top. To give you a bit of history, let s take a look back at the past year. At this point in 2012, LSI was on top of the world. The LSI MegaRAID SAS 9265/66 was the RAID card to beat and had been for almost a year. While LSI was leading the performance market, Adaptec was still getting its feet settled after the PMC-Sierra acquisition from In October of 2012, Adaptec released the Series 7 line of RAID adapters. For the first time in years, LSI was put on notice. As we reviewed earlier this year, the Series 7, specifically the ASR-72405, had a few tricks up its sleeve. Instead of relying on low native port counts and expanders, Adaptec released a RoC (RAID-on-Chip) that had 24 native ports. This allowed for a massive upgrade in bandwidth over an 8-port design. As the summer of 2013 approached, LSI struck back with their SAS 9300 line of 12Gbps controllers. Even though port counts stayed at 8, the doubling of bandwidth made the SAS 9300 very competitive with the Series 7, even if their approaches were completely opposite. At this point, we figured that the market would stabilize for a while, with high port count (Adaptec) battling it out with high port bandwidth (LSI). We were pleasantly surprised when Adaptec approaches us just a month later asking us to take an exclusive good look at their new Series 8 RAID Adapter The SSD Review. All Rights Reserved. Page 1

3 The Series 8 gives us the best of both worlds; high port count (16) and high port bandwidth (12Gbps). There are actually 5 different adapters in the Series 8 family. The first is the ASR ZQ which has 16 internal ports with integrated Zero Maintenance Cache Protection. Next we have the ASR-8885 and ASR-8885Q. Both have 8 internal and 8 external ports, with the 8885Q coming with the AFM-700 flash-based cache protection module. Adaptec also has the ASR-8805, which has only 8 internal ports. Finally, the ASR-8885E, which is identical to the ASR-8885, but only has 512MB of cache protection, versus 1024MB for all other flavors. It also only supports RAID levels 0, 1, 1E and 10, which might be a deal breaker for many. All other version support RAID levels 0, 1, 1E, 5, 6, 10, 50, 50 and 60. The ASR-8885 lists for $725, while the ASR-8805 is $640. The bargain of the group is the ASR-8885E, which lists for just $440. All versions are MD2, low profile cards that should be at home in a server environment. They connect via a x8 PCIe GEN3 interface. All internal ports are connected via SFF-8643 connectors, while the external ports use SFF You may also see them referred to as min SAS HD, but they are the same connectors used on the Series 7 and LSI SAS Unlike LSI, which initially released a 12Gbps HBA with a RAID adapter to follow on, Adaptec is leading with the RAID adapter and leaving the HBA for later this year. When it comes to performance, Adaptec is leading the way. The Series 8 has the same 6.6GB/s sequential read rate as the Series 7, but doubles the RAID 5 sequential write speed from 2.6 to 5.2GB/s. Maximum IOPS also jumps from 450K to 700k! Since LSI only has an HBA available, we decided to pit the Adaptec ASR-8885 against its sibling, the ASR In our Adaptec review we were able to get impressive numbers when paired with 24 SMART Optimus SSDs. We got even crazier numbers when we doubled up the number of RAID adapters. While we didn't have access to the SMART Optimus SSDs for this comparison, we got something just as impressive. HGST sent us over eight SSD800MM 12Gbps SAS enterprise SSDs! These are the same drives that stole the show in our LSI 12Gbps review earlier in the year. The HGST SSD800MM is a monster of a device. We have never seen performance anywhere close in a 2.5" SATA/SAS form factor. With sequential read/write speeds of 1150/700MB/s and random read/write speeds of 145K/70K IOPS, the SSD800MM is the perfect drive to stress the Series 8. Before we do that, lets tear into the ASR The SSD Review. All Rights Reserved. Page 2

4 INTRODUCTION Front and center on any RAID adapter is the RAID-on-Chip (RoC). For the Series 8, Adaptec rolled out the PM8063. This chip allows the ASR-8885 to connect to the host via x8 PCIe GEN3, with 8GB/s of bandwidth. It also allows inline XOR to reduce DDR3 access. On the adapter side, the PM8063 had 16 ports at 12Gbps. Even though the ASR-8885 lists 1GB of DDR3 RAM, there are actually five Micron DRAM packages, each with 256MB each, for a total of 1.25GB. Three are located near the controller and heat sink, with 2 more on the back The SSD Review. All Rights Reserved. Page 3

5 Near the back of this MD2 PCIe card, you can see the connector used for the AFM-700 flashbased cache protection module. Finally, you can see the SFF-8643 internal connectors and SFF-8644 external connectors. With both external and internal ports, the ASR-8885 has a lot of flexibility The SSD Review. All Rights Reserved. Page 4

6 TEST BENCH AND PROTOCOL In testing the Adaptec ASR-8885, along with all enterprise products, we focus on long term stability. In doing so, we stress products not only to their maximum rates, but also with workloads suited to enterprise environments. We use many off-the-shelf tests to determine performance, but we also have specialized tests to explore specific behaviors we encounter. With enterprise drives, you will see that we do not focus on many consumer level use-cases. Our hope is that we present tangible results that provide relevant information to the buying public The SSD Review. All Rights Reserved. Page 5

7 SOFTWARE As we covered in our Series 7 review, there are many ways to access your Series 8 adapter. The maxview Storage Manager is a browser-based application that gives you access to all of the major functions of the adapter. It is clean, simple and easy to use. Adaptec also includes a command line interface (CLI) where you can control most functions of the adapter. In fact, we were able script many of our tests using the CLI. If all else fails, you can always configure the adapters via the option ROM during POST. CONFIGURATION The Adaptec ASR-8885, like all RAID controllers, can be configured in countless ways. Unfortunately, it would take us months to test them all, so we are using the most common modes. As recommended by the MaxVIEW Storage Manager, the write caching was disabled for all tests. All other options on the controller and the logical drive were left in their default states. The Series 8 adapters also support multiple modes of operation: Physical Drive, Simple Volume and RAID. Physical drive mode operates the adapter in HBA mode, where all caching is disabled. Simple volume mode adds metadata to each drive so that you can use write caching. RAID mode opens up all of the features of the adapter. All of our tests were performed in RAID mode The SSD Review. All Rights Reserved. Page 6

8 LATENCY To specifically measure latency, we use a series of 512b, 4K, and 8K measurements. At each block size, latency is measured for 100% read, 65% read/35% write, and 100% write/0% read mixes. Normally, with single SSDs, we limit queue depth to 32, but on RAID cards with multiple drives, or other devices that can handle it, we push the queue depths much higher. In this case, we found that 384 was a sweet spot for performance. Considering the high queue depths, the response times were great. Both 512 byte and 4K were nearly identical across the board. The maximum latency numbers were also impressive, especially for 100% read operations. This wasn't much of a surprise, since RAID adapters normally give good latency results. Since we had all caching disabled and the access pattern was across the entire span, you can expect even better results when all optimizations are utilized. Now lets take a look at how consistent those results are across many hours of testing The SSD Review. All Rights Reserved. Page 7

9 ADVANCED WRITE TESTING As we talked about in our Micron P400m SSD Review, SSDs have different performance states. Since the Adaptec and HGST products are enterprise, we will focus on steady state performance. With the following tests, we stressed the drive using random 4KB write workloads across the entire span for at least 24 hours. This is more than enough to achieve steady state. The following graph is showing the latency and IOPS across an 11 hour span. Well, we weren't quite expecting that. The first thing that jumps out is that we were getting 500K write IOPS, which is an outstanding number. The second is how incredibly consistent the results were across 11 hours of testing. In 11 hours of testing, once the system achieved steady-state, the average latency per minute varied by less than 2 microseconds. Now lets zoom in on that last hour The SSD Review. All Rights Reserved. Page 8

10 Wow. Even though there is a slope, look at the scale. Over the last hour, the results varied by less than 500 nanoseconds. This is by far the best consistency we have ever seen. Until we get a proper 12Gbps RAID solution from LSI, we won't be able to tell who to heap the praise on, Adaptec or HGST. Until then, we will give the credit to both. Amazing performance The SSD Review. All Rights Reserved. Page 9

11 MEASURING PERFORMANCE As with all of our tests, the following tests were performed after a secure erase of the drive. The drive was also conditioned with a predefined workload until it reached steady state. We also test across the entire span of the drive. We took our ASR-8885 and paired it with both 4 and 8 SSD800MM's from HGST. We then compared it to the ASR with 24 SMART Optimus SSDs and then to a pair of ASR s with the same SMART SSDs, split evenly. It should be noted that with the Series 7, the performance topped out before 16 drives. Even though we did not have 16 identical SSDs on hand, we believe that the Series 8 would be nearly even on most tests, based on specifications. So, we wanted to know what would perform better, a bunch of 6Gbps SSDs or a handful of 12Gbps SSDs. Lets find out The SSD Review. All Rights Reserved. Page 10

12 For sequential writes, eight 12Gbps SSDs were just not enough to match the parallelism and raw power of 24 SMART Optimus SSDs. Even at nearly 700MB/s, the eight SSD800MMs were not quite enough to saturate the ASR We would have needed at least one or two more to clear 6GB/s. Read operations didn't have nearly as many problems reaching 6GB/s. In this test, the 8 HGST SSDs held their own against the 24 SMART SSDs. The Series 8 solution trailed at lower queue depths mainly because there was only 1/3 of the drives to spread the IOs across. At higher queue depths, there was only a 200MB/s difference The SSD Review. All Rights Reserved. Page 11

13 Random operations is where the superiority of the 12Gb solutions shows itself. In 4K writes, the Series 8 was 35% faster than the loaded Series 7. It was still no match to a pair of ASR s, but 500K IOPS is still incredibly impressive. Even more impressive than 500K write IOPS is the 700K read IOPS that we got out of our Series 8 system. A single ASR-8885 with eight SSD800MMs beat out a pair of ASR-72405s with 24 SMART Optimus SSDs at 4 out of 7 queue depths. Even though the Series 8 fell a few thousand IOPS short of taking the crown for most IOPS, it lived up to all expectations The SSD Review. All Rights Reserved. Page 12

14 SNIA IOPS TESTING The Storage Networking Industry Association has an entire industry accepted performance test specification for solid state storage devices. Some of the tests are complicated to perform, but they allow us to look at some important performance metrics in a standard, objective way. SNIA s Performance Test Specification (PTS) includes IOPS testing, but it is much more comprehensive than just running 4KB writes with IOMeter. SNIA testing is more like a marathon than a sprint. In total, there are 25 rounds of tests, each lasting 56 minutes. Each round consists of 8 different block sizes (512 bytes through 1MB) and 7 different access patterns (100% reads to 100% writes). After 25 rounds are finished (just a bit longer than 23 hours), we record the average performance of 4 rounds after we enter steady state. Preconditioning: 3x capacity fill with 128K sequential writes Each round is composed of.5k, 4K, 8K, 16K, 32K, 64K, 128K, and 1MB accesses Each access size is run at 100%, 95%, 65%, 50%, 35%, 5%, and 0% Read/Write Mixes, each for one minute. The test is composed of 25 rounds (one round takes 56 minutes, 25 rounds = 1,400 minutes) The SSD Review. All Rights Reserved. Page 13

15 As we mentioned before, we performed most of our tests at a queue depth of 384. SNIA does not actually specify a specific queue depth, so we default to 32 unless otherwise noted. The Adaptec/HGST solution had absolutely no problems with the SNIA testing. You can see that we hit the card maximums at both 512 bytes and 4KB. We even hit an impressive 650K IOPS for 8KB reads. Across the testing, we saw wonderful scaling, even in some of the most strenuous workloads. Once we hit steady-state, the performance was 100% reliable, whether we ran our tests for a minute, an hour, or a day The SSD Review. All Rights Reserved. Page 14

16 SERVER PROFILES While synthetic workloads do a great job of testing the underlying technology and reporting easy to understand results, they aren t always indicative of how the drive will be used by the end user. Workloads that simulate enterprise environments try to bridge that gap without being overly complex. The database profile is 8K transfers, and 67% percent of operations are reads. Our Series 8 setup took no mercy on the database profile. With nearly 500K IOPS, it easily blew by the Series 7. Only a pair of Series 7 and 3x the number of drives was able to get a higher score, and even then, by less than 15% The SSD Review. All Rights Reserved. Page 15

17 The fileserver profile is based on an 80% read/20% write mix. Its made up of blocksizes from 512 to 64K, each making up a different percentage of the access pattern. The pattern is: 512 bytes=10%, 1k=5%,2k=5%, 4k=60%, 8k=2%, 16k=4%, 32k=4%, 64k=10%. In this test, the Series 7 and 8 were pretty evenly matched. The larger block sizes probably account for the closeness of the results. The 24 SSDs in the Series 7 setup give it a clear advantage with large block sequential performance The SSD Review. All Rights Reserved. Page 16

18 The webserver profile is similar to the fileserver profile, but has some additional 128K and 512K accesses thrown in for good measure. Additionally, the profile is 100% read. Considering the webserver profile is so close to fileserver, but with more large block transfers, it was no surprise that the Series 8 fell behind the Series 7. Now is a good time to reiterate that the Series 8 would do just as good, or better than the Series 7, given the same drive setup. We were more interested in seeing if a 12Gbps solution was superior at lower drive counts. Except for large block sequential transfers, where the number of drives play such an important role, the Series 8 held its own in many of the tests The SSD Review. All Rights Reserved. Page 17

19 REPORT ANALYSIS AND CONCLUSION Before we get to our final thoughts, we want to spend a minute thanking HGST for providing us with eight SSD800MM SSDs. There is no way we could have achieved the results that we did without HGST's support. As we found out in our LSI 12Gbps review, the SSD800MM is a great drive with amazing, consistent performance. With that said, we also have to take our hats off to Adaptec. Their ASR-8885 is a great RAID adapter. It offers 16 ports of 12Gbps SAS with amazing throughput. We had no issues hitting 700K IOPS and 6GB/s. Those are amazing maximums, but we were more impressed with the incredible consistency that we observed. With sub-microsecond variance, the Series 8 destroyed our latency tests The SSD Review. All Rights Reserved. Page 18

20 Although prices may be high for the consumer market, the pricing for enterprise customers is very competitive, especially on a per port basis. We applaud Adaptec for not resting on the success of the Series 7. It would have been easy for them to let LSI take the lead in the 12Gbps market, especially since 12Gbps SAS is in its infancy and there are only a handful SSD that can saturate 6Gbps SAS. Nobody would have blamed them for taking more time to get this product out. But, this is the exact reason why the PMC-Sierra acquisition was so important. It allowed Adaptec to lead in the market and not just follow. Without a full LSI 12Gbps RAID adapter to compare it to, we can't quite call Adaptec the winner for this round. But, with the reemergence of Adaptec, it doesn't matter, the real winner is the consumer The SSD Review. All Rights Reserved. Page 19