CAPEX Savings and Performance Advantages of BigTwin s Memory Optimization

Transcription

1 Table of Contents 2 Introduction 3 Memory Population Rules for Fully Balanced Configurations 6 CAPEX Saving from BigTwin s Design Advantage White Paper CAPEX Savings and Performance Advantages of BigTwin s Memory Optimization 7 Performance Improvements with BigTwin s Balanced Memory Design 9 Conclusions Executive Summary BigTwin is the 5th generation of Supermicro s award-winning Twin multi-node architecture design. It is the first multi-node dual-processor system in the world that features a no-compromise design to support 24 slots per node, typically only offered in 1U single-node server products. Other high-performance multi-node systems on the market are usually designed with up to 16 slots per node, which prohibit a fully balanced memory configuration at the maximum memory capacity. Performance testing results have shown 52% improvements on BigTwin in a 16 configuration over other comparable systems, which could have significant impact on real-world applications. Super Micro Computer, Inc. 980 Rock Avenue San Jose, CA USA Customers can also take advantage of the extra slots to achieve reduced CAPEX by acquiring less expensive s for the same total memory capacity. With 16 s, BigTwin offers the highest performance per dollar with its balanced memory configuration compared to other similar multi-node systems with unbalanced 16 configurations. - September 2018

2 Introduction The Supermicro BigTwin platform offers the most powerful multi-node server solution that provides the most compute, memory, storage and power within a 2U rack space today. It is the 5th generation of Supermicro s award winning Twin multi-node architecture, and the first in the industry to offer 24 slots per node in a 2U 4-node design with support for free-air cooled data centers. When compared with other high-performance multi-node systems that are designed with 16 slots per node, BigTwin s balanced memory architecture and higher maximum memory capacity allow customers to exploit significantly better memory throughput performance and achieve the highest performance/dollar benefits. This paper shows how customers can achieve optimal memory performance and CAPEX in the following ways: Specific population rules shall be followed to ensure that a balanced memory configuration with dual memory channels on the BigTwin is applied to ensure optimal performance; The balanced memory architecture and additional s slots available on the BigTwin system allows it to outperform other similar systems for the same amount of memory installed, and at a lower price point when the price of s is factored in; Performance benchmarks have shown clear evidence of throughput improvements of a balanced memory system vs an unbalanced one. Figure 1. Supermicro 2U 4-Node BigTwin System (rear system view) 1 of 4 dual-processor nodes This paper is applicable to the 5th generation BigTwin servers supporting both the current and next generation Intel Xeon Scalable processors (codenamed Cascade Lake). 2 CAPEX Savings and Performance Advantages of BigTwin s Memory Optimization

3 CAPEX Savings and Performance Advantages of BigTwin s Memory Optimization Memory Population Rules for Fully Balanced Configurations Balanced memory configurations enable optimal interleaving across all attached memory channels so memory bandwidth can be maximized. The Intel Xeon Scalable platform is designed to provide up to 6 memory channels per, and up to 2 s per channel. This is illustrated in the figure below: UPI Up to 6 memory channels per Up to 2 s per memory channel Figure 2. Memory Architecture of Intel Skylake-SP Intel provided the following general guidelines prioritized in the following order: 1. Use identical types throughout the platform: Same size, speed, and number of ranks 2. Maximize the same number of channels populated in each memory controller 3. Use a balanced platform configuration: Populate memory channels equally Identical s in all locations (size/speed/rank) 4. Use a near-balanced platform configuration: Populate all memory controllers equally Identical s in each row, but different sized s in row #1 vs. row #2 With this basic understanding, we can examine in more detail the architectural differences between BigTwin and other high-performance multi-node systems on the market today. Since the BigTwin can support full 6 memory channels with 2 slots per channel from each, a fully balanced memory configuration can be achieved as below with identical s, White Paper - September

4 UPI Figure 3. Example of BigTwin s Fully Balanced Memory Configuration When Populated with 12x 32GB s per (per node) BigTwin features 12 slots per socket that can maximize the potentials of the memory controller on the Intel Xeon Scalable platform. Other high-performance multinode systems are usually designed with up to 8 slots per socket, which prohibit a fully balanced memory configuration with maximum capacity. This design compromise is illustrated in the diagram below, UPI Figure 4. A Fully Populated 8 slots per Multi-Node System with an Unbalanced Memory Configuration (per node) 4 CAPEX Savings and Performance Advantages of BigTwin s Memory Optimization

5 CAPEX Savings and Performance Advantages of BigTwin s Memory Optimization Since an unbalanced memory system can introduce a significant amount of performance degradation (see performance benchmarks section of this white paper), Supermicro has recommended configurations for each system or motherboard included in the user manuals to help customers optimize their system performance. The figure below shows the configurations recommended for the BigTwin. Table 1. Possible Configurations on the BigTwin System (per per node) Population Recommendation for Both Sockets on BigTwin C1 C2 B1 B2 A1 A2 D2 D1 E2 E1 F2 F1 1 2 s 3 s 4 s 5 s* 6 s 7 s* 8 s 9 s* Supermicro Recommended Configurations for Optimal Bandwidth 10 s* 11 s* 12 s All rows with an * illustrate imbalanced memory configurations. White Paper - September

6 CAPEX Saving from BigTwin s Design Advantage Not only does BigTwin s unparalleled 24 slots per node design provide higher maximum memory capacity to enable a wider range of applications, customers can also take advantage of the dual channel memory slots to achieve reduced CAPEX by acquiring less expensive s for the same total memory capacity. Special Notes BigTwin has a 52% performance advantage over other systems with 16 s populated per node. 8 and 12 configurations per node are available through multiple vendors, but the total memory cost is higher because 64GB DDR MHz memory modules are more expensive than 32GB DDR MHz memory modules. For example, based upon the cost of memory today, the sweet spot in terms of market price and capacity is 32GB s. To populate 512GB of total memory capacity per node, 16 s can be used. As shown in the previous section, memory performance is optimal with dual-channel balanced configurations. With BigTwin, a balanced configuration can either be 16 or 24 s installed per node. With 16 s, BigTwin offers the highest performance per dollar with its balanced memory configuration compared to other similar multi-node systems with unbalanced 16 configurations. When we look at the cost of memory today, 8x 64GB s is more expensive than 16x 32GB s. Consequently, the cost of 12x 64GB s is more expensive than 24x 32GB s. We believe the price delta and BigTwin s balanced configurations are the two main drivers for buying behaviors of these memory configurations per compute node. Of course, the total cost delta between 1x 64GB memory and 2x 32GB configurations become more evident at high server volumes, typically seen by our hyperscale data center customers. Here s a simple equation to understand the cost delta between 64GB and 32GB s: 1x 64GB = 2x 32GB s + (~$20 to ~$30) Therefore, the cost per node breaks down as: 8x 64GB s = 16x 32GB s + 8x (~$20 to ~$30) for 512GB per node 12x 64GB s = 24x 32GB s + 12x (~$20 to ~$30) for 768GB per node Essentially, the total cost of delta between 64GB and 32GB s for a total of 512GB per node is about $240 (8x $30) at the upper limit. For 768GB per node, the cost delta per node is about $360 (12x $30). Furthermore, the total cost delta at the system level with 4 nodes becomes $960 or $1,440 respectively. For hyperscale deployments, the cost delta between memory types becomes significant. 6 CAPEX Savings and Performance Advantages of BigTwin s Memory Optimization

7 CAPEX Savings and Performance Advantages of BigTwin s Memory Optimization Table 2. Summary of Competitive Solutions BigTwin (4-node) Competitor A (4-node) Competitor B (4-node) Form Factor 2U / 4 Independent Nodes 2U / 4 Independent Nodes 2U / 4 Independent Nodes Processors Dual Intel Xeon Scalable Processors Dual Intel Xeon Scalable Processors Dual Intel Xeon Scalable Processors Memory 24 DDR slots, 3TB max per node 16 DDR slots, 512GB max per node 16 DDR slots, 1.5TB Max per node Drive Bays 6x 2.5 with configurations for 6 NVMe or 4 NVMe/SAS/SATA + 2 SAS/ SATA 6x 2.5 with configurations for 2 NVMe/SAS/SATA + 4 SAS/ SATA 6x 2.5 configurations for 2 NVMe/SAS/SATA + 4 SAS/ SATA Internal Storage 2 M.2 NVMe/SATA slots 1 SuperDOM port 1 M.2 SATA slot 1 MicroSD slot 1 M.2 SATA slot 1 MicroSD slot PCI-E Expansion 2 PCI-E x16 slots 1 SIOM (PCI-E x16) slot 1 PCI-E x16 slot 1 OCP Mezz (PCI-E x16) 1 Mezz for storage controller (PCI-E x8) 1 PCI-E x16 slot 1 PCI-E x16 or FlexibleLOM (PCI-E x16) Onboard NIC SIOM OCP Mezz Dual 10GbE mlom Note: The cost of memory for either one of competitor's nodes with 768GB (12x 64GB) is ~$ more than BigTwin s nodes with 768GB (24x 32GB). For one complete system with 768GB of memory on each of the 4 nodes, customers can save ~$ per system by selecting 32GB s instead of 64GB s. Performance Improvements with BigTwin s Balanced Memory Design The performance impact of balanced and unbalanced memory configurations were measured using the Stream Triad utility* with the following setup, Dual Intel Xeon Gold processor x Samsung 32GB DDR s Stream.icc17 * Stream Triad is the de facto industry standard benchmark for measuring memory bandwidth. Intel uses Stream-Triad to test performance at the system level and in the cloud to show the improved TCO and performance over previous generations. (Source: benchmarks/xeon-scalable-benchmark.html) White Paper - September

8 The test results are shown in the table below, Table 3. Performance Comparison (16 Slots Populated) Motherboards 24 -Slot Purely MB 16 -Slot Purely MB Stream Triad Performance (MB/s) 142, , The 16 configuration showed a 52% performance improvement with the BigTwin s balanced memory design when compared to other multi-node systems with 16 s, which have unbalanced memory configurations. Intel also reported a ~50% gain in memory performance if motherboards have a balanced memory architecture, which supports 2+2+0, similar to the BigTwin s motherboard design. UPI Figure 5. Another Example of BigTwin s Balanced Memory Configuration When Populated with 8 s per (per node) 8 CAPEX Savings and Performance Advantages of BigTwin s Memory Optimization

9 CAPEX Savings and Performance Advantages of BigTwin s Memory Optimization Conclusions Supermicro BigTwin is the first multi-node platform introduced to the market featuring 24 slots per node that can fully exploit the potentials of the memory controller on the Intel Xeon Scalable platform. This unique design advantage is a results of engineering innovations at all system levels, including the compact Power Stick design. Other high-performance multi-node systems are usually designed with up to 16 slots, which prohibit a fully balanced memory configuration with maximum capacity. Performance results have shown 52% performance improvements of BigTwin in a 16 configuration over similar systems which could have significant impact on realworld applications. The additional slots allowing for balanced memory controllers on the BigTwin can potentially decrease datacenter CAPEX and provide extra flexibility for future upgrades. Since the real-world performance per dollar depends on how much memory is required per node, your Supermicro sales representative and solutions engineers can provide reference configurations at the sweet spot for your software stack. For More Information Supermicro BigTwin Solutions Supermicro FatTwin Solutions Supermicro 1U TwinPro Supermicro 2U TwinPro and 2U TwinPro 2 Supermicro 2U Twin 2 Supermicro 2U Twin White Paper - September

10 About Super Micro Computer, Inc. Supermicro (NASDAQ: SMCI), the leading innovator in high-performance, high-efficiency server technology is a premier provider of advanced server Building Block Solutions for Data Center, Cloud Computing, Enterprise IT, Hadoop/Big Data, HPC and Embedded Systems worldwide. Supermicro is committed to protecting the environment through its We Keep IT Green initiative and provides customers with the most energy-efficient, environmentally-friendly solutions available on the market. Learn more on No part of this document covered by copyright may be reproduced in any form or by any means graphic, electronic, or mechanical, including photocopying, recording, taping, or storage in an electronic retrieval system without prior written permission of the copyright owner. Supermicro, the Supermicro logo, Building Block Solutions, We Keep IT Green, SuperServer, Twin, BigTwin, TwinPro, TwinPro², SuperDoctor are trademarks and/or registered trademarks of Super Micro Computer, Inc. Ultrabook, Celeron, Celeron Inside, Core Inside, Intel, Intel Logo, Intel Atom, Intel Atom Inside, Intel Core, Intel Inside, Intel Inside Logo, Intel vpro, Itanium, Itanium Inside, Pentium, Pentium Inside, vpro Inside, Xeon, Xeon Phi, and Xeon Inside are trademarks of Intel Corporation or its subsidiaries in the U.S. and/or other countries. All other brands names and trademarks are the property of their respective owners. Copyright 2018 Super Micro Computer, Inc. All rights reserved. Printed in USA Please Recycle 14_BigTwin-Memory_180913_FA