Updating the HPC Bill Punch, Director HPCC Nov 17, 2017

Transcription

1 Updating the HPC 2018 Bill Punch, Director HPCC Nov 17, 2017

2

3 Unique Opportunity The plan for HPC and the new data center is to stand up a new system in the DC, while maintaining the old system for awhile before moving. The opportunity is to create a very new HPC setup in the DC, test it under load, while keeping our traditional system running

4 New everything For 2018, we plan on a lot of new elements given this opportunity: new compute cluster new file system upgraded OS on compute nodes potentially new network new software We are considering all possibilities!

5 Purchase Power We presently have around $2M to spend, probably a bit more. We are soliciting input from potential buyin users to augment that initial purchase We need your feedback given the "clean slate" approach we are taking.

6 Buyin indications We would like to get an indication (not a full commitment) of who and how much is available. Not only does this help with quantity of purchase, but your input, your money helps us steer what we are buying

7 Timeline indication of interest and money before the end of the year RFP to go out before the end of the year Responses back in January cull those down to a smaller set, do some benchmarking Contract in March Up in April Money for purchase due after sign-off

8 You remember buyin right? Buyin gets you/your-group access to nodes you purchases on a priority basis: a node is marked as "yours" no one can run on that node for more than 4 hours on "your" node Thus within 4 hours (max) you get access to your node standard runtime is 1 week, but we can change that.

9 You/your-group Remember, HPC doesn't discriminate among your group. For example you have people A,B,C,D in your group and 4 nodes. person A schedules all 4 nodes for a week B,C,D are now locked out from priority access for the week You have to manage your own group!!!!

10 Compute Cluster The landscape is more wide-open than in recent years and we are considering all possibilities Thus we are considering Intel, AMD and Power9 as options for the new cluster. They each have the advantages and disadvantages. Intel in particular has made it difficult to gauge with a large number of variations

11 Higher Cost! What is clear from this SC17 show is that the cost of a node is definitely going up. This is for two reasons: CPU's are more expensive (but also more capable) Memory prices are way up and a much more significant portion of the cost Be Ready!

12 The basic picture The basic picture, across architectures, is around 20 cores on a socket, two sockets per node perhaps more cores, perhaps less with less cores going faster memory bandwidth is typically faster than before.

13 Three classes Broadly, there are three classes of compute node with some subclasses: - basic compute - GPU - Huge memory compute

14 Basic compute Within that, we see roughly 3 classes: - basic node with say 192GB memory - $10K (roughly, say ± $2K) - Middle class of memory, say 756GB, ~$20K - A big one, say 1.5Tb, ~$50K We could go less, say 96GB and save 10% or so if there is reason to do so

15 GPU Well, honestly there is good news and bad news here. We see two classes of GPU nodes v100 volta. Awesome machine, expensive at $10K each Nvidia DGX1 is $150K, 8 voltas "consumer cards", say 1080Ti maybe $1000 So what's the bad news???

16 Nvidia throttling Nvidia is aggressively throttling access to "consumer" cards for server uses. Aggressively! no distributor will now sell a server with 1080Ti cards for example. access to cards is limited and comes in dribs and drabs We will buy a chassis(s) with empty slots and fill them with cards as we can get them but

17 Huge Memory There has been need for huge memory machines in the past, 3-6TB that will not likely be affordable in the present market if you need that, we can buy Broadwell servers with older memory, but time will run out on those so we need to know.

18 The mix This is our big problem, the mix: if basic compute, how many and with what memory if GPU, how many consumer vs v100, and are you willing to wait for consumer cards as they become available, which is out of our control Anybody need huge memory?

19 We need feedback HPC is a service organization but we cannot serve effectively without feedback. We are buying, but the mix can be determined by your feedback otherwise, we guess and you take what we pick.

20 Other stuff looking at a completely new FS, both for scratch and for home. More standard, industry supported stuff we are looking at new tech. FPGAs were very hot, Intel has a good plan and a product available right now (happy to talk about that) we are looking at new scheduling software as well (why not)