Purchasing Services AOC East Fowler Avenue Tampa, Florida (813) Web Address:

Transcription

1 Purchasing Services AOC East Fowler Avenue Tampa, Florida (813) Web Address: November 10, 2015 Invitation to Negotiate No. Entitled: Opening Date: MH High Performance Computing Equipment November 20, 2015 at 3:00 p.m. Addendum No. 1 Review the following changes/additions/clarifications to Invitation to Negotiate (ITN) No MH High Performance Computing Equipment to be addressed in submitted proposals: Below are the vendor questions we received for ITn MH, the USF response are in blue. 1. Question: would the University consider a Cloud (actually Managed) HPC solution whereby CTL purchases the gear then rents it to the University, as well as managing the cluster of servers? Basically a hosted HPC solution offering. Or, is the ITN specifically requiring a capital purchase of equipment to be installed on campus and managed by USF? Only capital purchase installed on campus will be considered. 2. Regarding the 100 nodes, given similar price do you prefer cores or clock? As outlined in the ITN the metric that will be used is Price performance ratio (pg 16 and 18) specifically: Section III. I. This Invitation to Negotiate provides a model through which vendors can provide innovative solutions based on price/performance ratio. Vendors will be asked to provide benchmarks for the systems that they propose as outlined below.

2 Section VIII Tab C Price/Performance Ratio Provide results for a similarly configured set of nodes to number not less than 64. For the HPC challenge benchmark suite ( These results will be used to determine a price performance ratio for comparison. All rules to the HPC challenge must be followed. Results for the Baseline Run must be provided. Vendors are strongly encouraged to provide results for the Optimized Run. Provide a detailed listing of all libraries and software tools (including pre-processors, compilers, static and dynamic linkers, operating systems) used. 3. Will submissions with refurbished product be accepted? No. 4. Do you require professional services (e.g. installation)? Rack and stack will be considered based on price, but no other services will be needed. 5. Do you have a brand preference? One of the criteria is the viability of the vendor, for which the stability of the manufacture will be considered. 6. Will partial submissions be accepted? No. 7. Do vendors need to be authorized by the manufacturer? Yes. 8. How many nodes require a video card? Nodes will only require CLI access so minimal video needed. 9. What would the preferred video card be (Quadro, FirePro and Ect.)? Nodes will only require CLI access so minimal video needed. 10. How many Nodes in total? Our initial estimates are for 100 nodes, but the final amount will be based on the final price. 11. On Infiniband Infrastructure and Gb Ethernet, what models are preferred (name of models preferred and amount)? Research Computing currently employs Qlogic port QDR switches for edge and core switching for Infiniband traffic. For Ethernet traffic, Brocade 7250 ICX swithes are required for integration in to campus network. 12. Can we get a more detailed info about the current hardware environment? The current environment is a condominium heterogeneous cluster. The infiniband network uses a 2:1 blocking factor. Storage will consist primarily of a GPFS system. Please see the previous question for details of the network infrastructure. 13. What is the average amount of Hardware wanted (HD size, Ram and # Processors)? Minimum HD size 500GB primarily for OS. The RAM standard is 2GB per core. In general we have had the best results with 1U dual processor systems, but other form factors may be considered base on price/performance and ease of integration with current configuration.

3 14. Is any storage Necessary If so what type/size of HD are preferred (amount of storage, mechanical or solid state)? Single local disk (minimum 500GB), no need for SSD or other high end drives. 15. The first line of the Attachment 1 LOT 1 says GPU Enabled Nodes. Is this supposed to read Compute Nodes Yes that line should read Compute Nodes 16. In Section II USF Environment b. Hardware it states that current nodes are connected with QDR infiniband. It then states that the cluster can be portioned by the queuing software so that the different infiniband technologies are not used for single computations. However on page 17 under c. Infiniband infrastructure it states We will also acquire leaf switches (QDR) and cabling to support the purchased nodes.. Please clarify your IB fabric requirements for the new nodes to be purchased. Will they be connected via QDR o do you prefer to have them connected with the latest Infiniband HCAs and switches and use queuing software to ensure that different IB technologies are not used for single computations? The nodes will be connected via QDR with a 2:1 blocking factor. There are currently 3 core (36 port) switches with 19 free ports each for edge switch connections. Proposals should include edge IB switches as necessary to expand this fabric. Additional core switches, if necessary will be provided outside of this ITN. 17. In Lot 1 Equipment list/scope of work b3. States All proposed CPU s must have a release date less than 9 months old. Intel s Haswell E52600 V3 processors are the latest shipping processor but they have a release date more than 9 months old. Are you asking vendors to propose and price products which have not yet been announced by Intel? This is our standard for processor description. In the case that there hasn't been a new processor in the last 9 months, the latest previous processor will be considered. 18. Section VII Proposal Format Tab C states: Provide results for a similarly configured set of nodes to number not less than 64. For the HPC challenge benchmark suite ( If vendors are required to provide pricing on unannounced products from Intel to meet the requirement of all processors must have a release date of less than 9 months old, will the University allow vendors to run the benchmark on systems with a different processor and provide extrapolated results for the processors being bid? Please see our response to question Section VII Proposal Format Tab C states: Provide results for a similarly configured set of nodes to number not less than 64. Will the University accept benchmark results run on 64 nodes that do not have the processor being bid as long as the vendor provides extrapolated performance and the methodology of how the results were calculated? Yes. 20. In section III. B.4 on page 17, it states price/performance will be determined by the overall core count within the purchase. Please clarify how the University will calculate the price/performance ratio. Typically, Price/performance is a calculation of $/Gflop. For example with a given budget, you could have fewer nodes using a particular bin of processor and have few overall cores but a better price/performance solution ($/Gflops) than using more nodes with more overall cores within the given budget.

4 We will primarily consider the 64 node test as the benchmark for price/performance ratio, and will extrapolate to the number of cores in each proposed solution. 21. III: E states Standard racks and PDU s are already in place for housing the equipment to be purchased as part of this ITN. However, in Attachment 1 on page 19 there is a place to provide pricing on Racks and PDUs. Are vendors supposed to provide pricing for racks and PDU s? We currently have sufficient racks for 100 nodes provided by central IT, so rack and PDU pricing is not necessary. 22. Will USF provide the current network topology for the 2 level fat tree including the blocking factor and number of available ports on the core switches? Please see our response to question What are the top 10 applications (measured by compute resources consumed) currently running in the HPC cluster environment? The cluster is a central resource and runs a wide variety of applications. It is too dynamic to consider the applications as a primary decision point for hardware. This is the motivation in requesting HPC challenge benchmark as specified in the ITN. a. Will these remain the top applications in the new HPC environment This is addressed in the answer to the main question. 24. What application performance metrics are you considering in your evaluation criteria? Please see our response to question Do you require Operating Systems (RHEL) Licenses and Software Support as part of the response? No. 26. Do you require other cluster software, scheduler, etc. licenses and support as part of the response? No. 27. What is the minimum amount of memory required per node? The standard for the cluster is 2GB per core. 28. What are your current Infiniband switch make and models? Please see our response to question 16. a. Please provide a topology diagram of your current environment Please see our response to question 16. We do not have diagram due to recent changes. 29. Confirm the model of the Brocade TOR switches is the Brocade ICX port? We confirm that the ICX port is in use with a POD enabling 10 GB uplinks on 1/2/1 and 1/2/3 installed at the factory, prior to shipment. a. Do you have and/or use Brocade Network Advisor? No. b. Should it be included in the proposal?.no 30. Is a copy of the tabulation sheet available? There is no bid tab at this point we have not received any proposals. We do not have a sheet other than what is in the ITN document.

5 31. Is there a requirement for internal disk storage in the compute nodes? 500GB primarily for the OS a. If so, what capacity is required? Addressed in answer to main question 32. Do you require an external disk solution as part of the response? No. a. If so, what is the usable capacity needed? 33. In Section III, Topic III, paragraph e, it states you have standard racks and PDUs in place for this equipment, but Attachment 1 has rows for rack and PDU pricing. Do you require racks and PDUs quoted as part of the response? We currently have sufficient racks for 100 nodes provided by central IT, so rack and PDU pricing is not necessary. 34. Page 7, lot 2; states valid price thru 6/30/15 please verify if that date is correct? The date should be 6/30/ Page 17, lot 2 IV; discount listed please verify valid date thru 6/30/15, please verify if that date is correct? The date should be 6/30/ Can you provide additional details including hardware used in the current HPC Cluster that Lot 1 will be adding too? Please see the answer to questions 12 and Please provide additional network details of the current HPC Cluster (manufacture/model # and current devices/topology in use as well as open ports to tie into for the proposed hardware). Please see the answer to questions 12 and How is the Fat Tree InfiniBand network currently configured? Is there any blocking factor? Any open ports to tie into for proposed hardware? Please see the answer to questions 12 and Node Hardware: a. Is a redundant power supply required No b. Is there a minimum GB per Core requirement? 2GB c. Is there a preferred processor model/speed/core count? No specific preferences other than price/performance metrics detailed in the ITN d. Is there any required local storage? 500GB (minimum) for OS e. If local storage is required preferred (SATA/SAS/SSD), size, qty, any RAID? SATA f. Is there a specific form factor preferred? Our best results have been with 1U servers g. Would you consider dense solutions that offer more nodes in a single 1U, 2U, or 4U Chassis? Yes

6 40. Is there a requirement for a Head/Master Node or any Admin/Login nodes? If so any specific requirements/qty? No 41. Is USF open to seeing multiple options/upgrades for the proposed hardware? Yes 42. Is it preferred that the vendor install the hardware? If so is there any additional services preferred or required (such as training, onsite warranty support)? Rack and Stack may be considered based on price. Onsite warranty may be considered based on price, but is not necessary. No other services are required. 43. Is the hardware to be delivered bare metal or configured with an OS and software stack? Bare metal 44. If an OS or software stack is preferred what is the preferred configuration? (RHEL, Centos ) N/A 45. If an OS with a license is preferred will USF or the vendor to supply? If the vendor is to supply what is the support level required? N/A 46. For the Lot 2 GPU Nodes do they need to be the same hardware proposed for the general compute nodes in Lot 1? No, but in general as much homogeneity as possible is beneficial. 47. What type of GPU cards and qty. is preferred for the GPU Nodes? Because this is a future possibility no specifics are needed at this time. 48. Will the vendor need to show pricing for the GPU Cards individually or integrated? Neither. 49. What Qty of GPU Nodes is USF looking for? Lot 2 is a request for applying the discount to future purchases, as such there are no current requirements. 50. Will the GPU Nodes be connected via InfiniBand and if so will it tie into the network for Lot 1 or separate? Lot 2 is a request for applying the discount to future purchases, as such there are no current requirements. 51. What is the preferred warranty and type for all the hardware? Three (3) Years. 52. Does the vendor need to include the GbE Mgmt and InfiniBand Cable(s)? If yes what is the preferred lengths (data center diagram)? And type (fiber/cooper) W e utilize 10M active QDR QSFP+ cables for edge to core connections, and passive 3-5M cables for edge node connections. GbE cables should be included for nodes to rack top switches. 53. What is the PDU specs of the power for the proposed systems are going into? PDU's and Racks are not necessary. 54. What is the Rack model and quantity provided by USF? PDU's and Racks are not necessary.

7 55. What is the preferred specs for the Large Mem Nodes? Lot 2 is a request for applying the discount to future purchases, as such there are no current requirements. 56. Are you looking to use the same hardware for the Lot 1 compute nodes for the Large Mem Nodes? If possible, but Lot 2 is a request for applying the discount to future purchases, as such there are no current requirements. 57. Will the Lot 1 compute nodes need to be upgradable? (Memory, drives ) No. 58. Is an onsite spare parts kit required? If so what is the preferred hardware/qty? No. 59. What is the power limitation per rack? We will move nodes as necessary to balance power 60. Section VIII; Tab C. Do you mean 64 cores or 64 nodes for benchmarking? Nodes. 61. Will the benchmarking be done on Lot 1 or Lot 2 hardware? Lot 1 only. 62. Is this solely an addition to an existing cluster or will this mean building a separate cluster? Addition to an existing cluster only. 63. Are there certain performance levels expected? If so what are they? No. 64. What version of OS and kernel will be used? For the purposes of the ITN please use current RHEL installation. 65. What is the layout and hardware in the existing Infiniband network environment? Please see our response to question 16.

8 Note: Please note receipt of this addendum by signing and returning with your proposal response Authorized Signature & Date Print Name Company Name