Tender No TECHNICAL SPECIFICATION FOR HIGH PERFORMANCE COMPUTING (HPC) SOLUTION WITH PARALLEL FILE SYSTEMS (PFS) BASED STORAGE

Transcription

1 INDIAN INSTITUTE OF TECHNOLOGY BOMBAY MATERIALS MANAGEMENT DIVISION Direct : (+91-22) (DR) / 8803 (Local) / 8804 (Import) / 8805 (Enquiry & Bill tracking), 8802(Progress) drmm@iitb.ac.in, bkganesh@iitb.ac.in Tender No TECHNICAL SPECIFICATION FOR HIGH PERFORMANCE COMPUTING (HPC) SOLUTION WITH PARALLEL FILE SYSTEMS (PFS) BASED STORAGE HIGH PERFORMANCE COMPUTING (HPC) SOLUTION WITH PARALLEL FILE SYSTEMS (PFS) BASED STORAGE (Technical specification along with terms and conditions as per the attached sheet) SCHEDULE OF REQUIREMENTS Design parameters - Linux cluster with InfiniBand Interconnect and PFS Based Storage at IIT Bombay S.N. Items Description 1. MASTER NODES - 2 Qty 1 Processor Dual Intel Xeon E v4 (30M cache, 12 Cores, 24 Threads, 2.20 GHz (105W) 9.6 GT/sec) (Total 24 Cores) 2 RAM 64 GB TruDDR4 2400MHz (Balanced configuration), Server supporting scalability to 1.5TB with 64GB LRDIMM. 3 Network interface 2 GbE ports, EDR InfiniBand port 4 DVD One number DVD RW/DVD combo (Internal or External) 5 HDD 4 x 4TB 7.2K RPM hot swappable SATA drives (Scalable up to 8 Drives) 6 RAID Support RAID 0, 1, 5 7 Systems management Advance Failure analysis support on systems for CPU, memory, HDD, Power supply and fans. LED diagnostics on server even without Power, Automatic Server Restart, management port 8 OS support TPM 1.2, RedHat Linux, Suse Linux, Windows PCIe 3.0 Slots Up to 6 PCIe Gen 3.0 slots 10 Power Supply Redundant and hot pluggable power supply 11 Cables Power cables compatible with the quoted rack 12 Fans Redundant and Hot swap 13 Form Factor 2U rack-mountable 14 Rail Kit Rack mounting sliding rail kit Page 1 of 11

2 2. COMPUTE NODES 20 Qty 1 Processor Dual Intel Xeon E v4 (30M cache, 12 Cores, 24 Threads, 2.20 GHz (105W) 9.6 GT/sec) (Total 24 Cores) 2 RAM 64 GB TruDDR4 2400MHz (Balanced configuration), Server supporting scalability to 1TB with 64GB LRDIMM. 3 Network interface 2 GbE ports, EDR InfiniBand port 4 HDD 1 x 1TB 7.2K RPM SATA drives 5 Systems management Predictive Failure analysis support on systems for CPU, memory. LED diagnostics on server even without Power, Automatic Server Restart, management port 6 OS support TPM 1.2, RedHat Linux, Suse Linux, Windows PCIe 3.0 Slots 1 PCIe Gen 3.0 slots 8 Power Supply Redundant and hot pluggable power supply 9 Cables Power cables compatible with the quoted rack 10 Fans Redundant and Hot swap 11 Rail Kit Rack mounting sliding rail kit 12 Chassis All Compute nodes to be Chassis based, chassis should support minimum of 12 or more servers with N+1 Redundancy. 3. GPU COMPUTE NODE - 1 Nos. 1 Processor- 2 Qty Dual Intel Xeon E v4 (30M cache, 12 Cores, 24 Threads, 2.20 GHz (105W) 9.6 GT/sec) (Total 24 Cores) 2 RAM 64 GB TruDDR4 2400MHz (Balanced configuration), Server supporting scalability to 1TB with 64GB LRDIMM. 3 Network interface 2 GbE ports, EDR InfiniBand port 4 HDD 1 x 1TB 7.2K RPM SATA drives 5 GPU 1 x Nvidia Tesla K40 GPU Card upgradable to 2 Systems Predictive Failure analysis support on systems for CPU, memory. LED diagnostics on 6 management server even without Power, Automatic Server Restart, management port 7 OS support TPM 1.2, RedHat Linux, Suse Linux, Windows PCIe 3.0 Slots Up to 2 PCIe Gen 3.0 slots 9 Power Supply Redundant and hot pluggable power supply 10 Cables Power cables compatible with the quoted rack 11 Fans Redundant and Hot swap 12 Rail Kit Rack mounting sliding rail kit Eligibility Criteria 4. STORAGE - 1 Set. Specifications for 100TB with 6 GBps Write throughput Following documents satisfying eligibility criteria must be supplied. In absence of any of these documents, bids will be rejected and bidders will be disqualified from the process without providing any explanation. 1. Entire storage solution including storage array and file system must be from single OEM, having 24x7 support centre in India. An undertaking must be submitted on OEM letterhead stating solution functionality & installation is Page 2 of 11

3 Storage Capacity responsibility of the OEM and support will be provided by OEM directly during period of installation, commissioning and warranty. 2. Documentary proof in the form of letter of acceptance, signed by the existing customers must be submitted for following references. If bidder is not OEM, then bidder must supply proof of at least one successful installation with same OEM and with same PFS for minimum 300TB storage in past 3 years. Storage OEM must have at least 1 reference in India for minimum 500TB PFS storage. Both data and metadata capacities must be quoted as shown below: Connectivity EDR InfiniBand or OPA. Storage Performance Scalability Availability PFS Data: 100TB usable capacity for PFS data in RAID6 (8+2) layout using 2TB or higher capacity 7.2K RPM NL-SAS disks. Minimum 1 hot spare disk per tray must be included. PFS Metadata: 1% of the OST quoted separately if required. Storage solution must be designed for aggregate performance of 6 GB/s for sequential read as well as sequential write operations, when ran separately on 8, 16, 32 clients. Bidders to submit output of IOR/IOZone benchmarks to show this capability. Same performance should be demonstrated at the time of acceptance. Similarly, storage solution must be designed for parallel filesystem metadata performance of minimum 60,000 files creates/sec and 400,000 stat/sec. Bidders to submit output of MDTEST ( benchmark to show this capability. PFS must be scalable up to 2.5 PB usable capacity just by adding disks and enclosures in same configuration (without adding controllers, I/O servers or network ports) without affecting aggregate performance of the system. Performance should be scalable to 11GBps without adding any controllers, disk enclosures and Licenses. There must not be any single point of failure in entire storage solution. Storage solution must have minimum two active-active controllers, battery backed & mirrored cache, redundant power supplies and fans and ability to tolerate failures of up to 2 disks in a RAID6 volume. Footprint Entire storage solution must not consume more than 10U of rack space. Bidder must provide peak power in KW, heat in BTU/Hr, weight in Kg and airflow in CFM for cooling for the storage solution. PFS Features Installation, Support and Delivery OEM Supported (Intel Lustre or IBM GPFS) User & Group Quota Ability to read and write in parallel to same file or different files Data striping across multiple headers and RAID6 LUNS Ability to transparently recover from client, server and network failures without losing data. Snapshots, remote replication, automated storage tiering, RDMA over Infiniband support. All PFS features of the file system mentioned in the OEM datasheet must be included. Stripped down versions are not accepted. Storage installation and configuration must be carried out by OEM engineers only. Page 3 of 11

4 3 years 24x7 support/warranty must be included with Next Business Day (NBD) parts delivery. 5.1 Ethernet Switch 5. MANAGEMENT COMMUNICATION NETWORK - 2 Nos. L2 managed 48 ports Gigabit Ethernet switch with sufficient no of ports to connect supplied servers for management network with required no. of cables and rack mounting kit. 6. INFINIBAND SWITCH - 1 No. 6.1 EDR IB Switch 6.2 Cables 100Gb/s 36 Port EDR InfiniBand Switch Systems for faster and primary interconnect. Other features should be as below: 36 EDR (100Gb/s) ports in a 1U switch 7.2Tb/s switching capacity, IBTA Specification 1.3 and compliant, Low latency and low power design, InfiniBand router, Quality of Service enforcement, Port Mirroring, Adaptive routing, Congestion control, Redundant power supplies, Replaceable fan drawers etc. The solution should be quoted with required no. of cables with suitable length to connect Master nodes, Compute Nodes, GPU Nodes, Storage and others as required in solution. 7. KVM CONSOLE - 1 No. 7.1 Console 1U 17'' rack mountable Console with KVM. 7.2 Cables Required no. of cables with suitable length. 8. RACK - 1 No. 8.1 Size and Dimension 42U Server Rack with 800x1200 dimension. 8.2 PDU Single Phase 9. SOFTWARE - 1 Set. Commercial Softwares with Support from it s OEM s Proposed cluster management tool to be fully supported by it s OEM. 9.1 Cluster Management Software: Proposed cluster management tool should have web based graphical remote access interface. Should be compatible to work on RHEL 5.x / RHEL 6.x / CentOS 5.x/ CentOS 6.x/ Fedora Should be able to provision above operating systems to compute nodes. Should be able to auto-provision applications to compute nodes Should allow HPC management console to be accessible from any system in the network. Should support role based access to the HPC system. Every role should be able to be remotely managed using the Graphic User Interface. Should provide profile-based and fully automated provisioning features. Page 4 of 11

5 9.2 Job scheduler: Libraries, MPI and Compilers Jobs Submission Portal: Operating System Proposed cluster management tool should have a graphic user interface. Should support Add/Modify/Delete compute nodes from GUI window. GUI (Web) based monitoring feature to be part of proposed solution. Should support major power management modules like bullpap, wti, apc_snmp, ether_wake, ipmilan, drac, ipmitool, ilo, rsa, lpar, bladecente as per the hardware proposed. Bidder should propose a job scheduler which is fully supported by them. Job Scheduler proposed should only be OGE / Torque / Open PBS or Licensed and supported version of PBS Pro. No other scheduler to be proposed. Bidder should propose GNU Compilers & Intel Cluster Studio single user floating license for one year. Bidder should implement Open MP and MPI network. Bidder should set up the Cluster for MPI Communication over IB/GigE Vendor specific InfiniBand stack on Linux OS, if available, should be supplied. Vendor specific MPI implementation on Linux OS should be supplied. Open source software/platforms like Python, Perl, Abaqus, Matlab, Comsol, openfoam should be integrated with the cluster. Integration of NAMD will be a major plus. IIT Bombay also has licenses from Abaqus, Matlab, Comsol, ANSYS, Erdas etc. These also to be integrated with the cluster. Bidder should propose for ISV supported GUI Based Job submission portal. Proposed GUI Based job submission portal should be in production mode with at least 3 organizations in India of which 2 have to be Govt/research organization. At least 1 location from the same to be in production mode for last 3 years. Proposed portal should be integrated with existing LDAP or NIS authentication. Integration of NAMD into the cluster will be a major plus. Proposed job submission Portal should be fully integrated with proposed Cluster management tool and Scheduler. RHEL latest version in HA to be proposed for master nodes with 3 years of support. For compute nodes as well, RHEL should be proposed. Scheduler Scheduler proposed should be compatible with hardware and software infrastructure proposed and too be supported by bidder. 10. LTO TAPE DRIVE - 2 Nos. LTO-7 drive should be able to store up to 15TB of data on a single cartridge and provides performance of up to 2.7TB per hour. The LTO-7 tape drive should be backward writecompatible through one generation and read-compatible two generations LTO-7 Drive Capacity: 6TB / 15TB (native/compressed*) Performance up to 2.7 TB/Hr (compressed*) Enterprise features and performance in a small form factor Should have future proof with 10-generation roadmap Reliability MTBF of 250,000 hrs at 100% duty cycle LTFS, WORM and AES 256-bit encryption 30 year archival life 3 years Advanced Replacement Service 10.2 LTO-7 Cartridge 10 Nos. of LTO-7 Tape/Cartridges should be quoted along with each Drives. Page 5 of 11

6 11. BENCHMARK REQUIREMENT Benchmark Rules for Bidder: The benchmark data/input files will be made available to bidders. Each bidder must provide the purchaser with the timings and results of the program using the proposed system. If same system is not used for benchmark, bidder MUST submit the actual result along with the estimated completion time on the proposed system. However, it is mandatory that the benchmark system used should be of the same processor family (i.e E v4). 1. Abaqus / Standard Input file The bidders should provide the benchmark result to demonstrate the scalability of the proposed architecture with reference to the provided input file starting from 2 nodes (48 cores) to 8 nodes (192 cores). The benchmark should run by the OEM only. Benchmark submitted by any third party will not be accepted. Bidders must submit benchmark results with the runtime configuration & software environment used. Bidders are also required to provide all the original output & log files for verification. If required, bidder must give remote access to the purchaser for verification. During evaluation, it is necessary that the benchmark program run correctly and producing results within the estimated time. A tolerance limit of 5% deviation on the estimated time shall be granted. Non-compliance to any of these conditions may result in disqualification. 2. WRF Input file Prerequisites: 1. It is mandate to use WRF version for the benchmark 2. 18KM/6KM resolution (2 nested domain) 24 hours prediction 28 verticals 3. Physics options provided in the namelist should NOT be modified. Any such modification will lead to disqualification. 4. Model output must be saved at every 15 minutes interval (ie. history_interval=15,15) 5. WRF Benchmark results to be submitted on 200 and 400 cores respectively WFR Input file would be provided by Civil Engg. Dept., interested bidders are requested to collect the data from the same and required to submit the output file on to an optical media or a thumb drive. Link for downloading the codes for benchmarking is given below : Page 6 of 11

7 SCOPE OF WORK Rack Mounting & Cabling and Integration of the Storage system to the supplied Factory Integrated HPC Cluster followed by the civil work if any Head Node Installation and Configuration Compute Nodes Installation and Configuration Storage Installation, Configuration and integration with HPC solution Application Installation which would be provided by the user Benchmarking: Application benchmarking followed by Storage benchmarking Acceptance Test and fine tuning Training on cluster usage to all the users Years of onsite comprehensive warranty and support. May go up to 5 years if AMC opted. 13. EVALUATION CRITERIA The Evaluation of bid would be based on two criteria s: Technical as well a Commercial. Both criteria s would be governed by it s respective scoring. Bid Evaluation 1. The Evaluation process to identify the successful bidder is based on the combined techno-commercial evaluation. The bids received from the bidders will be evaluated by the Technical Committee constituted by the Institute. The decision of the technical committee is final and binding on all bidders. 2. The technical bids are evaluated first. The mandatory conditions mentioned elsewhere must be adhered to and failure of the same will result in disqualification of the bid. 3. Each bidder has to obtain the minimum score under each category of the Technical Evaluation to qualify for opening of the commercial tender. S.N. Description Max Score Min Score Technical Bid Evaluation 1 Solution Superiority Operation Cost Benchmark Performance Bidders strength/experience in HPC in India (In terms of market shares/installation base across industries) + Clarity and quality of RFP response document Commercial Bid Evaluation 1 Price Evaluation 120 TOTAL MARKS Negative marks may also be awarded. No minimum marks to qualify. Page 7 of 11

8 Technical Bid Evaluation: Solution Superiority: Solutions meeting the technical requirement of the tender will be given 15 marks. Additional marks are given if the proposed solution has advantages of superior interconnect, higher reliability of product such as redundant power supplies, hot swappable fans, hot swappable power supplies, hot swappable compute blades etc. Kindly state clearly the superiority of the solution in your bid separately. Operational Cost: Lower operating cost will attract higher marks. This is evaluated using the sum of rated IT load of the proposed HPC solution by the bidder. The bidder with the lowest requirement of IT load will receive full 55 marks. m = 55-(x-xmin)*0.8 where, x = kw of power consumed by given bidder xmin = the lowest quoted value in kw among all bidders m = marks obtained for the operation cost by the specific bidder Benchmark performance: Requested benchmark to be conducted on current generation CPU platform which is Broadwell. Bidder MUST submit the actual result along with the estimated completion time on the proposed system. Total time required in seconds is computed as T = Σ np * t where, np = no. of cores over which the run was made t = CPU time for that run The summation is over all the runs that are successful. A run is said to be successful if the output reproduces the results. If a run is successful, then it automatically gets certain number of marks. Additional marks are allotted depending on the time it takes to run. Shorter the time taken to run, larger the score (m). m = [135 + (j 1) * 280]/2 where j = Time taken for a given vendor. The time taken is computed with equal weightage for each of the two benchmark datasets/programs. All the submitted benchmark timings will be normalized with respect to number of cores and clock rate of the processor. Bidders strength/experience in HPC in India (In terms of market shares/installation base/satisfied customers across industries) along with Clarity and quality of the RFP response document: Large, difficult-to-read documents where the important information is hidden deep within will attract less marks. Clear, concise documents with all details of important information will obtain more marks. Information should not be scattered but should all be at one place for a particular item. Bidders should have maximum nos. of installation in India to show it s strength. The bidder who has largest market share in HPC market will attract more marks. Commercial Bid Evaluation: Price Evaluation: Marks will be calculated based on the total cost of ownership (TCO). This is the sum of the cost of the proposed solution which includes HPC cluster, Storage, Interconnect, Software etc., In addition to these, cost of Service Level Agreement (SLA) and Operation Cost for contract period added. The marks for this is calculated as: m=120 - [TCO(bidder) TCO(lowest)]/2 where, TCO is in lakhs and TCO (bidder) is TCO of the bidder and TCO (lowest) is the lowest TCO amongst all the bidders. Page 8 of 11

9 TERMS & CONDITIONS Bidder to provide complete details about Schematic Diagram, Rack layout, Power & Cooling requirement, 1 Electrical infrastructure requirement and anything else required at IIT Bombay. All products quoted should be with 3 years of warranty and support. Any new version available of the 2 software proposed in the respective domain should be provided and implemented at no extra cost for entire contract period. The bidder shall provide the Registration number of the firm along with the GST/LST/CST/WCT/ST No. and 3 the PAN Number allotted by the concerned authorities. The bidder must have experience of supplying 5 similar orders of HPC within the last 3 financial years for any Govt. organization preferably to Educational and R&D organizations. The documentary evidence in this regard must be attached. Contact details of the concerned person of institutions where installation have 4 been made by the bidder should also be enclosed. Bidder should have at least 5 HPC cluster Installations India in last 3 years, 15 Teraflop each and. PO and Installation report to be in Bidders Name. This applies even if bidder is one of Server OEM. Bidder must have provided at least one HPC cluster with 50TF in last 3 years. The bidder must not be blacklisted any other university/institution/government department. An 5 undertaking to this effect must be submitted. 6 Compliance to eligibility criteria must be attached along with unpriced technical bid. This is a two-part tender. The bid should be submitted in two parts Technical and Commercial. Technical Bid 7 should not contain any indication about the prices. If this is not followed, the bid would be outrightly rejected. OEM should have at least 4 systems and Bidder should have at least 2 systems in top supercomputers list of 8 IISc Bangalore latest publication (July 2017). 9 Bidder should be an ISO certified System Integrator. 10 One OEM should quote with one bidder only & vice versa. 11 OEM should have executed at least one HPC order above 50TF or minimum 2 orders above 25TF each in last 3 years in India. The details of the same to be provided. 12 The quoted prices can be in INR or in any valid foreign currencies (e.g. US Dollar). For INR quotations, delivery should be up to IIT Bombay. No pre-bid meeting will be held for this requirement. For any technical queries, undersigned authority can 13 be contacted through only and subsequently, corrigendum can be raised on satisfactory reasoning, if required. 14 Vendor must not quote bundled price. Price should be quoted item-wise. IITB may procure more or less quantity. 15 The bidder should have a turnover of at least Rs 100 Crore each year during last 3 years. Hardware OEM should have direct local sales & support office in Mumbai. The OEM should have direct 16 presence in India. i.e. OEM office & spare depots should be present in India. A letter confirming same from OEM is to be submitted. 17 OEM must provide authorization certificate to one Bidder only. 18 OEM should have good reputation and service experience at all IITs including IITB. 19 OEM must have 20 installations/entries in current list June 2017 of TOP500.org. 20 The bidder must submit the printed catalogues, literature of the proposed product in solution The firms are required to quote Servers of the branded/mnc make only. OEM can quote either directly or through an authorized partner or service provider. In case of partner, MAF is compulsory Point by point compliance to all the above mentioned features should be provided by the firm. There should not be any deviation and if any, should be stated clearly. Full Server configuration as listed above should be tested and integrated at OEM manufacturing plant which includes all major components, power supply, cooling fan cabinet etc. No local site integration of server components will be allowed. Except Storage, entire HPC solution to be supplied as Factory Integrated Page 9 of 11

10 The firm must be authorized by the manufacturer to supply, install and maintain the system. The specific authorization by the manufacturer for participating in this tender should be enclosed, otherwise quotation may be rejected. The bidder must agree to install and configure the user sought operating systems, mostly open source Linux versions, MPI libraries, Job schedulers plus cluster management tools and demonstrate its running in parallel as part of the system acceptance. Installation and maintenance charges for 3 years, should be mentioned separately along with training on cluster usage to it s users. Zero or unrealistic cost quoted here will be liable to rejection. Vendor must provide the Bill of Material with manufacturer s part numbers for the items proposed to meet the complete solution. Vendor should verify and certify that the items proposed are sufficient to integrate the proposed solution into a production mode. 27 Special instructions for site preparation if any should be clearly mentioned in the technical bid. The Supply, installation and commissioning of the system should be completed within 12 weeks from the 28 date of purchase order. The project / activity must be completed on turnkey basis, integrating hardware, system software and application software by a single vendor / system integrator The total cost for the purposes of comparison, will be determined by considering the total cost of the entire solution i.e HPC components, all software, storage and including three years warranty. 4th and 5th years AMC quote is optional but mandatory to quote which may or may not be considered for the price evaluation of lowest bidder. Warranty: Vendor should provide minimum Three years comprehensive On-site support warranty for all the deliverables, Hardware and software at no additional charges. Post Warranty AMC for two years should be quoted for the entire solution. Bidder should submit undertaking of warranty from OEM that warranty is onsite for 3 years from OEM along with back-end support to bidder for 2 years of additional AMC. No EOL/EOS product should be quoted and the same should be stated on OEM letter head that EOL/EOS would not be there for quoted products for at least 3 years. Bidder/OEM should provide Training on cluster usage to it s users and it s cost should be quoted separately. If bidder is not OEM, then bidder must supply proof of at least 1 successful installation with same OEM for minimum 300TB storage in past 3 years. Storage OEM must have at least 1 reference in India for minimum 1PB PFS storage 34 Price should be quoted on the basis of 24x7 response against call log by IITB to the Bidder/OEM. 35 Bidders must mention GST and HSN/SAC codes of all the items quoted. Page 10 of 11

11 Financial Details (to be submitted by the bidder) Annual Turnover for Last 3 Years (in Rs. Crore) Annual Turnover from Similar Works for Last 3 Years (in Rs. Crore) Enclosures: Audited Annual Financial Reports for , , and certified by Chartered Accountant Date of submission Signature of Applicant (With Seal) Page 11 of 11