http://openhpc.community Meeting of the Technical Steering Committee (TSC) Board Tuesday, October 17th 11:00am ET
Meeting Logistics https://www.uberconference.com/jeff_ef United States : +1 (510) 224-9559 (No PIN needed).
Antitrust Policy Notice Linux Foundation meetings involve participation by industry competitors, and it is the intention of the Linux Foundation to conduct all of its activities in accordance with applicable antitrust and competition laws. It is therefore extremely important that attendees adhere to meeting agendas, and be aware of, and not participate in, any activities that are prohibited under applicable US state, federal or foreign antitrust and competition laws. Examples of types of actions that are prohibited at Linux Foundation meetings and in connection with Linux Foundation activities are described in the Linux Foundation Antitrust Policy available at http://www.linuxfoundation.org/antitrust-policy. If you have questions about these matters, please contact your company counsel, or if you are a member of the Linux Foundation, feel free to contact Andrew Updegrove of the firm of Gesmer Updegrove LLP, which provides legal counsel to the Linux Foundation. 3
Agenda SC 17 BoF Update - Schedule update: Wednesday, Nov 15: 12:15-13:15 - TODO: JEFF -> send out google doc sign-up sheet for booth duty ISC submission(s) Singularity image submission LLVM question/request OpenHPC on Bright cluster PMIx update Warewulf aarch64 update 4
ISC Submission(s) I m assuming we will want to do the usual BoF submission: - deadline is Feb 28th Do folks want to consider trying for a tutorial this year? Submission Deadline Tuesday, February 13, 2018 Notification of Acceptance Wednesday, March 21, 2018 Tutorials Sunday, June 24, 2018 Final Presentation Slides in PDF due Friday, June 29, 2018 The tutorials will be held on Sunday, June 24, 2018 and will be either half-day (9:00 am 1:00 pm or 2:00 pm 6:00 pm) or full-day (9:00 am 6:00 pm). Could presumably do something similar to what we did for PEARC 17 5
Singularity image submission? Received a note from Greg (Singularity lead) suggesting we might want to consider submitting an OpenHPC image to SingularityHub - https://singularity-hub.org Question: is this of general interest? - If so, what end-user/system type would the image be targeted for? - Presumably, reproducing the dev environment dictates the likely items to package in a shareable image should be easy to work in single node environment what about non-openhpc clusters? sensitivity to MPI stacks and resource manager on the cluster would we try to test on a non-openhpc system, if so where? could possibly publish table on wiki for sites which have been able to run MPI executables - could alternatively propose to provide container definition template to ship with Singularity instead of binary image 6
LLVM question/request Had an issue/request come in last week regarding packaging of LLVM/Clang - request is to split out llvm library for use with other compilers chuckatkins commented 4 days ago Contributor It's possible this has been already discussed and I may have missed it, so my apologies if that's the case. I am very happy to see the llvm-based!*lang! compilers as an option now. However, as more codes in the HPC space are beginning to utilize llvm as a library, it would be helpful to separate the compilers as libraries to allow for libllvm to be made available with other compilers. Packaging wise, I think this could look something like this:!llvm5*compilers*ohpc! The LLVM compilers and std libs (clang, clang++, flag, llvm-ar, llvm-ld, etc.) but no actual llvm libraries. (maybe build with llvm libs statically linked into the tools)?!libllvm5*{gnu,gnu7,intel,llvm5}*ohpc! LLVM libraries that can be linked to by applications built with the associated compilers. This would allow for things like!module!load!intel!libllvm5! to build something with the Intel compiler that uses LLVM as a library, also built by the Intel compiler. Thoughts? 7
OpenHPC with Bright Cluster I sent out email related to this topic earlier today Follow on to conversations at ISC with an academic user who was enabling OpenHPC packages on his Bright cluster Bright proposing to document enabling OHPC with Bright; starting effort in doc that was attached Seems to mostly do three things: - play some soft link games so that items installed from OHPC into /opt/ohpc will be visible under /cm/shared - enable OpenHPC repo via ohpc-release - fair bit of module setup hackery (e.g. create their own ohpc module) Admin can then install OHPC packages and utilize Thoughts on the approach and/or encouraging this? - not something we can test directly at the moment - might give some more exposure/usage to OpenHPC (mention on our Wiki?) - they may not know about our lmod-defaults variants which supply an ohpc module or they may not want to set default to ohpc compiler/mpi variant we could offer a vanilla lmod-defaults which just provides ohpc module (no compiler/mpi default) encourage them (or us) to provide compatibility package that enables modules functionality directly 8
PMIx Update Recall discussion from last time, general consensus to try for Option #1: - leave current MPI build configurations mostly as is (without PMIx), but enable additional variant(s) for MPIs where it works - based on this, have created openmpi3-pmix-slurm-gnu7-ohpc - have verified we can use non-pmix aware MPI family builds with version of SLURM that has PMIx support [test@sms001 ~]$ rpm -q openmpi3-gnu7-ohpc openmpi3-gnu7-ohpc-3.0.0-33.1.x86_64 [test@sms001 ~]$ mpicc hello.c [test@sms001 ~]$ srun -N 2 -n 4 --pty /bin/bash [test@c1 ~]$ prun./a.out [prun] Master compute host = c1 [prun] Resource manager = slurm [prun] Launch cmd = mpirun./a.out Hello, world (4 procs total) --> Process # 0 of 4 is alive. -> c1 --> Process # 1 of 4 is alive. -> c1 --> Process # 2 of 4 is alive. -> c2 --> Process # 3 of 4 is alive. -> c2 [test@sms001 ~]$ rpm -q openmpi3-pmix-slurm-gnu7-ohpc openmpi3-pmix-slurm-gnu7-ohpc-3.0.0-34.2.x86_64 [test@sms001 ~]$ mpicc hello.c [test@sms001 ~]$ srun -N 2 -n 4 --pty /bin/bash [test@c1 ~]$ prun./a.out [prun] Master compute host = c1 [prun] Resource manager = slurm [prun] Launch cmd = srun --mpi=pmix./a.out Hello, world (4 procs total) --> Process # 0 of 4 is alive. -> c1 --> Process # 1 of 4 is alive. -> c1 --> Process # 2 of 4 is alive. -> c2 --> Process # 3 of 4 is alive. -> c2 9
Warewulf updates Have updated WW builds for 1.3.3 effort to use newer devel branch (we call it 3.8pre) Newer version contains builds of ipxe (http://ipxe.org) which supports aarch64 and x86_64 - have verified x86 installations continue to work - note that we will likely have to call out some extra steps for those upgrading from previous WW versions bootstrap images are stored in different location - Renato previously demonstrated a successful boot of aarch64 with CentOS - SLES support was not working in devel branch, but Reese has created patches to re-enable and we have now demonstrated a successful boot of aarch64 with SLES12 SP3 at TACC - have also demonstrated booting aarch64 node from x86_64 provisioning server CI Master (x86_64) CI Pool x86_64 x86_64 x86_64 aarch64 aarch64 aarch64 ipxe 1.0.0+ -- Open Source Network Boot Firmware -- http://ipxe.org Features: DNS HTTP iscsi TFTP AoE EFI Menu net0: 00:50:b6:1a:f0:a3 using SNP on SNP-0000:00:11.0 (open) [Link:down, TX:0 TXE:0 RX:0 RXE:0] [Link status: Unknown (http://ipxe.org/1a086194)] Configuring (net0 00:50:b6:1a:f0:a3)... ok net0: 192.168.5.3/255.255.0.0 gw 192.168.0.2 Next server: 192.168.0.2 Filename: http://192.168.0.2/ww/ipxe/cfg/00:50:b6:1a:f0:a3 http://192.168.0.2/ww/ipxe/cfg/00%3a50%3ab6%3a1a%3af0%3aa3... ok 00:50:b6:1a:f0:a3 : 437 bytes [script] Now booting arm003 with Warewulf bootstrap (sles12sp3.aarch64) http://192.168.0.2/ww/bootstrap/aarch64/12513/initfs.gz... ok http://192.168.0.2/ww/bootstrap/aarch64/12513/kernel... ok EFI stub: Booting Linux Kernel... 10