Meeting of the Technical Steering Committee (TSC) Board

Transcription

1 Meeting of the Technical Steering Committee (TSC) Board Tuesday, December 5th th 11:00am ET

2 Antitrust Policy Notice Linux Foundation meetings involve participation by industry competitors, and it is the intention of the Linux Foundation to conduct all of its activities in accordance with applicable antitrust and competition laws. It is therefore extremely important that attendees adhere to meeting agendas, and be aware of, and not participate in, any activities that are prohibited under applicable US state, federal or foreign antitrust and competition laws. Examples of types of actions that are prohibited at Linux Foundation meetings and in connection with Linux Foundation activities are described in the Linux Foundation Antitrust Policy available at If you have questions about these matters, please contact your company counsel, or if you are a member of the Linux Foundation, feel free to contact Andrew Updegrove of the firm of Gesmer Updegrove LLP, which provides legal counsel to the Linux Foundation. 3

3 RPM dependency discussion With our current setup, some Base OS dependencies collide with OpenHPC dependencies: # repoquery --whatprovides "libopenblas.so.0()(64bit)" openblas-0: el7.x86_64 openblas-gnu7-ohpc-0: ohpc.x86_64 # repoquery --whatprovides "libltdl.so.7()(64bit)" libtool-ltdl-0: el7_3.x86_64 libtool-ohpc-0: x86_64 This can lead to *base* system package installation breakage in that most ohpc provided.so s are not automatically included in ld search path: - We have been lucky as (yum at least) selects shorter package name if same Provides exists. - Not sure how zypper handles it 8

4 RPM dependencies No issue with inter OpenHPC package dependencies, but can cause confusion between distro/ohpc dependency resolution Example: if an OpenHPC package is already installed (i.e. openblas-gnu7-ohpc) and a new *OS* package requires something provided by an OpenHPC package: - the OS package will be installed, but it will not work as the library is somewhere in /opt/ohpc and cannot be found by the dynamic linker # repoquery --whatrequires "libopenblas.so.0()(64bit)" grep -v ohpc wc -l 28 Up to 28 broken OS packages for openblas dependencies 9

5 A Proposal: Environment modules based dependencies Disable RPM internal dependency generator: %global _use_internal_dependency_generator 0 Create/manage dependencies based on environment modules - Modules are the OpenHPC package resolution level - No collisions with the base OS - Offers possibility to further automate modules handling in build - See as example 10

6 A Proposal: Environment modules based dependencies Scan module files to generate Provides: and Requires: # repoquery --requires openblas-gnu7-ohpc lmod-ohpc ohpc-filesystem ohpc-module(gnu7) # repoquery --provides openblas-gnu7-ohpc ohpc-module(openblas) ohpc-module(openblas/0.2.20) openblas-gnu7-ohpc = ohpc openblas-gnu7-ohpc(x86-64) = ohpc # repoquery --requires tau-gnu7-openmpi3-ohpc lmod-ohpc >= ohpc-module(gnu7-openmpi3) ohpc-module(papi) ohpc-module(pdtoolkit) 11

7 A Proposal: Environment modules based dependencies Dependency generation based on location of environment module file: moduledeps/gnu7-openmpi3/... Requires: ohpc-module(gnu7-openmpi3) depends-on openblas Requires: ohpc-module(openblas) prepend-path MODULEPATH.../gnu7/ Provides: ohpc-module(gnu7) modulefiles/gnu7/openblas/ Provides: ohpc-module(openblas) Provides: ohpc-module(openblas/0.2.20) 12

8 Possible SPEC file changes Previously: BuildRequires: cmake%{proj_delim} %ohpc_setup_compiler module load cmake Updated: %global ohpc_required_modules cmake %ohpc_load_modules The more modules are used and need loading, the easier it gets 13

9 Drawbacks Previously automatically detected base OS dependencies need to be explicitly listed New cmake.spec Requires: expat Requires: bzip2 Requires: libarchive Requires: libcurl Requires: ncurses-libs Requires: zlib Users who build their own 3rd party RPMs leveraging ohpc packages would have to also disable default RPM dependency resolution (or their resulting RPMs would be uninstallable) 14