HPC goodies in Guix 0.15.0

Version 0.15.0 of Guix was released today. As usual, it brings packages and features that we hope HPC users and sysadmins will enjoy. This release brings us close to our goals for 1.0, so it’s probably one of the last zero-dot-something releases.

Over the 1,200+ packages added and 2,200+ packages updated since the 0.14.0 release, there are many noteworthy HPC additions.

• Open MPI has been updated to 3.0.1. It now comes with plugins for Intel OmniPath and Intel TrueScale via the PSM2 and PSM packages.
• hwloc 2.0 is now available alongside the 1.x series, which some applications such as Slurm still require.
• fftw was upgraded to 3.3.7. It is now built with support for SIMD extensions selected at run-time, which addressed performance concerns we discussed a while back.
• superlu-dist was upgraded to 5.3.0.
• The libpfm4 performance monitoring library was added.
• Spindle, a tool to improve the performance of dynamic library and Python module loading on HPC clusters, as well as the companion LaunchMON tool are now available, thanks to Pierre-Antoine Rouby who is currently working as an intern at Inria.
• OpenCL headers, POCL, and Beignet were contributed by “Fis Trivial”, without support for the lock-in NVIDIA devices, but with support for modern Intel CPUs and GPUs.
• Other noteworthy changes include the addition of Elemental and QD, as well as upgrades to CERES and Eigen, thanks to Eric Bavier of Cray, Inc.

There have been many changes among the several hundreds of bioinformatics and statistics packages available in Guix and we won’t list them here. As part of the work on reproducible bioinformatics pipeline, Ricardo Wurmus of the Max Delbrück Center for Molecular Medicine contributed lots of fixes enabling bit-reproducible builds of packages, in particular Python and R packages.

Changes to core Guix relevant to HPC include:

• The guix pack command can now produce relocatable executables. The technique relies on user namespaces so it’s unfortunately not directly usable on some clusters. Nevertheless, it’s a simple and powerful way to distribute applications for use on non-Guix systems.
• guix pack can now produce application bundles in the form of SquashFS images, thanks to Ricardo Wurmus. These can in turn be executed by Singularity, a lightweight “container engine” that specifically targets HPC usage.
• The guix pull command, which upgrades Guix and its package collection, now supports rollbacks (like guix package) and it has a new --list-generations option to visualize past upgrades. This is probably a great user interface improvement to scientists who care about being able to reproduce and compare their software environments.
• guix-daemon will now reject attempts to run guix gc from a remote node. This is done such that, on cluster installations, garbage collection can only be triggered from the master node, which is typically only accessible to sysadmins. This was contributed by Roel Janssen of the Utrecht Bioinformatics Center.
• Roel Janssen’s hpcguix-web, the web interface to browse packages that you can see on this site, is now available as a Guix system system, thanks to the work of Pierre-Antoine Rouby at Inria.

That’s it for the HPC side of things. Do not miss the release notes to learn about the other goodies that the release brings.

Enjoy the new release!