HPC & reproducible research in Guix 1.2.0

Simon Tournier, Ludovic Courtès — November 24, 2020

Version 1.2.0 of GNU Guix was announced yesterday. Some 200 people contributed more than 10,000 commits since the previous release. This post focuses on important changes for HPC users, admins, and practitioners made since version 1.1.0 was released in April 2020.

Package customization and faster “relocatable packs” for HPC

We’re giving users more flexibility on the command line, with the addition of three package transformation options: --with-debug-info (always debug in good conditions!), --with-c-toolchain, and --without-tests. Consider this example:

guix build octave-cli \
  --with-c-toolchain=fftw=gcc-toolchain@10 \
  --with-c-toolchain=fftwf=gcc-toolchain@10

The command above builds a variant of the fftw and fftwf packages using version 10 of gcc-toolchain instead of the default tool chain, and then builds a variant of the GNU Octave command-line interface using them. GNU Octave itself is also built with gcc-toolchain@10.

This other example builds the Hardware Locality (hwloc) library and its dependents up to intel-mpi-benchmarks with the Clang C compiler:

guix build --with-c-toolchain=hwloc=clang-toolchain \
           intel-mpi-benchmarks

--with-c-toolchain can come in handy if you want to check the impact of the compiler toolchain on the performance of your software stack.

The guix pack command creates “application bundles” that can be used to deploy software on machines that do not run Guix (yet!), such as HPC clusters. Since its inception in 2017, it has seen a number of improvements. Code in relocatable packs can now run more efficiently thanks to the addition of the fakechroot engine, which we covered back in May.

Reproducible research workflows

The new option --path of guix graph, shows the shortest path between two nodes. The example below shows the shortest path between the packages gmsh and cunit:

$ guix graph --path gmsh cunit
gmsh@4.6.0
metis@5.1.0
openblas@0.3.9
cunit@2.1-3

Moreover, the guix repl command can now be passed a script, which eases package exploration in Guile, notably when dealing with the new Scheme (guix transformation) module for package transformations.

On the side of long-term archival of all the software Guix packages refer to, Guix now serves the file sources.json that is ingested by Software Heritage via the nixguix loader. In addition to the “archival” check of guix lint which sends a “save” request to Software Heritage for the specified packages. More packages are continuously archived.

Packages

Here are highlights among the 2,179 packages added and 4,487 packages upgraded since the previous release:

  • Gmsh has been upgraded to 4.6.0.
  • MPICH was added, version 3.3.2.
  • Open MPI is now at 4.0.5.
  • GCC 10.2.0 is available and LLVM, Clang 11.0.0 as well.
  • Julia has been upgraded to 1.5.2.
  • MPI4PY is at 3.0.3.
  • For statisticians, there’s now a total of 1,488 R packages, many of which come from Bioconductor 3.11.

The manual has been improved in various ways. In particular, the “Programming interface” section has been expounded and should be helpful for those writing package definitions. It is fully translated into French, German, and Spanish, with partial translations in Russian and Chinese.

Do not miss the release notes for more.

Try it!

Check out the download options. If you’re on Debian or a derivative distribution, you will soon be able to apt install guix.

If you’re a first-time user, take a look at the new “Getting Started” section and try out guix help to get an overview of the available commands.

Enjoy the new release!

Unless otherwise stated, blog posts on this site are copyrighted by their respective authors and published under the terms of the CC-BY-SA 4.0 license and those of the GNU Free Documentation License (version 1.3 or later, with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts).

  • MDC
  • Inria
  • UBC
  • UTHSC