High-performance computing (HPC) often requires the use of multiple software packages and that they are optimised on the target machine on which these computations are running. The optimisation constraints are such that it is widely accepted that the deployment of this software can only be done manually or by relying on the work of the administrators of the target machine (typically via a load module). However, the complexity of the dependencies often results in strong constraints on the exact functionality, version and build processes of the requested software. As a result, many HPC codes choose to provide some functionality themselves, which in principle could be provided by third party libraries, contrary to the canons of software engineering.
In this talk, we will first review our quest (CMake, Spack, and now Guix) for an environment to reliably deploy HPC software in a portable, high-performance, and reproducible way, so that the use of third-party libraries is no longer a concern. We will then present our experience of deploying such a complex software stack on several machines in the French and European ecosystem. We show that we have been able to ensure a robust deployment while achieving top performance, not only on machines with Guix available but also on supercomputers where Guix is not (yet!) available.
Videos brought to you by the video team at Institut Agro and published under CC-BY-NC 3.0. Guix artwork by Luis Felipe published under CC-BY-SA 4.0.