Guix-HPC Activity Report, 2023

Céline Acary-Robert, Emmanuel Agullo, Ludovic Courtès, Marek Felšöci, Konrad Hinsen, Arun Isaac, Ontje Lünsdorf, Pjotr Prins, Simon Tournier, Philippe Virouleau, Ricardo Wurmus — February 16, 2024

This document is also available as PDF (printable booklet)

We are pleased to publish the sixth Guix-HPC annual report. Launched in 2017, Guix-HPC is a collaborative effort to bring reproducible software deployment to scientific workflows and high-performance computing (HPC). Guix-HPC builds upon the GNU Guix software deployment tool to empower HPC practitioners and scientists who need reliability, flexibility, and reproducibility; it aims to support Open Science and reproducible research.

Continue reading…

HIP and ROCm come to Guix

Ludovic Courtès, Thomas Gibson, Kjetil Haugen, Florent Pruvost — January 30, 2024

We have some exciting news to share: AMD has just contributed 100+ Guix packages adding several versions of the whole HIP and ROCm stack! ROCm is AMD’s Radeon Open Compute Platform, a set of low-level support tools for general-purpose computing on graphics processing units (GPGPUs), and HIP is the Heterogeneous Interface for Portability, a language one can use to write code (computational kernels) targeting GPUs or CPUs. The whole stack is free and “open source” software—a breath of fresh air!—and is seeing increasing adoption in HPC. And, it can now be deployed with Guix!

Continue reading…

Continuous integration and continuous delivery for HPC

Ludovic Courtès — March 6, 2023

Will those binaries actually work? This is a central question for HPC practitioners and one that’s sometimes hard to answer: increasingly complex software stacks being deployed, and often on a variety of clusters. Will that program pick the right libraries? Will it perform well? With each cluster having its own hardware characteristics, portability is often considered unachievable. As a result, HPC practitioners rarely take advantage of continuous integration and continuous delivery (CI/CD): building software locally on the cluster is common, and software validation is often a costly manual process that has to be repeated on each cluster.

Continue reading…

Guix-HPC Activity Report, 2022

Céline Acary-Robert, Ludovic Courtès, Yann Dupont, Marek Felšöci, Konrad Hinsen, Ontje Lünsdorf, Pjotr Prins, Philippe Swartvagher, Simon Tournier, Ricardo Wurmus — February 10, 2023

This document is also available as PDF (printable booklet).

Continue reading…

Back to the future: modules for Guix packages

Ludovic Courtès — May 6, 2022

Some things in our software world are timeless. The venerable Environment Modules are one of these. If you’ve ever used a high-performance cluster in the last three decades, chances are you’re already familiar with it. Modules is about managing software environments, just like Guix is—or, perhaps more accurately, guix shell.

Continue reading…

Guix-HPC Activity Report, 2021

Pierre-Antoine Bouttier, Ludovic Courtès, Yann Dupont, Marek Felšöci, Felix Gruber, Konrad Hinsen, Arun Isaac, Pjotr Prins, Philippe Swartvagher, Simon Tournier, Ricardo Wurmus — February 3, 2022

This document is also available as PDF (printable booklet).

Continue reading…

Tuning packages for a CPU micro-architecture

Ludovic Courtès — January 6, 2022

It should come as no surprise that the execution speed of programs is a primary concern in high-performance computing (HPC). Many HPC practitioners would tell you that, among their top concerns, is the performance of high-speed networks used by the Message Passing Interface (MPI) and use of the latest vectorization extensions of modern CPUs.

Continue reading…

Guix-HPC Activity Report, 2020

Lars-Dominik Braun, Ludovic Courtès, Pjotr Prins, Simon Tournier, Ricardo Wurmus — February 9, 2021

This document is also available as PDF (printable booklet).

Continue reading…

Faster relocatable packs with Fakechroot

Ludovic Courtès — May 18, 2020

The guix pack command creates “application bundles” that can be used to deploy software on machines that do not run Guix (yet!), such as HPC clusters. Since its inception in 2017, it has seen a number of improvements, such as the ability to create Docker and Singularity container images. Some clusters lack these tools, though, and the addition of relocatable packs was a way to address that. This post looks at a new execution engine for relocatable packs that has just landed with the goal of improving performance.

Continue reading…

Guix-HPC Activity Report, 2019

Ludovic Courtès, Paul Garlick, Konrad Hinsen, Pjotr Prins, Ricardo Wurmus — February 17, 2020

This document is also available as PDF (printable booklet).

Continue reading…

Optimized and portable Open MPI packaging

Ludovic Courtès — December 19, 2019

High-performance networks have constantly been evolving, in sometimes hard-to-decipher ways. Once upon a time, hardware vendors would pre-install an MPI implementation (often an in-house fork of one of the free MPI implementations) specially tailored for their hardware. Fortunately, this time appears to be gone. Despite that, there is still widespread belief that MPI cannot be packaged in a way that achieves best performance on a variety of contemporary high-speed networking hardware.

Continue reading…

Guix-HPC Activity Report, 2018

Eric Bavier, Ludovic Courtès, Paul Garlick, Pjotr Prins, Ricardo Wurmus — February 12, 2019

This document is also available as PDF (printable booklet).

Continue reading…

PiGx paper awarded at the International Conference on Genomics (ICG-13)

Ricardo Wurmus — January 11, 2019

December 2018 the Akalin lab at the Berlin Institute of Medical Systems Biology (BIMSB) published a paper about a collection of reproducible genomics pipelines called PiGx that are made available through GNU Guix. The article was awarded third place in the GigaScience ICG-13 Prize. Representing the authors, Ricardo Wurmus was invited to present the work on PiGx and Guix in Shenzhen, China at ICG-13.

Ricardo Wurmus presenting at ICG-13.

Ricardo urged the audience of wet lab scientists and bioinformaticians to apply the same rigorous standards of experimental design to experiments involving software: all variables need to be captured and constrained. To demonstrate that this does not need to be complicated, Ricardo reported the experiences of the Akalin lab in building a collection of reproducibly built automated genomics workflows using GNU Guix.

Due to technical difficulties the recording of the talk was lost, so Ricardo re-recorded the talk a few weeks later.

Paper on reproducible bioinformatics pipelines with Guix

Ricardo Wurmus — May 9, 2018

I’m happy to announce that the bioinformatics group at the Max Delbrück Center that I’m working with has released a preprint of a paper on reproducibility with the title Reproducible genomics analysis pipelines with GNU Guix.

Continue reading…

Guix-HPC debut!

Ludovic Courtès, Roel Janssen, Pjotr Prins, Ricardo Wurmus — September 5, 2017

This post marks the debut of Guix-HPC, an effort to optimize GNU Guix for reproducible scientific workflows in high-performance computing (HPC). Guix-HPC is a joint effort between Inria, the Max Delbrück Center for Molecular Medicine (MDC), and the Utrecht Bioinformatics Center (UBC). Ludovic Courtès, Ricardo Wurmus, Roel Janssen, and Pjotr Prins are driving the effort in each of these institutes, each one focusing specific areas of interest within this overall Guix-HPC effort. Our institutes have in common that they are users of HPC, and that, as scientific research institutes, they have an interest in using reproducible methodologies to carry out their research.

Continue reading…

  • MDC
  • Inria
  • UBC