HPC & reproducible research in Guix 1.3.0
Version 1.3.0 of GNU Guix was announced a few days ago. Some 212 people contributed to more than 8,300 commits since version 1.2.0 released in November 2020. This post focuses on important changes for HPC users, admins, and scientific practitioners.
A distinguishing Guix feature is its support for declarative deployment:
instead of running a bunch of
guix install and
guix upgrade commands, you
guix package --manifest=manifest.scm, where
lists the software you want to install in a snippet that looks like this:
;; This is 'manifest.scm'. (specifications->manifest (list "emacs" "firstname.lastname@example.org" "gcc-toolchain"))
Doing that installs exactly the packages listed. You can have that file under version control or share it — a very handy tool from a reproducible science perspective.
guix package --export-manifest
produces a manifest based on
the contents of an existing profile. That makes it easy to transition from
the classic “imperative” model, where you run
guix install as needed, to the
more formal declarative model. Moreover, this paves the
way to an interesting use case:
replicating a Docker image built with
guix pack where the
useful information is lost.
Likewise, the new
guix package --export-channels
exports a channel spec “pinned” to the Guix revision that was used to
deploy the packages found in your profile. The channel spec allows you
while the manifest allows you to redeploy the package set of interest —
on a different machine, or at a different point in time. Assuming
you’ve stored these two files as
redeploying the exact same software environment boils down to running
guix time-machine -C channels.scm -- package -m manifest.scm
Our vision is that these two files should accompany scientific publications as an executable, transparent, time-independent spec to recreate the software environment used in the computational experiment.
New package transformations were added to the already long list:
--with-patchprovides a way to build a bunch of packages with a patch applied to one or several of them.
--with-latestinstructs Guix to (attempt to) build the latest upstream version of a given package.
Packagers, do not miss the new Go recursive importer available with
that can create package definitions or templates thereof for whole sets of Go
guix import crate command, for Rust packages, now honors
“semantic versioning” when used in recursive mode.
Last but not least, the introduction of the
GUIX_EXTENSIONS_PATH Guix search
path makes it possible for Guix extensions, such as the Guix Workflow
Language (GWL), to have their modules automatically
One thing you will hopefully notice is that substitute installation
(downloading pre-built binaries) became faster, as we explained
This is in part due to the opportunistic use of zstd compression, which
has a high decompression throughput. The daemon and
support zstd as an additional compression method, next to gzip and lzip.
Another change that can help fetch substitutes more quickly is local
substitute server discovery. The new
--discover option of
instructs it to discover and use substitute servers on the local-area
network (LAN) advertised with the mDNS/DNS-SD protocols, using Avahi.
guix publish has a new
to advertise itself on the LAN.
On some machines with limited resources, building the Guix modules is
an expensive operation. A new procedure,
channel-with-substitutes-available from the
(guix ci) module, can
now be used to pull Guix to the latest commit which has already been
built by the build farm. Refer to the documentation for an
More scientific packages
Here are highlights among the 2,000 packages added and 3,100 packages upgraded since the previous release:
- GCC 10.3.0 is available.
- Julia has been upgraded to 1.5.3, and Guix now provides 45 Julia packages.
- Octave has been upgraded to 6.2.0.
- Scipy has been upgraded to 1.6.0 and Python to 3.8.2.
- GHC 8.8.3 is available and OCaml 4.11.1 as well. OCaml is now bootstrapped entirely from source via camlboot.
- Open MPI is now at 4.1.0.
- For statisticians, R has been upgraded to 4.0.4 and there’s now a total of 1,639 R packages, many of which come from Bioconductor 3.12.
The Guix reference manual is fully translated into French, German, and Spanish, with preliminary translations in Russian, Chinese, and other languages. Guix itself is fully translated in French, German, and Slovak, and partially translated in almost twenty other languages. Translations are now handled on Weblate, and you can help!
Do not miss the release notes for more.
The installation script has been improved to allow for more automation. Now, you could run it with:
# yes | ./install.sh
to proceed all the Guix binary install on any Linux system without any prompt!
You may also be interested in trying the Guix System demonstration VM image which now supports clipboard integration with the host and dynamic resizing thanks to the SPICE protocol.
If you’re on Debian or a derivative distribution, you will soon be able to
apt install guix.
For first-time users, do not miss the “Getting
section and try out
guix help to get an overview of the available commands.
Enjoy the new release!