Skip to content

Managing your software environment

What? Be able to produce and reproduce the context of build and execution of your software: all dependencies, tools ... required to achieve the complete production (from sources to binary)

  • tools
  • their versions
  • the install parameters (e.g. hdf5 with parallel mode enabled, fftw with double precision ...)

Definition

For a complete and exhaustive definition of software environment, see glossary. In this text, we precise that "software environment" concerns all the software installed in the computer where your code will be executed (your laptop for exemple). For example, in case you run a python code, you must have python installed on the computer, some additional packages in a special version number, etc ...

If you try to execute your code on another computer, the software environnement will be different (another python version, another package version and so on..), that leads to potential execution problems or to some different results. Controlling and sharing your software envrionement becomes crucial !

The easiest way is to be able to control the build. In this case, you can be sure of any version of any package you need. If it is the case, few years later you are able to re-build the same environment and then re-use your code ! You are also able to share your environment with collaborators !

The way to do that depends on your own context, language, os and so on ... but the common way is to write in a "file" the packages you need and their version. The format of these kind of file depends on the used tools and language.

Remark: it is always a good practise to save it through git or other version control tool !

Examples of managers

Some tools are designed to make it very easy:

  • nix
  • guix
  • conda family (be aware that the use of conda is subject to special rules in the academic world and micromamba is very often a better solution)
  • os tools (apt, yum, deb, ...)
  • venv, renv
  • containers (Docker, apptainer)
  • script

How to use them ?

Whatever the chosen tool, the way to use them are roughly the same:

Steps: - build your environement with a config file or install some specific packages - activate it - use it building your software

  1. Install the needed packages 
    guix install <pkg-name>
nix-env -i <pkg-name>
micromamba install <pkg-name>
  2. Capture the environment in a file 
    guix package --export-manifest > manifest.scm
nix-env --switch-profile $NIX_USER_PROFILE_DIR/test_profile
micromamba create -n <env_name> <pkg-name1> <pkg-name2> <pkg-name3>
  3. Reuse it 
    guix shell -m manifest.scm
nix-env --switch-profile $NIX_USER_PROFILE_DIR/test_profile
micromamba activate /path/to/env

If you are able to re-use your software environment later, you are also able to share it!

Here are some external resources to learn to use such a software environment:

Moreover, these tools allows you to capture the version of the used packages, store it in a file and thus share it with the rest of the world !

Benefits:

  • ability to reproduce your env
  • ability to share your env
  • ability for anybody to reproduce the same env