Docker Images for Panda¶

Disclaimer¶

This directory is a work in progress. I want to provide you with a starting point if you want to try PandaRoot in a Docker container, but there is still a lot to do here.

TL;DR¶

On a Workstation¶

Download PandaRoot Source

Run rklasen/cvmfs docker container which provides FairSoft/FairRoot via CVMFS on /cvmfs

$ docker run -d --name cvmfs --privileged --cap-add SYS_ADMIN --device /dev/fuse --volume /cvmfs:/cvmfs:shared rklasen/cvmfs

The container continues to run in background. Check after about a minute whether it works

$ ls /cvmfs/fairsoft.gsi.de
centos7  centos8  debian10  debian8  macos-10.15  ubuntu1804

Run rklasen/fairroot:mini container that includes all that’s needed for PandaRoot

$ docker run -u $(id -u):$(id -g) -it --rm --mount type=bind,source=/path/to/PandaRoot,target=/mnt/work/PandaRoot --volume="/tmp/.X11-unix:/tmp/.X11-unix:rw" --volume /cvmfs:/cvmfs:shared --env="DISPLAY" rklasen/fairroot:mini

Done. Compile PandaRoot and start simulations in the second container (“rklasen/fairroot:mini”). Everything you need is already there.

On a HPC System with Singularity¶

If you cannot run the docker image (for example on cluster systems), it is easily possible to convert the mini image to a Singularity image. Because on these HPC systems the cvmfs mount often is already present, most of the work is already done. On your workstation, install Singularity and convert the mini docker image:

singularity build fairroot-mini.sif docker://rklasen/fairroot:mini

Copy that to the HPC system. On there, make sure that the PandaRoot source directory is at ~/PandaRoot (it will then automatically be found). On the cluster, you can execute the image:

module load tools/Singularity
singularity exe fairroot-mini.sif bash

And build PandaRoot directly:

cd PandaRoot
mkdir build
cd build
cmake ../ -DUSE_DIFFERENT_COMPILER=true

Nothing else is required, that’s already it. A detailed explanation is [down below].

PandaRoot in a Docker Container¶

You can compile, install and run PandaRoot in a Docker container. The advantages are:

You don’t have to install all needed packages system-wide
You can use system versions that are incompatible with PandaRoot (either too old or too new)
You can keep multiple different PandaRoot versions side by side

And probably others.

What is or should be included¶

Usually, a complete PandaRoot installation meant you have to download FairSoft, compile and install it, download FairRoot, compile and install it and download PandaRoot and compile it (and of course yous system needs to have the needed packages installed). But since FairSoft and FairRoot are being built at the GSI and made available via CVMFS mount, not all of these steps are required anymore.

Currently, these scenarios are considered (see also these slides):

All-inclusive package. That means FairSoft/FairRoot and PandaRoot are included and compiled in this image and it’s ready from the get-go. Pros: All inclusive, no internet connection is required to run. Cons: Large image size, PandaRoot code can’t be (easily) modified.
Everything except PandaRoot. This is an image that includes FairSoft/FairRoot and all necessary packages, but not the PandaRoot Code. The PandaRoot code must be present on the host system and passed to the container, where it can be compiled and run. Pros: Edit PandaRoot code on host with favorite IDE, easy and reliable installation. Cons: Image is still quite large.
Base System and required packages, but nothing else. FairSoft/FairRoot must be present in the host system (for example via CVMFS mount) and passed to the container. Pros: Small image, no further configuration required to compile PandaRoot. Cons: Requires FairSoft/FairRoot from host. Either compiled and installed on host, which defeats the point, or via CVMFS mount, which requires internet connection.

For now, only the second case is implemented here. That means you still have to download the PandaRoot source code to your machine and pass it to the docker container. This is shown below, and an example Dockerfile is provided in this directory.

Building¶

You can build the docker images with:

docker build -t fairroot:full -f Dockerfile-Full .

for the full image that includes the FairSoft/FairRoot source code, or:

docker build -t fairroot:latest -f Dockerfile-Small .

For a reduced images that contains only the compiled FairSoft/FairRoot binaries.

Note that this requires an internet connection to clone the FairSoft/FairRoot repositories. Build time may take several hours. Because of that, an image is already built and uploaded to the Docker Hub as rklasen/fairroot:latest.

Pre-Built Docker Images¶

There are three docker images ready for you:

Full: Includes FairSoft/FairRoot source and compiled
Small: Includes FairSoft/FairRoot compiled, but without source
Mini: Includes only required system packages for PandaRoot. FairSoft/FairRoot must be passed as shared volume.

The “Mini” is probably the best all-rounder; it is small and still includes everything needed to compile and run PandaRoot.

Mini¶

Because this image needs an external FairSoft/FairRoot installation, you must already have those in /cvmfs/fairsoft.gsi.de/.... On most HPC systems (HimsterII, GSI Compute farm), this should already be there. On your workstations, you can mount those two ways. Either, you do it from the host system, or you run a second docker container that mounts this to your host. Only this method is described here. Because this second container uses the hosts fuse, it requires the --privileged flag.

docker run --privileged --cap-add SYS_ADMIN --device /dev/fuse --volume /cvmfs:/cvmfs:shared rklasen/cvmfs

On your host system, the mount /cvmfs/fairsoft.gsi.de/ should now be present. Pass it to the fairroot:mini container:

docker run -u $(id -u):$(id -g) -it --rm --mount type=bind,source=/path/to/PandaRoot,target=/mnt/work/PandaRoot --volume="/tmp/.X11-unix:/tmp/.X11-unix:rw" --volume /cvmfs:/cvmfs:shared rklasen/ --env="DISPLAY" fairroot:mini

And you should be good to go. Note that this image is based on Debian, whereas the other two are based on Ubuntu.

Building mini image¶

Build the image with

docker build -t fairroot:mini -f Dockerfile-Mini .

Small¶

Still a work in progress.

Full¶

A built image is ready at the Docker hub. You can test it out with:

docker run -u $(id -u):$(id -g) -it --rm --mount type=bind,source=/path/to/PandaRoot,target=/mnt/work/PandaRoot --env="DISPLAY" --volume="/tmp/.X11-unix:/tmp/.X11-unix:rw" rklasen/fairroot

Some explanations for these flags:

-it --rm Run the container interactively and remove the container once you left it. Because nothing in the image is modified, it’s not really necessary to keep it.
--mount type=bind,source=/path/to/PandaRoot,target=/mnt/work/PandaRoot pass your local copy of the code at /path/to/PandaRoot to the container. It is then accessible on the host system for development and in the container to compile and run.
-u $(id -u):$(id -g) By default, the docker daemon runs as root, so everything created inside the container is owned by hosts root. That means all compiled artifacts and simulation data. Using user and group mapping, we can change that so that everything created in the container is owned by the hosts user instead of root.
--env="DISPLAY" --volume="/tmp/.X11-unix:/tmp/.X11-unix:rw" Enables X11 pass-through from the container to you host. This way, you can use the TBrowser for example.
rklasen/fairroot The ready-to-use fairroot image into which PandaRoot can be built.

Building a full image¶

Build the image with

docker build -t fairroot:full -f Dockerfile-Full .

Example: Build PandaRoot on Himster2¶

Short Recipe¶

Convert Docker image to FairRoot-Mini.sif Singularity Image
On Login Node, clone PandaRoot code into your home directory (Singularity mounts home by default)
On Login Node, go into Singularity container and run cmake (do this on login node, cmake downloads some stuff and compute nodes don’t have internet access)
Request interactive job on compute node to compile
On Compute Node, go into Singularity container and run make -j64

Convert Docker image to `FairRoot-Mini.sif` Image¶

Do this on your workstation, not on the Himster2.

Get Docker image:

docker pull rklasen/fairroot:mini

Install Singularity and convert to Singularity image:

singularity build fairroot-mini.sif docker://rklasen/fairroot:mini

And copy that one to the Himster2.

On Login Node, run `cmake`¶

module load tools/Singularity
singularity exe fairroot-mini.sif bash

You should be in the singularity container.

cd PandaRoot
mkdir build
cd build
cmake ../ -DUSE_DIFFERENT_COMPILER=true

The compiler option may be unnecessary. CMake should finish without errors and the build config should be written.

We could also compile here, but we are limited to 4 cores, which means long compile time. Instead, we compile on a compute node.

Request Interactive Job¶

srun --pty -p <<your_partition>> -A <<your_account>> bash --cpus-per-task 32 -i

You should get an interactive session soon. On Himster2, <<your_account>> is not your username. Verify that you are on a compute node:

hostname

Should yield something like x2002.mogon, or any other compute node. Just not a login node. Run the singularity container again:

module load tools/Singularity
singularity exe fairroot-mini.sif bash

In there, go to the build dir and start compiling:

cd PandaRoot/build
make -j32

Bask in the glory of 32 threads compiling. It should finish without errors.

Example Simulations¶

TODO (Merge requests welcome!)

Conversion to Singularity Image¶