Build development copy of UBDL with container

The ubdl container described here has a copy of the built ubdl code in it already.

However, that copy is read-only, and often you want to build your own copy in order to do development on various pieces. What we'll do then, is use the fact that the container has all of the dependencies the ubdl code needs and build inside of the container.

You can use either docker or singularity to do this. There are pros/cons to both. docker is probably easier to install on OS X. Also, you can modify the contents in the container (when running as root inside of it.) However, singularity is the container framework used on the Tufts cluster.

docker

(to be added)

singularity

First, get a copy via

singularity pull shub://LArbys/larbys-containers:ubdldev

Or, if on the Tufts cluster, you can find a copy at

/cluster/tufts/wongjiradlab/larbys/larbys-containers/

As a concrete example, we'll assume to be on the Tufts cluster.

First, go to your userspace in the wongjiradlab folder. It is usually your username in the wongjiradlab folder:

cd /cluster/tufts/wongjiradlab/twongj01

Clone the ubdl repository

git clone --recursive https://github.com/larbys/ubdl

If you forgot the recursive tag, go into the folder and run

git submodule init
git submodule update
cd ublarcvserver
git submodule init
git submodule update

simple script

If on the Tufts cluster, there is a script in the scripts directory to build your local copy:

sbatch scripts/tufts_submit_build.sh

This launches a grid job which runs the buildall.sh script inside the container.

by hand

If on the Tufts cluster, need to load singularity module (only done once per new shell)

module load singularity

Now start the container

singularity shell /cluster/tufts/wongjiradlab/larbys/larbys-containers/ubdl_singularity_031219.img

You'll get a prompt. Type bash to start a bash shell. It'll go something like

[twongj01@login001 larbys-containers]$ singularity shell /cluster/tufts/wongjiradlab/larbys/larbys-containers/ubdl_singularity_031219.img
Singularity: Invoking an interactive shell within container...

Singularity ubdl_singularity_031219.img:~> bash
twongj01@login001:~$ 

A note about the network folder in different contexts

The path to the wongjiradlab folder changes depending on the context. This occurs due to the way the cluster was set up to behave.

Context	wongjiradlab path
logged into the interactive node	/cluster/tufts/wongjiradlab
on a worker node	/cluster/kappa/90-days-archive/wongjiradlab
inside the container	/cluster/kappa/wongjirad

Thus when writing scripts or programs, one needs to remember in what context a given command in the script is in.

back to the instructions

Go to the cloned folder. (Remember, you're now in the container, so you have to use the right path)

cd /cluster/kappa/wongjirad/twongj01/ubdl

Setup the environment variables

source setenv.sh

Now run

source configure.sh

Then run the build script

source buildall.sh

Should be it in principle.