Environment variables

You want to write re-usable code, but want to allow the person running the code to customise behaviour without having to change the code itself.

A common way of achieving this is through environment variables. Unfortunately, understanding exactly when you have access to a particular variable in a Linux context can be confusing.

Note

In general, for code that will run only on SherlockML, we suggest abstracting behaviour like this into a file in the workspace, rather than through environment variables (for instance, read how to do this for database credentials).

If your code needs to run both in SherlockML and in other environments, environment variables are a good choice.

Environment variables in the current shell

If you want to have an environment available in the current terminal session, enter the following command in that terminal:

export MY_ENVIRONMENT_VARIABLE=my_value

You can now check that that environment variable is available:

echo $MY_ENVIRONMENT_VARIABLE

All scripts that you start from this terminal session will have access to $MY_ENVIRONMENT_VARIABLE. However, any other terminal sessions will not be able to use it.

Setting environment variables for a server

You may want to set an environment variable for a server so that it is available everywhere in that server. For this, create a terminal in that server and open /etc/sherlockml_environment.d/{app}.sh, replacing {app} with any name that makes sense to you. For instance to open the file with the text editor nano, run:

nano /etc/sherlockml_environment.d/app.sh

The file name must end in .sh. In that file, write the following:

export MY_ENVIRONMENT_VARIABLE=my_value

Any file in /etc/sherlockml_environment.d that ends in .sh gets sourced (executed in the current shell, rather than in a sub-shell) when you open a new terminal or when the Jupyter or API or app process starts. You can therefore use this to give all programs in your server access to the same set of environment variables.

Any shell or program that is already running will need to be restarted. In particular, to have the environment variables available in Jupyter or JupyterLab, you will need to restart Jupyter:

sudo sv restart jupyter

You will then need to restart the kernel by clicking on Kernel > Restart in the Jupyter or JupyterLab menu.

Setting environment variables for R and RStudio

R and RStudio read environment variables in /project/.Renviron. Add your variables there by adding a line like:

MY_ENVIRONMENT_VARIABLE=my_value

This variable will only be available in new R sessions. You can restart the R session through the RStudio user interface: choose Session in the RStudio menu, then Restart R.

Setting environment variables through SherlockML environments

To set an environment variable as part of a custom environment, write the following in the scripts section of the environment:

echo "export MY_ENVIRONMENT_VARIABLE=my_value" > /etc/sherlockml_environment.d/app.sh

# Optionally force a restart of the app running on that server:
sudo sv restart jupyter
../_images/environment_variable_custom_environment.png