Setting up ERISOne
ERISOne is a “computing cluster with a job scheduling system for batch jobs, remote desktops for graphical applications and GPUs running on a Linux OS” maintained by Partners Healthcare.
Ideally, users do not need to purchase and maintain compute workstations. Rather, we use Macbooks to connect to a cluster behind the Partners firewall, This is ideal for working on a shared data repository, especially since we analyze PHI.
Unfortunately, setting up a modern data science stack on ERISOne is challenging. The default software packages are outdated, and obtaining desired memory, GPU, and CPU resources on a compute node adds friction to workflow.
This guide walks you through setup of Zsh, git, . It assumes you requested and have an ERISOne account, and connect via SSH (see RISC guide here).
Note: I recommend you follow the steps in order. For example, you should install git before oh-my-zsh because the commands for oh-my-zsh require a recent version of git.
Install Zsh shell
ERISOne comes with Zsh 4.3.11, but the current version is 5.8. Unfortunately since users lack root access on ERISOne, zsh cannot be installed via
Note: you may need to first install
This means we have to install Zsh from source:
(base) -bash-4.1$ curl -L https://downloads.sourceforge.net/project/zsh/zsh/5.8/zsh-5.8.tar.xz > zsh.tar.xz (base) -bash-4.1$ tar xf zsh.tar.xz (base) -bash-4.1$ cd zsh (base) -bash-4.1$ ./configure --prefix="$HOME/local" \ CPPFLAGS="-I$HOME/local/include" \ LDFLAGS="-L$HOME/local/lib" (base) -bash-4.1$ make -j && make install
Now that Zsh is installed, let’s change our default logging shell. However, users on ERISOne do not have root, and thus cannot use
To set Zsh as the default shell, add the following to
export PATH=$HOME/local/bin:$PATH export SHELL=`which zsh` [ -f "$SHELL" ] && exec "$SHELL" -l
source ~.bash_profile to make the changes take effect. Now
zsh will be activated each time you log in to ERISOne.
The next time you log in, you will be prompted to set up the
zsh shell. The prompt should look different:
Delete Zsh source files from your home directory:
[ab123@eris1n3]~% rm -rf ~/zsh*
If you log out and back in, your shell should look much improved:
ERISOne has git 1.7.1 by default, which was released in 2010. Subsequent installation of oh-my-zsh involves a shell script that invokes
git clone, which was introduced in git 1.7.7. Thus, we must upgrade git.
You can install git from source, but there are many other dependencies, so doing this would take a lot of effort.
Instead we can get a sufficiently recent (albeit not most current) version of git using the
load command of
module load git/2.17.0
Check if a more recent version of
MODULE_NAME is available via
module avail MODULE_NAME.
.zshrc loads newer versions of
The newest version of Anaconda on ERISOne (via
4.4.0-p3, whereas the current Linux version is
4.8.3. Ideally, one would use the existing Anaconda module on ERISOne and set up a new environment with the desired Python version and packages. Unfortunately, there is an error with older versions of Conda that prevents users from even creating new environments:
RemoveError: 'setuptools' is a dependency of conda and cannot be removed from conda's operating environment.
To set up the desired Python environment, install Miniconda following the instructions in my Linux setup post.
Like most Linux machines, ERISOne uses the bash shell by default. Therefore Conda will modify
/.bashrc with the appropriate path for you to call
conda. However, only
/.bash_profile is called when you log in to ERISOne.
To ensure Conda is on your path so you can call
echo 'export source ~/.bashrc' >> ~/.bash_profile
After you install Anaconda, log out, and log back in ERISOne, your command line prompt should reflect that you are now using the base environment:
Don’t forget to delete the Anaconda installer.
Interactive session on compute node
Request a compute node on
interact-big instead of running jobs on the login nodes (
bsub -q interact-big -Is -XF -n 4 $HOME/local/bin/zsh
Read more about compute nodes on ERISOne here.
Mount ERISOne as a network drive
First, create the directory on your local machine in which to mount your ERISOne home directory:
$ USER="ab123" $ MOUNTDIR="/mnt/$user" $ sudo mdkir $MOUNTDIR
This creates an empty directory in
If you use
zshyou may need to do
sudo \mkdir $MOUNTDIR.
You have two options to mount:
sudo sshfs -o allow_other $USER@erisone.partners.org:/PHShome/$USER $MOUNTDIR
sudo mount -t cifs -o username=$USER,domain=partners,vers=1.0,uid=1000 //eris1fs2.partners.org/$USER /$MOUNTDIR
You will be prompted to enter your password.
If successful, you won’t get any errors. You can then navigate to the local mount location and access your home directory on ERISOne:
$ cd $MOUNTDIR $ pwd /mnt/ab123 $ erik@localmachine > ls -l total 1024 drwxrwx---+ 5 erik 10064 0 Jan 12 01:07 local drwxrwxr-x+ 3 erik 10064 0 Nov 3 2017 lsf drwxrwx---+ 2 erik 10064 0 Jul 29 09:00 perl5 drwxrwx---+ 4 erik 10064 0 Jan 12 10:10 repos lrwxrwxrwx 1 erik root 16 Jul 26 2019 scratch -> /scratch/e/ab123