2020-11-18

Docs

Nodes:

we will typically use two types:

RSM with 128 G ram - analagous to SMP

AI-GPU (V100's) - for GPU

Two places to typically store files

Home space: 10 Gb only!

pylon5 ($SCRATCH): persistant or scratch (data storage) /pylon5/bio200037p/sammlapp We have 2000 something (Gb?)

Node-local ($LOCAL): disk that can be used from within a node on a job (temporary!)

Memory files ($RAMDISK)

Can use globus to transfer files

Anaconda and ffmpeg are available. We'll need to add them as some sort of module in an environment.

module -v to get help

search for modules, such as anaconda: module avail anaconda

load: module load anaconda3/2019.10 #python 3.7.4 module purge #remove all modules

pip install --user virtualenvwrapper pip install --user poetry

now run which virtualenvwrapper_lazy.sh and copy the path of that document

then open your ~/.bashrc and add these lines:

source ~/.local/bin/virtualenvwrapper_lazy.sh
export WORKON_HOME=~/.cache/pypoetry/virtualenvs

just this time: run source ~/.bashrc since it isn't sourced yet

use virtualenvwrapper as normal mkvirtualenv testenv

A replacement for JupyterHub / R Studio

First, make sure you have ssh set up by loging into xsede website and setting up dual authentication.

ssh [USER]@login.xsede.org use duo-push to log in gsissh bridges

sacctmgr show cluster sinfo

check a node: scontrol show node r007

Check our allocations projects