Remote Jupyter Kernel at CSCS

.a README.md

Configuration

Direct Access to Computing Systems

1. Get an Account and store your account name export CSCS_USER=unixname.
2. Configure direct access to the computing system. You can use the ssh_config template:

   $ cat >> ~/.ssh/config <<EOF
   Host *
    AddKeysToAgent yes  # only for MacOS
    UseKeychain yes  # only for MacOS
    IdentityFile ~/.ssh/id_rsa
   
   Host ela.cscs.ch
    User $CSCS_USER
   
   Match host !ela.cscs.ch,*.cscs.ch
    ControlMaster auto
    ControlPath ~/.ssh/%r@%h:%p
    ControlPersist 1
    User $CSCS_USER
    ProxyCommand ssh -q -Y ela.cscs.ch -W %h:%p
   EOF
   

   NOTE: check that you don't have any conflicting rules in ~/.ssh/config.
3. Connect to the computing system (in following example use export CSCS_SYSTEM=daint.cscs.ch or export CSCS_SYSTEM=dom.cscs.ch) using ssh $CSCS_SYSTEM.

Compute Node

0. Make sure you are connected to $CSCS_SYSTEM (check previous section).
1. Configure direct access to nodes without password:

   $ ssh-keygen
   $ cat .ssh/id_rsa.pub >> .ssh/authorized_keys

2. Start new compute node in interactive mode: srun -C gpu -J test -v -u bash -i and check the output of uname -a.
3. From new terminal start new SSH connection to $CSCS_SYSTEM and check the job:

   (local)$ ssh $CSCS_SYSTEM
   ($CSCS_SYSTEM)$ squeue -u $USER -l
   Tue May  9 07:43:20 2017
     JOBID PARTITION     NAME     USER    STATE       TIME TIME_LIMI  NODES NODELIST(REASON)
    123456    normal     test    $USER  RUNNING       0:15   1:00:00      1 nid00000
   
   ($CSCS_SYSTEM)$ export CSCS_NODE=nid00000  # the name in last column 
   ($CSCS_SYSTEM)$ ssh $CSCS_NODE
   ($CSCS_NODE)$ uname -a
   Linux nid00000 3.12.60-52.49.1_2.0-cray_ari_c #1 SMP Mon Nov 21 15:40:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

   Compare the output from step 2 and if they are equal the connection was successful. Now close this terminal.
4. Close the running srun command by pressing ctrc+c twice.
5. Include lines from .bashrc and run source ~/.bashrc to test it.
6. Install IPython kernel: pip install --user ipython[kernel] and check its version: ipython kernel --version.
7. Verify the IPython kernel version on new compute node: srun -C gpu -J test -v -u ipython kernel --version

Remote IKernel

0. Make sure that you have Jupyter notebook installed otherwise run pip install --user jupyter[notebook] and check the version:

   $ jupyter notebook --version
   5.0.0

1. Install Remote IKernel package: pip install --user remote_ikernel and check the version:

   $ remote_ikernel --version
   Remote Jupyter kernel launcher (version 0.4.5). ...

2. Configure remote kernel using SLURM interface:

   $ remote_ikernel manage --add \
       --kernel_cmd="ipython kernel -f {connection_file}" \
       --name=CSCS \
       --language=Python3 \
       --interface=slurm \
       --tunnel-hosts $CSCS_SYSTEM \
       --launch-cmd="srun -C gpu"

3. Launch Jupyter notebook and select the CSCS kernel: jupyter notebook.

Using --remote-precmd

In case you don't want to edit your .bashrc, you can use --precmd option:

$ export CSCS_PRECMD=`cat - <<EOF
env -i bash --norc --noprofile
export PATH=$PATH:~/.local/bin
module load daint-gpu
module load TensorFlow/1.0.0-CrayGNU-2016.11-cuda-8.0-Python-3.5.2
EOF
`
$ ... --remote-precmd="$CSCS_PRECMD"

NOTE: check that resulting kernel.json does not include double backslashes.

.bashrc

# Include local bin path with Python packages.
export PATH=$PATH:~/.local/bin

# Make sure that all modules you need are automatically loaded.
module load daint-gpu
module load TensorFlow/1.0.0-CrayGNU-2016.11-cuda-8.0-Python-3.5.2

EULER.md

.bashrc

# Include local bin path with Python packages.
export PATH=$PATH:~/.local/bin

# Make sure that all modules you need are automatically loaded.
module load new
module load python/3.4.3

Dependencies

$ pip install --user ipython[kernel]

Remote IKernel

$ remote_ikernel manage --add \
  --kernel_cmd="ipython kernel -f {connection_file}" \
  --name="Euler" --cpus=1 --interface=lsf --tunnel-hosts euler.ethz.ch \
  --verbose --workdir /cluster/home/${USER} \
  --remote-launch-args "-R light -W 00:30" \
  --reverse --reverse-job-connect "bjob_connect {jobid}"

SageMath.md

(WIP) MacOS

$ PATH=/bin:/usr/bin /Applications/SageMath-6.10.app/Contents/Resources/sage/sage 
$ jupyter kernelspec install /Applications/SageMath-6.10.app/Contents/Resources/sage/local/share/jupyter/kernels/sagemath

Setup R.md

Local R kernel

$ pip install --user cython numpy pandas
$ pip install --user feather-format
$ R
> install.packages(c('repr', 'IRdisplay', 'evaluate', 'crayon', 'pbdZMQ', 'devtools', 'uuid', 'digest'))
> devtools::install_github('IRkernel/IRkernel')
> IRkernel::installspec()  # to register the kernel in the current R installation
> install.packages('feather')

Compute Node

0. Connect to $CSCS_SYSTEM.

   (local)$ ssh $CSCS_SYSTEM

1. Make sure the R module is always loaded. TODO: upgrade R to 3.4

   ($CSCS_SYSTEM)$ echo "module load R/3.3.2-CrayGNU-2016.11" >> ~/.bashrc
   ($CSCS_SYSTEM)$ echo "export R_LIBS=~/R" >> ~/.bashrc
   ($CSCS_SYSTEM)$ source ~/.bashrc

2. Install everything as shown in Local R kernel section. Make sure you are running it on $CSCS_SYSTEM.
   • Q: Would you like to use a personal library instead? A: y
   • Q: Please select a CRAN mirror for use in this session. A: 38 (even if 42 might sound as the right aswer for everything)

Remote R IKernel

0. Make sure the you have followed Remote IKernel section in README.md.
1. Configure remote kernel using SLURM interface:

   $ remote_ikernel manage --add \
       --kernel_cmd='R --slave -e "IRkernel::main()" --args "{connection_file}"' \
       --name=CSCS_R \
       --language=R \
       --interface=slurm \
       --tunnel-hosts $CSCS_SYSTEM \
       --launch-cmd="srun -C gpu"

2. Install (if you have not done it before) SOS-Remote-IKernel: pip install --user git+https://github.com/SwissDataScienceCenter/sos-remote-ikernel.git
3. Configure SOS_R_SLURM_NAME to be same as remote ikernel (see output of step 1):

   $ export SOS_R_SLURM_NAME=rik_slurm_cscs_r_sruncgpu_via_${CSCS_SYSTEM//./_}

   DEPRECATED: Simply use latest SoS package from GitHub.
4. Launch Jupyter notebook and select the SOS kernel: jupyter notebook.

SOS.md

Installation

DEPRECATED: Simply use latest SoS package from GitHub.

The integration of remote IPython kernel and SOS is enabled by small Python package SOS-Remote-IKernel.

0. Make sure that Compute Node and Remote IKernel are configured properly.
1. Install SOS-Remote-IKernel: pip install --user git+https://github.com/SwissDataScienceCenter/sos-remote-ikernel.git
2. Configure SOS_PYTHON_SLURM_NAME to be same as remote ikernel (see output Remote IKernel: step 2):

   $ export SOS_PYTHON_SLURM_NAME=rik_slurm_cscs_sruncgpu_via_${CSCS_SYSTEM//./_}

3. Launch Jupyter notebook and select the SOS kernel: jupyter notebook.