Jupyter notebook

From CsWiki
Jump to: navigation, search

Working with Jupyter Notebook on iCore clusters

The Jupyter Notebook is a web application that allows you to create and share documents that contain live code, equations, visualizations, and narrative text.

There are 2 options when working with Jupyter - working with the cluster web browser and working with a local web browser installed on your PC or laptop.

Cluster Browser

Close the Firefox browser on your PC.

Linux and OSX

Login to the cluster with X11 forwarding.

Use capital X in the ssh command.

For cs cluster:

ssh -X YOUR_USER_NAME%river@gw.cs.huji.ac.il

For astro cluster:

ssh -X YOUR_USER_NAME%astro-gw@gw.cs.huji.ac.il

For hm cluster:

ssh -X YOUR_USER_NAME%hm-gw@gw.cs.huji.ac.il

After succesful login submit an interactive job:

srun -c2 --mem=5G --time=02:00:00 --pty $SHELL

You will get a shell on one of the nodes in the cluster. Start Jupyter with this command:

jupyter-notebook

Windows with Mobaxterm

Check that X server is running (button at the top right corner of Mobaxterm window).

Verify that X11 forwarding is enabled in the advanced settings of the SSH session:

Mobaxterm x11 forwarding.png

After succesful login submit an interactive job:

srun -c2 --mem=5G --time=02:00:00 --pty $SHELL

You will get a shell on one of the nodes in the cluster. Start Jupyter with this command:

jupyter-notebook

Windows with Putty

Check that X11 forwarding is selected under Connection -> SSH -> X11:

Putty x11 forwarding.png

After succesful login submit an interactive job:

srun -c2 --mem=5G --time=02:00:00 --pty $SHELL

You will get a shell on one of the nodes in the cluster. Start Jupyter with this command:

jupyter-notebook

Local Browser

Linux and OSX

Login to the cluster.

For cs cluster:

ssh YOUR_USER_NAME%river@gw.cs.huji.ac.il

For astro cluster:

ssh YOUR_USER_NAME%astro-gw@gw.cs.huji.ac.il

For hm cluster:

ssh YOUR_USER_NAME%hm-gw@gw.cs.huji.ac.il

After succesful login submit an interactive job:

srun -c2 --mem=5G --time=02:00:00 --pty $SHELL

You will get a shell on one of the nodes in the cluster. Get the IP of that node:

hostname -i

Copy the IP and start Jupyter without a browser with this command:

jupyter-notebook --no-browser --ip=0.0.0.0

You will see at the end of the output in the terminal a line like this:

http://0.0.0.0:8888/?token=56bae041a8440d1d2c10af3260aa63a597969ebb4989b048

Notice the port number after the 0.0.0.0. In this case it's the default one for Jupyter Notebook: 8888.

Open a new terminal on your PC and run this command to create a tunnel to the astro cluster:

ssh -L PORT_NUM:IP_OF_NODE:PORT_NUM YOUR_USER_NAME%astro-gw@gw.cs.huji.ac.il

Or hm cluster:

ssh -L PORT_NUM:IP_OF_NODE:PORT_NUM YOUR_USER_NAME%hm-gw@gw.cs.huji.ac.il

Copy and paste in your local browser. Change IP 0.0.0.0 to localhost:

http://localhost:PORT_NUM/?token=56bae041a8440d1d2c10af3260aa63a597969ebb4989b048

You can now work with Jupyter on the cluster from a local browser on your PC.

Windows with Mobaxterm

After succesful login submit an interactive job:

srun -c2 --mem=5G --time=02:00:00 --pty $SHELL

You will get a shell on one of the nodes in the cluster. Get the IP of that node:

hostname -i

Copy the IP and start Jupyter without a browser with this command:

jupyter-notebook --no-browser --ip=0.0.0.0

You will see at the end of the output in the terminal a line like this:

http://0.0.0.0:8888/?token=56bae041a8440d1d2c10af3260aa63a597969ebb4989b048

Notice the port number after the 0.0.0.0. In this case it's the default one for Jupyter Notebook: 8888.

Create a tunnel in Mobaxterm:

Press the Tunneling button and in the window that opens press "New SSH Tunnel". Enter ports, login node and IP you copied as in the picture below.

Mobaxterm tunnel.png

Save, give it a name (i.e. jupyter) and press the start button.

Next time you want to work with Jupyter just edit this entry.


Copy and paste in your local browser. Change IP 0.0.0.0 to localhost:

http://localhost:8888/?token=56bae041a8440d1d2c10af3260aa63a597969ebb4989b048

You can now work with Jupyter on the cluster from a local browser on your PC.

Windows with Putty

After succesful login submit an interactive job:

srun -c2 --mem=5G --time=02:00:00 --pty $SHELL

You will get a shell on one of the nodes in the cluster. Get the IP of that node:

hostname -i

Copy the IP and start Jupyter without a browser with this command:

jupyter-notebook --no-browser --ip=0.0.0.0 

You will see at the end of the output in the terminal a line like this:

http://0.0.0.0:8888/?token=56bae041a8440d1d2c10af3260aa63a597969ebb4989b048

Notice the port number after the 0.0.0.0. In this case it's the default one for Jupyter Notebook: 8888.

Create a tunnel in Putty:

Go to Connection -> SSH -> Tunnels. Enter source port and destination in the format IP:port and click Add button.

Putty tunnel.png


Copy and paste in your local browser. Change IP 0.0.0.0 to localhost:

http://localhost:8888/?token=56bae041a8440d1d2c10af3260aa63a597969ebb4989b048

You can now work with Jupyter on the cluster from a local browser on your PC.