Upload Files to Astro

From CsWiki
Jump to: navigation, search

Linux & Mac

Create A Tunnel

In order to transfer files to and from Astro cluster, you must first create a tunnel.

ssh -L 12345:astro-gw:22 user_name%astro-gw@gw.cs.huji.ac.il

For HM cluster use:

ssh -L 12345:hm-gw:22 user_name%hm-gw@gw.cs.huji.ac.il

The first number in command (in this example 12345) is a local port number on your computer and it can be any free port.

Rsync

The commands in the examples for rsync are all run on the local computer!
After you run the command you will be prompt for your UNIX password on the cluster.

To rsync files and folders TO the cluster:

rsync -ave "ssh -p 12345" /path/on/local/machine  user_name@localhost:/path/on/cluster

To rsync files and folders FROM the cluster:

rsync -ave "ssh -p 12345" user_name@localhost:/path/on/cluster /path/on/local/machine 

SCP

The commands in the examples for scp are all run on the local computer!
After you run the command you will be prompt for your UNIX password on the cluster.

To scp files and folders TO the cluster:

scp -P 12345 /path/on/local/machine  user_name@localhost:/path/on/cluster

To scp files and folders FROM the cluster:

scp -P 12345 user_name@localhost:/path/on/cluster /path/on/local/machine

FileZilla

Download: FileZilla

Create a new site and in the general tab and enter the following details:

Host: localhost
Port: the local port you used to create a tunnel (in this example 12345).
Protocol: SFTP
Logon type: ask for password.

FileZilla site manager.png

Press connect and enter your user name (without %astro-gw) and Unix password (not OTP).

Filezilla user and pass.png

Windows

Putty

First open a tunnel to the cluster with Putty.

In the Session category for Astro cluster enter host name and port:

Host Name: user_name%astro-gw@gw.cs.huji.ac.il
Port: 22

For HM cluster:

Host Name: user_name%hm-gw@gw.cs.huji.ac.il
Port: 22


Putty session.png

In the SSH category under Tunnels enter Source port and Destination:

Source port: 22222
Destination: astro-gw:22

For HM cluster:

Source port: 22222
Destination: hm-gw:22

Putty tunnels.png

Source port can be any number you choose. It's recommended to use a five-digit port number so you won't be using a port that is already in use. Save the configuration for later use.

Now you can use FileZilla to move files from/to the cluster.

WinScp

To upload files from your local Windows station you can use WinScp.
Download: WinScp
Press on "New Site" and enter this details:

File protocol: SFTP
Port number: 22
User name: YOUR_USER_NAME

For Astrophysics cluster:

Host name: astro-gw

For Bioinformatics cluster:

Host name: hm-gw

WinScp login settings.png

Press "Advanced" and under "Connection" select "Tunnel" and enter the following details:

Select "Connect through ssh tunnel"
Host name: gw.cs.huji.ac.il
Port number: 22


For Astrophysics cluster:

User name: YOUR_USER_NAME%astro-gw


For Bioinformatics cluster:

User name: YOUR_USER_NAME%hm-gw

WinScp advanced settings.png

Press OK and SAVE. You can connect now by pressing Login.

Winscp login.png