Difference between revisions of "Biocomputingpage"

From CsWiki
Jump to: navigation, search
(Blanked the page)
 
(4 intermediate revisions by the same user not shown)
Line 1: Line 1:
+
 
<html><head><meta content="text/html; charset=UTF-8" http-equiv="content-type"><style type="text/css">@import url('https://themes.googleusercontent.com/fonts/css?kit=wAPX1HepqA24RkYW1AuHYA');ol{margin:0;padding:0}table td,table th{padding:0}.c3{border-right-style:solid;padding:5pt 5pt 5pt 5pt;border-bottom-color:#000000;border-top-width:1pt;border-right-width:1pt;border-left-color:#000000;vertical-align:top;border-right-color:#000000;border-left-width:1pt;border-top-style:solid;border-left-style:solid;border-bottom-width:1pt;width:105pt;border-top-color:#000000;border-bottom-style:solid}.c13{border-right-style:solid;padding:5pt 5pt 5pt 5pt;border-bottom-color:#000000;border-top-width:1pt;border-right-width:1pt;border-left-color:#000000;vertical-align:top;border-right-color:#000000;border-left-width:1pt;border-top-style:solid;border-left-style:solid;border-bottom-width:1pt;width:50.2pt;border-top-color:#000000;border-bottom-style:solid}.c15{border-right-style:solid;padding:5pt 5pt 5pt 5pt;border-bottom-color:#000000;border-top-width:1pt;border-right-width:1pt;border-left-color:#000000;vertical-align:top;border-right-color:#000000;border-left-width:1pt;border-top-style:solid;border-left-style:solid;border-bottom-width:1pt;width:72pt;border-top-color:#000000;border-bottom-style:solid}.c26{border-right-style:solid;padding:5pt 5pt 5pt 5pt;border-bottom-color:#000000;border-top-width:1pt;border-right-width:1pt;border-left-color:#000000;vertical-align:top;border-right-color:#000000;border-left-width:1pt;border-top-style:solid;border-left-style:solid;border-bottom-width:1pt;width:64.5pt;border-top-color:#000000;border-bottom-style:solid}.c21{border-right-style:solid;padding:5pt 5pt 5pt 5pt;border-bottom-color:#000000;border-top-width:1pt;border-right-width:1pt;border-left-color:#000000;vertical-align:top;border-right-color:#000000;border-left-width:1pt;border-top-style:solid;border-left-style:solid;border-bottom-width:1pt;width:65.2pt;border-top-color:#000000;border-bottom-style:solid}.c12{-webkit-text-decoration-skip:none;color:#000000;font-weight:400;text-decoration:underline;vertical-align:baseline;text-decoration-skip-ink:none;font-size:11pt;font-family:"Arial";font-style:normal}.c6{margin-left:54pt;padding-top:0pt;text-indent:-18pt;padding-bottom:8pt;line-height:1.15;orphans:2;widows:2;text-align:left}.c9{color:#222222;font-weight:400;text-decoration:none;vertical-align:baseline;font-size:10pt;font-family:"Courier New";font-style:normal}.c20{color:#222222;font-weight:400;text-decoration:none;vertical-align:baseline;font-size:11pt;font-family:"Arial";font-style:normal}.c7{color:#000000;font-weight:700;text-decoration:none;vertical-align:baseline;font-size:11pt;font-family:"Arial";font-style:normal}.c1{color:#000000;font-weight:400;text-decoration:none;vertical-align:baseline;font-size:11pt;font-family:"Arial";font-style:normal}.c28{color:#000000;font-weight:400;text-decoration:none;vertical-align:baseline;font-size:11pt;font-family:"Arial"}.c0{padding-top:0pt;padding-bottom:0pt;line-height:1.15;orphans:2;widows:2;text-align:left}.c25{border-spacing:0;border-collapse:collapse;margin-right:auto}.c16{text-decoration-skip-ink:none;-webkit-text-decoration-skip:none;color:#1155cc;text-decoration:underline}.c4{background-color:#ffffff;font-style:italic}.c30{max-width:468pt;padding:72pt 72pt 72pt 72pt}.c23{font-family:"Calibri";font-weight:400}.c2{color:inherit;text-decoration:inherit}.c24{height:145pt}.c11{font-weight:700}.c14{margin-left:18pt}.c8{background-color:#ffffff}.c27{height:37pt}.c22{text-indent:18pt}.c19{height:51pt}.c10{font-style:italic}.c5{height:11pt}.c31{height:226pt}.c18{margin-left:90pt}.c29{text-indent:36pt}.c17{height:78pt}.title{padding-top:0pt;color:#000000;font-size:26pt;padding-bottom:3pt;font-family:"Arial";line-height:1.15;page-break-after:avoid;orphans:2;widows:2;text-align:left}.subtitle{padding-top:0pt;color:#666666;font-size:15pt;padding-bottom:16pt;font-family:"Arial";line-height:1.15;page-break-after:avoid;orphans:2;widows:2;text-align:left}li{color:#000000;font-size:11pt;font-family:"Arial"}p{margin:0;color:#000000;font-size:11pt;font-family:"Arial"}h1{padding-top:20pt;color:#000000;font-size:20pt;padding-bottom:6pt;font-family:"Arial";line-height:1.15;page-break-after:avoid;orphans:2;widows:2;text-align:left}h2{padding-top:18pt;color:#000000;font-size:16pt;padding-bottom:6pt;font-family:"Arial";line-height:1.15;page-break-after:avoid;orphans:2;widows:2;text-align:left}h3{padding-top:16pt;color:#434343;font-size:14pt;padding-bottom:4pt;font-family:"Arial";line-height:1.15;page-break-after:avoid;orphans:2;widows:2;text-align:left}h4{padding-top:14pt;color:#666666;font-size:12pt;padding-bottom:4pt;font-family:"Arial";line-height:1.15;page-break-after:avoid;orphans:2;widows:2;text-align:left}h5{padding-top:12pt;color:#666666;font-size:11pt;padding-bottom:4pt;font-family:"Arial";line-height:1.15;page-break-after:avoid;orphans:2;widows:2;text-align:left}h6{padding-top:12pt;color:#666666;font-size:11pt;padding-bottom:4pt;font-family:"Arial";line-height:1.15;page-break-after:avoid;font-style:italic;orphans:2;widows:2;text-align:left}</style></head><body class="c8 c30"><p class="c0"><span class="c12">How to set up a unix account:</span><p class="c0 c5"><span class="c12"></span><p class="c0"><span>To obtain a unix user name and password, create an account by registering at</span><span><a class="c2" href="https://www.google.com/url?q=https://registrar.cs.huji.ac.il/account&sa=D&ust=1531038465218000"> </a></span><span class="c16"><a class="c2" href="https://www.google.com/url?q=https://registrar.cs.huji.ac.il/account&sa=D&ust=1531038465218000">https://registrar.cs.huji.ac.il/account</a></span><span class="c1">.</span><p class="c0 c5"><span class="c1"></span><p class="c0"><span class="c1"> </span><p class="c0"><span class="c12">Connecting to the hm server:</span><p class="c0 c5"><span class="c12"></span><p class="c0"><span>Before connecting to the hm server, an SSH connection needs to be set up. Instructions for set-up can be found at</span><span><a class="c2" href="https://www.google.com/url?q=http://wiki.cs.huji.ac.il/wiki/Connecting_from_outside&sa=D&ust=1531038465220000"> </a></span><span class="c16"><a class="c2" href="https://www.google.com/url?q=http://wiki.cs.huji.ac.il/wiki/Connecting_from_outside&sa=D&ust=1531038465220000">http://wiki.cs.huji.ac.il/wiki/Connecting_from_outside</a></span><span class="c1">.</span><p class="c0"><span class="c1"> </span><p class="c0"><span>Every time you connect to the system, you will need to use a One-Time password (OTP) in addition to your unix (IDng) password. Instructions for obtaining OTPs can be found at</span><span><a class="c2" href="https://www.google.com/url?q=http://wiki.cs.huji.ac.il/wiki/Password_and_OTP&sa=D&ust=1531038465220000"> </a></span><span class="c16"><a class="c2" href="https://www.google.com/url?q=http://wiki.cs.huji.ac.il/wiki/Password_and_OTP&sa=D&ust=1531038465220000">http://wiki.cs.huji.ac.il/wiki/Password_and_OTP</a></span><span class="c1">.</span><p class="c0"><span class="c1"> </span><p class="c0"><span class="c1">It is helpful to set up an FTP connection for easy file transfer between your desktop computer and the server. To do this with MobaXterm:</span><p class="c0"><span class="c1">Create a new SFTP session by clicking on 'Session' and then 'SFTP'. Fill in the basic settings so that remote host = localhost, username = <your unix user name>, and the port = 9001. For example:</span><p class="c0"><span class="c1"> </span><p class="c0"><span style="overflow: hidden; display: inline-block; margin: 0.00px 0.00px; border: 0.00px solid #000000; transform: rotate(0.00rad) translateZ(0px); -webkit-transform: rotate(0.00rad) translateZ(0px); width: 554.00px; height: 436.00px;"><img alt="" src="images/image3.png" style="width: 554.00px; height: 436.00px; margin-left: 0.00px; margin-top: 0.00px; transform: rotate(0.00rad) translateZ(0px); -webkit-transform: rotate(0.00rad) translateZ(0px);" title=""></span><p class="c0"><span class="c1"> </span><p class="c0"><span class="c1">You will then need to create an SSH tunnel by clicking on 'Tunneling' and then 'New SSH Tunnel', and fill in the fields (using your unix user name) as follows:</span><p class="c0 c5"><span class="c1"></span><p class="c0"><span> </span><span style="overflow: hidden; display: inline-block; margin: 0.00px 0.00px; border: 0.00px solid #000000; transform: rotate(0.00rad) translateZ(0px); -webkit-transform: rotate(0.00rad) translateZ(0px); width: 554.00px; height: 359.00px;"><img alt="" src="images/image2.png" style="width: 554.00px; height: 359.00px; margin-left: 0.00px; margin-top: 0.00px; transform: rotate(0.00rad) translateZ(0px); -webkit-transform: rotate(0.00rad) translateZ(0px);" title=""></span><p class="c0"><span class="c1"> </span><p class="c0"><span class="c1">Before you start the transfer you will need to:</span><p class="c0"><span>1 - start the tunnel by clicking on 'Tunneling' and press  the  </span><span style="overflow: hidden; display: inline-block; margin: 0.00px 0.00px; border: 0.00px solid #000000; transform: rotate(0.00rad) translateZ(0px); -webkit-transform: rotate(0.00rad) translateZ(0px); width: 23.00px; height: 25.00px;"><img alt="" src="images/image1.png" style="width: 23.00px; height: 25.00px; margin-left: 0.00px; margin-top: 0.00px; transform: rotate(0.00rad) translateZ(0px); -webkit-transform: rotate(0.00rad) translateZ(0px);" title=""></span><span class="c1"> button</span><p class="c0"><span class="c1">2- provide an OTP to open the tunnel</span><p class="c0"><span class="c1">3- start the FTP connection by double clicking on the SFTP session</span><p class="c0"><span class="c1">4- enter your unix password</span><p class="c0"><span>The connection will eventually time out (you will see that the </span><span style="overflow: hidden; display: inline-block; margin: 0.00px 0.00px; border: 0.00px solid #000000; transform: rotate(0.00rad) translateZ(0px); -webkit-transform: rotate(0.00rad) translateZ(0px); width: 23.00px; height: 25.00px;"><img alt="" src="images/image1.png" style="width: 23.00px; height: 25.00px; margin-left: 0.00px; margin-top: 0.00px; transform: rotate(0.00rad) translateZ(0px); -webkit-transform: rotate(0.00rad) translateZ(0px);" title=""></span><span class="c1"> button will no longer be pressed) after which you will need to re-do steps 1-4 to re-open it.</span><p class="c0"><span class="c1"> </span><p class="c0 c5"><span class="c1"></span><p class="c0"><span class="c12">Basic linux commands:</span><p class="c0 c5"><span class="c12"></span><p class="c0"><span class="c1">For an introduction to some basic linux commands, take a look at the Intro_hm_cluster.pptx presentation.</span><p class="c0 c5"><span class="c1"></span><p class="c0"><span class="c1"> </span><p class="c0"><span class="c12">Understanding the file system:</span><p class="c0"><span class="c1">The file system is divided into two main directories – icore and lustre.</span><p class="c0 c5"><span class="c1"></span><p class="c0"><span>Your icore directory can be found in </span><span class="c10">/cs/icore/<your unix user name>. </span><span>Icore is backed up with snapshots so a few previous versions of all files will be saved even after they are modified/deleted (assuming that the file system is up and running), however it has limited space. </span><span class="c11">Icore should be used mainly to store your scripts.</span><span> You should keep a copy of your scripts on Git, which allows you to save different versions. To use Git to back up scripts, see</span><span><a class="c2" href="https://www.google.com/url?q=http://wiki.cs.huji.ac.il/wiki/Github&sa=D&ust=1531038465224000"> </a></span><span class="c16"><a class="c2" href="https://www.google.com/url?q=http://wiki.cs.huji.ac.il/wiki/Github&sa=D&ust=1531038465224000">here</a></span><span class="c1">.</span><p class="c0 c5"><span class="c1"></span><p class="c0"><span>Your lustre directory can be found in </span><span class="c10">/mnt/lustre/hms-01/fs01/<your unix user name></span><span>. </span><span class="c11">Lustre should be used mainly to store the data</span><span> that you are currently working on as it is meant for quick and efficient computations. </span><span class="c11">IMPORTANT:</span><span> </span><span class="c11">lustre is not backed up! </span><span>This means that files on lustre that are accidentally deleted will be lost forever, so please </span><span class="c11">be sure to back up all important files </span><span class="c1">(e.g. on an external hard drive).</span><p class="c0 c5"><span class="c1"></span><p class="c0"><span class="c1">There are graphical interfaces that can make navigating the file system easier. For example, the nautilus file manager (can be opened with the command 'nautilus >& /dev/null &') and gedit text editor ('gedit >& /dev/null &'). For more information, take a look at the Intro_hm_cluster.pptx presentation.</span><p class="c0"><span class="c1"> </span><p class="c0 c5"><span class="c1"></span><p class="c0"><span class="c12">Using slurm on the hm cluster:</span><p class="c0 c5"><span class="c12"></span><p class="c0"><span class="c1">The hm cluster currently has 44 nodes on which jobs can run. Slurm is a job scheduler that decides which node(s) a job will run on based on the job's requested resources (e.g. number of cpus, amount of memory, etc…), the availability of idle nodes, and the priority of the user which depends on how many jobs were run in the past.</span><p class="c0 c5"><span class="c1"></span><p class="c0"><span class="c1">All jobs should be submitted as sbatch scripts to slurm where they will be queued and will start running as soon as all of their requested resources become available.</span><p class="c0"><span class="c1">Some resource arguments that most users will want to tweak are:</span><p class="c0 c5"><span class="c1"></span><a id="t.9f2222f9277dbab6d2a0f9d4ce0ba80b7ef940f6"></a><a id="t.0"></a>
+
{| class="c25"<tbody>
+
|- class="c27"
+
| class="c26" colspan="1" rowspan="1" | <p class="c0"><span class="c7">Argument</span>
+
| class="c13" colspan="1" rowspan="1" | <p class="c0"><span class="c7">Default value</span>
+
| class="c21" colspan="1" rowspan="1" | <p class="c0"><span class="c7">Maximum value</span>
+
| class="c15" colspan="1" rowspan="1" | <p class="c0"><span class="c7">Description</span>
+
| class="c3" colspan="1" rowspan="1" | <p class="c0"><span class="c7">Notes</span>
+
|- class="c17"
+
| class="c26" colspan="1" rowspan="1" | <p class="c0"><span class="c1">-c, --cpus-per-task</span>
+
| class="c13" colspan="1" rowspan="1" | <p class="c0"><span class="c1">2</span>
+
| class="c21" colspan="1" rowspan="1" | <p class="c0"><span class="c1">32</span>
+
| class="c15" colspan="1" rowspan="1" | <p class="c0"><span class="c1">number of cpus for job</span>
+
| class="c3" colspan="1" rowspan="1" | <p class="c0"><span class="c1">Maximum depends on CPU used in cluster, hm I-CORE cluster is 32.</span>
+
|- class="c31"
+
| class="c26" colspan="1" rowspan="1" | <p class="c0"><span class="c1">--mem</span>
+
| class="c13" colspan="1" rowspan="1" | <p class="c0"><span class="c1">50MB</span>
+
| class="c21" colspan="1" rowspan="1" | <p class="c0"><span class="c1">~64GB or ~128GB</span>
+
| class="c15" colspan="1" rowspan="1" | <p class="c0"><span class="c1">memory allocation for job</span>
+
| class="c3" colspan="1" rowspan="1" | <p class="c0"><span class="c1">Depends on memory of nodes, we have nodes with 64GB and 128GB of memory, you can't request all the memory since some is needed for the OS, if you want it all and don't care which node you can use: MaxMemPerNode. See below for more details.</span>
+
|- class="c19"
+
| class="c26" colspan="1" rowspan="1" | <p class="c0"><span class="c1">-t, --time</span>
+
| class="c13" colspan="1" rowspan="1" | <p class="c0"><span class="c1">2 hours</span>
+
| class="c21" colspan="1" rowspan="1" | <p class="c0"><span class="c1">31 days</span>
+
| class="c15" colspan="1" rowspan="1" | <p class="c0"><span class="c1">time limit for job</span>
+
| class="c3" colspan="1" rowspan="1" | <p class="c0"><span class="c1">Currently 31 days may be reverted to 21 days later.</span>
+
|- class="c24"
+
| class="c26" colspan="1" rowspan="1" | <p class="c0"><span class="c1">-N, --nodes</span>
+
| class="c13" colspan="1" rowspan="1" | <p class="c0"><span class="c1">1</span>
+
| class="c21" colspan="1" rowspan="1" | <p class="c0"><span class="c1">44</span>
+
| class="c15" colspan="1" rowspan="1" | <p class="c0"><span class="c1">number of nodes to run job on</span>
+
| class="c3" colspan="1" rowspan="1" | <p class="c0"><span class="c1">Currently 44, may be increased in the future, either due to additional hardware purchases or reassigning of submit/test hosts to 'active duty' compute nodes.</span></tbody>
+
|}<p class="c0"><span class="c1"> </span><p class="c0"><span>For more information on all sbatch arguments, e.g., the proper format for writing time limits, etc…, read the</span><span><a class="c2" href="https://www.google.com/url?q=https://slurm.schedmd.com/sbatch.html&sa=D&ust=1531038465233000"> </a></span><span class="c16"><a class="c2" href="https://www.google.com/url?q=https://slurm.schedmd.com/sbatch.html&sa=D&ust=1531038465233000">sbatch manual</a></span><span class="c1">.</span><p class="c0 c5"><span class="c1"></span><p class="c0"><span class="c1"> </span><p class="c0"><span class="c12">Submitting a job to the slurm queue:</span><p class="c0 c5"><span class="c12"></span><p class="c0"><span class="c1">There are 2 general ways to submit jobs using sbatch:</span><p class="c0 c5"><span class="c1"></span><p class="c6"><span class="c1">1.       one-liner</span><p class="c0"><span class="c1">Resources can be requested as arguments on the command line, for example:</span><p class="c0"><span class="c9">sbatch -N 2 --mem 8GB --wrap script.sh</span><p class="c0 c5"><span class="c9"></span><p class="c0"><span class="c8">The above command will submit </span><span class="c4">script.sh</span><span class="c1 c8"> as a job to slurm with two nodes and 8GB of RAM requested. The --wrap argument is needed so that the script (or command) is wrapped in an sbatch script.</span><p class="c0 c14"><span class="c1 c8"> </span><p class="c0"><span class="c1 c8">To run a command with arguments, you need to enclose the command and its arguments in quotes, for example:</span><p class="c0 c14"><span class="c1 c8"> </span><p class="c0"><span class="c9">sbatch –o output.txt --wrap 'head -n 20 file.txt'</span><p class="c0 c22"><span class="c8 c20"> </span><p class="c0"><span class="c8">The above code will submit the command 'head -n 20 file.txt' as a job to slurm and will write the output of the command (the first 20 lines of </span><span class="c4">file.txt</span><span class="c8">) to the file </span><span class="c4">output.txt</span><span class="c1 c8">.</span><p class="c0 c18"><span class="c1"> </span><p class="c6"><span class="c1">2.       sbatch script (recommended)</span><p class="c0"><span>Resources can be requested by adding special argument comments to bash scripts. These start with #SBATCH and then contain the sbatch argument, and will be parsed by sbatch until the script starts, so an #SBATCH argument written after the first command of the script will be ignored by sbatch. Here is an example of a bash script called </span><span class="c10">my-sbatch-script.sh</span><span class="c1"> with sbatch arguments:</span><p class="c0 c5"><span class="c1"></span><p class="c0"><span class="c9">#!/bin/bash</span><p class="c0"><span class="c9">#SBATCH --mem 10GB</span><p class="c0"><span class="c9">echo "hello world"</span><p class="c0"><span class="c9"># The following line will not be parsed by sbatch</span><p class="c0"><span class="c9">#SBATCH --time 10</span><p class="c0 c18"><span class="c1"> </span><p class="c0"><span class="c1">The above script can then be submitted to the slurm queue from the command line as follows:</span><p class="c0"><span class="c20 c8">sbatch my-sbatch-script.sh</span><p class="c0 c14"><span class="c10 c28"> </span><p class="c0"><span class="c10">my-sbatch-script.sh</span><span class="c1"> will be submitted as a job to the slurm queue with 10GB of RAM requested. The time limit for the job will be the default two hours since the time parameter will not be parsed by sbatch as it is written after a command in the script.</span><p class="c0 c5"><span class="c1"></span><p class="c0"><span class="c1"> </span><p class="c0"><span class="c12">Requesting resources:</span><p class="c0 c5"><span class="c12"></span><p class="c0"><span>Make sure that your job does not request more resources than the maximum (please refer to table above). </span><span class="c7">Jobs that request more than the maximum number of resources will not run.</span><p class="c0 c5"><span class="c7"></span><p class="c0"><span class="c11">Requests should be kept to as close to the needs of your job as possible</span><span class="c1"> for two main reasons:</span><p class="c6"><span class="c1">1-      jobs are prioritized in the queue mainly based on requested resources, meaning that the more resources requested, the longer the job will take to start running since it needs to wait for all resources to become available</span><p class="c6"><span class="c1">2-       while a job is running, all of its allocated resources will be unavailable to other users even if the job is not using them</span><p class="c0"><span class="c7">Requesting multiple nodes:</span><p class="c0 c5"><span class="c7"></span><p class="c0"><span>Requesting multiple nodes for a specific job does </span><span class="c11">not</span><span class="c1"> automatically divide the job among the nodes. There are a few ways to use multiple nodes simultaneously in order to save computation time:</span><p class="c6"><span class="c1">1-      using mpirun</span><p class="c6"><span class="c1">2-      using srun</span><p class="c6"><span class="c1">3-      using an outer submit script – this is recommended as it is the simplest method </span><p class="c0"><span class="c1">1- using mpirun:</span><p class="c0 c5"><span class="c1"></span><p class="c0"><span>If you know that you are running a program that depends on OpenMPI, you can use mpirun to run the program on multiple nodes in parallel – see the</span><span><a class="c2" href="https://www.google.com/url?q=https://www.mpi-forum.org/docs/&sa=D&ust=1531038465239000"> </a></span><span class="c16"><a class="c2" href="https://www.google.com/url?q=https://www.mpi-forum.org/docs/&sa=D&ust=1531038465239000">mpi manual</a></span><span class="c1"> for more information.</span><p class="c0 c5"><span class="c1"></span><p class="c0"><span class="c1">2- using srun:</span><p class="c0 c5"><span class="c1"></span><p class="c0"><span class="c1">Srun can be used within an sbatch script to utilize a subset of the total requested resources. It can be used to run a command using multiple nodes in parallel which is most useful if you want to run a script/command on many files at the same time, each as its own job on one node, for example:</span><p class="c0 c5"><span class="c1"></span><p class="c0"><span class="c9 c8">#!/bin/bash</span><p class="c0"><span class="c9 c8">#SBATCH -N 5</span><p class="c0"><span class="c9 c8"> </span><p class="c0"><span class="c9 c8">files=(file0 file1 file2 file3 file4)</span><p class="c0"><span class="c9 c8"> </span><p class="c0"><span class="c9 c8">for i in {0..4}</span><p class="c0"><span class="c9 c8">do        </span><p class="c0 c29"><span class="c9 c8">srun -N 1 –o output.$i.txt head –n 20 ${files[$i]} &</span><p class="c0"><span class="c9 c8">done</span><p class="c0"><span class="c1"> </span><p class="c0"><span>The above script will allocate five nodes as requested, and then will send five jobs to the slurm queue at the same time, each running on one node. The output of each job (the first twenty lines of the corresponding file) will be printed in the </span><span class="c10">output.{i}.txt</span><span class="c1"> file. If you submit the above script as a job without using the srun command, for example:</span><p class="c0 c5"><span class="c1"></span><p class="c0"><span class="c9 c8">#!/bin/bash</span><p class="c0"><span class="c9 c8">#SBATCH -N 5</span><p class="c0"><span class="c9 c8"> </span><p class="c0"><span class="c9 c8">files=(file0 file1 file2 file3 file4)</span><p class="c0"><span class="c9 c8"> </span><p class="c0"><span class="c9 c8">for i in {0..4}</span><p class="c0"><span class="c9 c8">do</span><p class="c0 c29"><span class="c9 c8">head –n 20 ${files[$i]} &</span><p class="c0"><span class="c9 c8">done</span><p class="c0"><span class="c1"> </span><p class="c0"><span class="c1">Once the above job starts running, it will hold five nodes unavailable to other users as requested, but it will only run on one node.</span><p class="c0 c5"><span class="c1"></span><p class="c0"><span class="c1">3- using an outer submit script:</span><p class="c0 c5"><span class="c1"></span><p class="c0"><span class="c1">The simplest way to run a script/command on many files at the same time, each as its own job on one node, is by using a for loop in a script that submits all of the jobs at once. For example:</span><p class="c0 c5"><span class="c1"></span><p class="c0"><span class="c9 c8">#!/bin/bash</span><p class="c0"><span class="c20 c8"> </span><p class="c0"><span class="c8 c9">files=(file0 file1 file2 file3 file4)</span><p class="c0"><span class="c9 c8"> </span><p class="c0"><span class="c9 c8">for file in ${files[*]}</span><p class="c0"><span class="c9 c8">do</span><p class="c0 c29"><span class="c9 c8">sbatch head.sh $file</span><p class="c0"><span class="c9 c8">done</span><p class="c0"><span class="c9 c8"> </span><p class="c0"><span>The above script loops through all files in the list 'files' and submits the script </span><span class="c10">head.sh</span><span> (with 'file' as an argument) as an sbatch job. The </span><span class="c10">head.sh</span><span class="c1"> script will contain the sbatch arguments needed to run on one file. For example:</span><p class="c0 c5"><span class="c1"></span><p class="c0"><span class="c9">#!/bin/bash</span><p class="c0"><span class="c9">#SBATCH –o output.txt</span><p class="c0"><span class="c9">#SBATCH –N 1</span><p class="c0"><span class="c9">file=$1 #first argument from outer script</span><p class="c0"><span class="c9">head –n 20 $file</span><p class="c0"><span class="c1"> </span><p class="c0"><span>The </span><span class="c10">head.sh</span><span class="c1"> script will be run as a job on one node (based on the sbatch arguments written in the script) on each of the 5 files in parallel.</span><p class="c0 c5"><span class="c1"></span><p class="c0"><span> </span><span class="c7">Requesting multiple cpus:</span><p class="c0 c5"><span class="c7"></span><p class="c0"><span class="c1">If you are running a program that supports multithreading, you can request multiple cpus (up to the maximum) and then run the program while specifying the number of cpus you want it to use as a parameter. For example:</span><p class="c0 c5"><span class="c1"></span><p class="c0"><span class="c9">#!/bin/bash</span><p class="c0"><span class="c9">#SBATCH –c 32 #request 32 cpus</span><p class="c0"><span class="c9">#SBATCH –o output.txt</span><p class="c0"><span class="c9">sample=$1</span><p class="c0"><span class="c9">bowtie2 -p 32 -x $genome -U $sample -S $sam >& bowtie2.log</span><p class="c0"><span class="c1"> </span><p class="c0"><span>The above script can be submitted as a job that will run bowtie2 using 32 cpus on 'sample' (with the bowtie2 output saved in the file </span><span class="c10">bowtie2.log</span><span class="c1">).</span><p class="c0"><span class="c7"> </span><p class="c0"><span class="c7">Requesting memory:</span><p class="c0 c5"><span class="c7"></span><p class="c0"><span class="c1">If the amount of memory requested is not enough for the job, it will be killed with an error message that the job exceeded the memory limit and you will need to increase the memory request. There is currently no simple method to know how much memory a job will need before it runs. Once an sbatch job is running you can check its accounting information with the 'sacct' command.</span><p class="c0"><span>For more information, read the</span><span><a class="c2" href="https://www.google.com/url?q=http://wiki.cs.huji.ac.il/wiki/Slurm&sa=D&ust=1531038465246000"> </a></span><span class="c16"><a class="c2" href="https://www.google.com/url?q=http://wiki.cs.huji.ac.il/wiki/Slurm&sa=D&ust=1531038465246000">CS slurm wiki</a></span><span class="c1">.</span><p class="c0 c5"><span class="c1"></span><p class="c0 c5"><span class="c12"></span><p class="c0"><span class="c12">FAQs</span><p class="c0 c5"><span class="c12"></span><p class="c0"><span class="c1">1.  How do I check the status of my job?</span><p class="c0 c5"><span class="c1"></span><p class="c0"><span>By running the command 'squeue –u username' with your username you will see all of your submitted jobs and their status (ST column). PD means your job is pending in the queue waiting to start, and R means it is running. For a list of all status codes, read the 'JOB STATE CODES' section of the</span><span><a class="c2" href="https://www.google.com/url?q=https://slurm.schedmd.com/squeue.html&sa=D&ust=1531038465247000"> </a></span><span class="c16"><a class="c2" href="https://www.google.com/url?q=https://slurm.schedmd.com/squeue.html&sa=D&ust=1531038465247000">squeue manual</a></span><span class="c1">.</span><p class="c0 c5"><span class="c1"></span><p class="c0"><span class="c1">2.  How do I cancel a job?</span><p class="c0 c5"><span class="c1"></span><p class="c0"><span class="c1">Jobs can be cancelled by running the command 'scancel jobid' where the jobid is the number that slurm assigned your job when it was submitted which you can see by running 'squeue –u username' and looking at the JOBID column.</span><p class="c0 c5"><span class="c1"></span><p class="c0"><span class="c1">3.  Why is my job taking so long to start running?</span><p class="c0 c5"><span class="c1"></span><p class="c0"><span class="c1">Jobs are prioritized in the slurm queue mainly based on the amount of resources requested. The more resources your job requests, the longer it will take to start running.</span><p class="c0 c5"><span class="c1"></span><p class="c0"><span class="c1">4.  How much memory/cpus did my job actually use?</span><p class="c0 c5"><span class="c1"></span><p class="c0"><span class="c1">When a job is running, you can use the following commands to see its accounting information:</span><p class="c0"><span>sacct – see the</span><span><a class="c2" href="https://www.google.com/url?q=https://slurm.schedmd.com/sacct.html&sa=D&ust=1531038465248000"> </a></span><span class="c16"><a class="c2" href="https://www.google.com/url?q=https://slurm.schedmd.com/sacct.html&sa=D&ust=1531038465248000">sacct manual</a></span><span class="c1"> for more details</span><p class="c0"><span>sstat – see the</span><span><a class="c2" href="https://www.google.com/url?q=https://slurm.schedmd.com/sstat.html&sa=D&ust=1531038465248000"> </a></span><span class="c16"><a class="c2" href="https://www.google.com/url?q=https://slurm.schedmd.com/sstat.html&sa=D&ust=1531038465249000">sstat manual</a></span><span class="c1"> for more details</span><p class="c0 c5"><span class="c1"></span><p class="c0"><span class="c1">5.  Can I submit a job and tell it to run at a later time?</span><p class="c0 c5"><span class="c1"></span><p class="c0"><span class="c1">You can tell a job what time you want it to start running with the parameter --begin:</span><p class="c0 c5"><span class="c1"></span><p class="c0"><span class="c9">#SBATCH --begin HH:MM MM/DD/YY</span><p class="c0"><span class="c20 c8"> </span><p class="c0"><span class="c1"> Additionally, you can add dependencies to your jobs so that, for example, job #2 will not start until job #1 satisfies the provided condition, by using the job id. This can be done in a bash script:</span><p class="c0 c5"><span class="c1"></span><p class="c0"><span class="c9">#!/bin/bash</span><p class="c0"><span class="c9">jobid1=$(sbatch job1.sh)           </span><p class="c0"><span class="c9">sbatch –-dependency afterok:jobid1 job2.sh        </span><p class="c0"><span class="c20 c8"> </span><p class="c0"><span class="c8">The above script submits </span><span class="c4">job1.sh</span><span class="c8"> as a job and saves jobid in the variable jobid1. It then submits </span><span class="c4">job2.sh</span><span class="c8"> as a job only once job1 finishes running with an exit code of 0 as defined by the condition 'afterok'. For more information on possible conditions, etc… please refer to the</span><span class="c8 c23"><a class="c2" href="https://www.google.com/url?q=https://slurm.schedmd.com/sbatch.html&sa=D&ust=1531038465250000"> </a></span><span class="c16 c8"><a class="c2" href="https://www.google.com/url?q=https://slurm.schedmd.com/sbatch.html&sa=D&ust=1531038465250000">sbatch manual</a></span><span class="c1 c8">.</span><p class="c0"><span class="c1"> </span><p class="c0"><span class="c1"> </span><p class="c0"><span class="c1"> </span><p class="c0 c5"><span class="c1"></span></body></html>
+

Latest revision as of 08:24, 9 July 2018