When you are ready to collect measurements, make your production runs using one of the batch queues. If your jobs are short, you could run interactively, but since you will be repeating jobs multiple times and running with differ settings, you'll likely decide to submit non-interactive jobs.
The batch queue provides dedicated access to resources—though the switch will be shared with other jobs. They implement a technique called space sharing, whereby each user gets a portion of the hardware until they are done using it. Compare this with time sharing, in which users get time slices of the available resources. We will use space sharing when collecting performance measurements, since the mechanism provides dedicated access.
While batch jobs provide dedicated access, the caveat is that you will have to wait for them to return. This method of submitting jobs may be new to you, and it takes some getting used to. However, if you think about it, it would be very expensive to give each user their own dedicated resources for an indefinite period of time. Most heavily subscribed high performance computing platforms employ space sharing.
We will use the Sun Grid Engine resource manager (SGE).
Submit your
batch job using the
qsub command:
qsub batch_bang.sh
where the batch_bang.sh is a job submission script containing the appropriate environment settings and one or more runs that you wish to make.
A message will be printed at the screen
Your job 257 ("TESTJOB") has been submitted
where 257 is the job number.
You will need to make only minimal changes to the script, as noted in the file. (qsub scripts indicate options with a # mark. These are not comments unless there are two or more # in a row.)
Once your job has been submitted, it will wait in the queue until the specified resources are available. You may check on the status of your job using the qstat command. If you specify your user ID as in qstat -u sbaden you'll see only your jobs. The command output shows the job Ids and queues, for example:
job-ID prior name user state submit/start at queue slots ja-task-ID
---------------------------------------------------------------------------------------------
257 0.00000 TESTJOB baden qw 10/01/2012 08:29:45 1
You can remove jobs with the qdel command, and you only need to specify the leading number in the job ID:
qdel 257
Here is the full documentation for SGE, the batch system.
The provided script is set up to write all output to a file whose name contains a unique identifier, in TESTJOB.o257. (Stderr is merged with Stdout, but you can disable this option so that you get a separate stderr file e.g. TESTJOB.e257.)
If you modify the line with the -M option, you will be notified by email when your job started, finished or exited abnormally. (See the provided batch script and be sure to set the correct email address!) You may disable any of these 3 options independently as described in the script.
Note that if there are problems with the job you’ll be notified. For example, the following message indicates that the program could not be found
./e1: Command not found.
This might have been a result of not setting the correct directory in the cd command which appears in the provided job submission script. By default, the batch subsystem puts your job in your home directory (this includes interactive mode). So, you must cd to the correct working directory.
You many submit batch jobs from the login node only. If you attempt to submit a job from a compute node, you'll get an error message
Unable to run job: denied: host "compute-0-17.local" is no submit host. Exiting.
Maintained by [Mon Sep 24 21:52:01 PDT 2012]