The "Bang" HPC cluster is exclusively reserved for students
enrolled in CSE 260 in the  Fall Qtr. 2012. We will use
this machine to do all assignments not involving GPUs,
and to turn in all assignments.

1) The hardware includes 3 components:

1 SuperMicro custom server  (frontend node = ccom-bang.ucsd.edu)
1 Dell PowerEdge 1950 server  (login node = ccom-bang-login.ucsd.edu)
35 Dell PowerEdge 1950 servers  (compute nodes = compute-0-0 to compute-0-34)

The Dell PowerEdge 1950 compute nodes
are interconnected with an infiniband switch.
Each compute node has two quad-core Intel Xeon E5345
("Clovertown") 2.33GHz CPUs (thus, a total of 8 cores per compute node) and 16GB of RAM.

2) SOFTWARE
    Rocks Cluster Linux 6.0
    The Gnu compiler suite, version 4.4.6, that is  bundled with Rocks 6.0
    The Rocks-bundled "SGE" batch job scheduler
    You can only "send out" email from the login node ccom-bang-login.ucsd.edu.
    It's not possible to receive email on the login node.


3) You can login to your account on the cluster's LOGIN server
   node (not the FRONTEND server node) by typing:

    ssh YourLogin@ccom-bang-login.ucsd.edu

  Use your current campus Active Directory (AD) password.
  This password is the same one as used for the campus
  UCSD-PROTECTED wireless network, campus VPN service, and most of the
  ACMS servers/labs.

  AD accounts for graduate students are generally managed by ACMS.
  Students can self-reset their own passwords by using the "ACMS Global
  Password Change Tool" at:

        http://acms.ucsd.edu/students/gpasswd.html
  
  Faculty and staff may reset their AD password at:
  "http://adpass.ucsd.edu/".

3) Setting up your environment

The first time you log in to Bang, you'll be asked to set up your ssh key:

    It appears that you have not set up your ssh key.
    Enter passphrase (empty for no passphrase): 

In response to the question, use an empty passphrase, that, just
hit return without any other input. If you use a non-empty passphrase,
then the compute nodes cannot authenticate the login, resulting in
an error message.

Once your ssh key has been set up, modify a "dot" file, .bash_profile,
as follows:

Append the file BASH_PROFILE, located in
/share/class/public/cse260-fa12/Profiles

to the end of the .bash_profile, located in your home directory.

You will now and certain handy command aliases at your disposal,
such as setting the number of OMP threads.
You'll also have a pre-defined environment variable, called PUB,
where all public files for the course are located (such as provided code
for the programming assignments and code examples):

/share/class/public/cse260-fa12

4) Running jobs
    The login node should be used to edit, transfer files and compile code.
    It should not be used to run computational jobs; the class can easily
    overwhelm the login node if several users run on it simultaneously.
    
    The 32 compute nodes are for running computationally intensive 
    jobs and are divided into <em>interactive</em> and <em>non-interactive</em>
    subsets.

4.a] Interactive jobs.
    A user may use not more than 1 interactive node at a time, that is,
    not more than 8 cores.  

    You can request an interactive shell session on a compute node
    by typing "qlogin".  You can not only run jobs interactively, but
    also edit, debug, compile files etc.  However, it is not
    possible to submit a non-interactive job during a qlogin session.
    You must submit non-interactive jobs via the SGE scheduler.

4.b] Non-interactive jobs.

    Non-interactive jobs are submitted via the qsub command,
    and are discussed here:

    http://cseweb.ucsd.edu/classes/fa12/cse260-b/non-inter.html

    A sample job script (with comments) is provided in
            $PUB/Examples/batch_bang.sh

    The job scheduler manages resources via batch queues.  
    2 queues have been set up for the course: "debug.q" and "normal.q"
    and you may run in either queue. The queues differ according
    to the resources they provide:

  debug.q
    Both INTERACTIVE and BATCH jobs
    A job can use any available compute node
        A maximum of 1 node = 8 CPU cores
    A maximum of 1 job per user can be run in this queue
    No limits for memory usage
    Maximum runtime = 30 minutes

  normal.q
    Only BATCH jobs
    A job can use any available compute nodes
        A maximum of 32 nodes = 256 CPU cores
    A max of 2 jobs per user can be run simultaneously in this queue
    No limits for memory usage
    Maximum runtime = 1 hour


The time limits correspond to a hard runtime limit as described by
the Sun/Oracle documentation  for queue parameters for SGE:

    http://docs.oracle.com/cd/E19279-01/820-3257-12/n1ge.html

After the specified hard runtime limit, Sun Grid Engine aborts
the job using the SIGKILL signal.  If the similar soft
limit is reached, Sun Grid Engine warns the job by sending the
job the SIGUSR1 signal. This behavior is effective only if the job
catches and handles that warning signal.