Here are some on-line resources for documentation, tutorial, reference materials and course materials.
Introduction to Parallel Computing, Blaise Barney. A comprehensive, terse set of notes covering a wide range of material. You can use this document to chart your progress in learning about parallel computation.
LLNL’s High Performance Computing Training and Workshops. A collection of matierals covering a wide range of topics.
Colorado School of Mines High Peformance Computing Workshop Many code examples + slides.
Numerical Recipes in C or Fortran, 2nd Ed., by Press et al., Cambridge University Press.
A multigrid tutorial, slides by William Briggs.
Scientific Computing: An Introductory Survey, 2nd Ed., by Michael Heath, McGraw-Hill, 2002. Includes links and software modules accessible via applets.
Introduction to Linear Algebra, 4th Ed., by Gilbert Strang.
UC Berkeley Applications of Parallel Computers, taught by Jim Demmel and others. See the links to course resources including web pages from prior years. Jim Demmel wrote extensive notes in spring 1996 which are a valuable source of information.
How to write Fast Numerical Code, Markus Püschel.
Cambridge University course notes. A wide range of subjects in addition to HPC.
Algorithms in the Real World, Guy Blelloch.
IBM Red books. Some handy references, with discussions of general interest in parallel and high performance computing. A Search portal.
Performance Tuning for the Origin 2000 Though written for the Origin 2000, this material remains an excellent source of information about performance tuning, especially for shared memory computers.
Models of Parallel Computation. SGI Publication Topics in Parallel Computation
Topics in IRIX Programming (Long SGI Manual, PDF, esp. Ch. 11 on pthreads)
Netlib repository of numerical software and related reports. (If you clone this window and click here, you will get a one-page listing of all the libraries available within netlib.)
U. Florida. Sparse matrix algorithm research, matrix collection and software (U. Florida) algorithm research, matrix collection and software.
NIST. Matrix market
Software optimization resources, by Agner Fog. Extensive materials, including Optimizing software in C++ and The microarchitecture of Intel, AMD and VIA CPUs.