The text for this course is The Art of Parallel Programming, 2nd Ed., by Bruce Lester, First World Publishing, 2006. In addition to the text, course readings will consist of class handouts and on-line resources. This schedule may change, so check frequently. Lecture slides will be made available after lecture.

There will be no lecture slides: we'll discuss the papers in class.
Lecture
Slides
 Date Subject Readings
1 9/21 Introduction to parallel computation Lester, Chapter 1 (ALL).
2 9/26 Theoretical basis, PRAMs, data parallelism, C* Lester, pp. 53-88 (skip pp 57-63)
3 9/28 Process communication; Message passing, Introduction to MPI Lester: pp. 121-130, 225-228, 265-274, 361-368
4 10/3 First applications with MPI
  • Lester: pp. 165-168, 382-387
  • A User's Guide to MPI, by Peter Pacheco, pp. 1-17.   pdf
    (Chapters 2 and 3 from Peter Pacheco's Parallel Programming with MPI, Morgan Kauffman, 1997)
  • Parallel Programming with MPI, by Peter Pacheco (Morgan Kauffman, 1997), pp 53-60   pdf
    • 5 10/5 N-body simulations, parallel sorting, stencil methods Lester, pp. 309-323, 341-354
    Misleading Performance Reporting in the Supercomputing Field, by David H. Bailey. Scientific Programming, vol. 1., No. 2 (Winter 1992), pp. 141-151.
    6 10/10 Parallel print function; Performance modeling and scalability; Managing communicators; Matrix multiplication
    A2 due in class     		Lester: pp. 375-382, re-read 345-352.
    Parallel Programming with MPI, P. Pacheco, Morgan Kaufmann Pub. (1997): pp 245-265 (Handed out in class).
    7 10/12 Revisiting MPI performance and semantics. Strategies for parallelizing an application
  • Lester, Chapter 2: pp 68-72, Chapter 9: pp 327-341, Chapter 10: 391-7.
  • For detailed discussion of Fox's method (an alternative matrix multiply algorithm), including code: A User's Guide to MPI, by Peter Pacheco Draft version: pp 29-43, or from the book: Parallel Programming with MPI, P. Pacheco, Morgan Kaufmann Pub. (1997), Chapter 7 (All).
    • - 10/13 Quiz #1 In section
    8 10/17 Advanced parallel sorting Lecture slides reposted
  • Lester, pp 140-143.
  • Reader on Bitonic Sort
  • Parallel Programming with MPI, by Peter Pacheco (Morgan Kauffman, 1997), pp 226-241, 316-324. Handed out in class.
    • 9 10/19 Solving systems of linear equations
  • Notes on LU Decomposition, Jim Demmel, UC Berkeley.
  • For reference: Numerical Recipies in C, 2nd Ed., by Press et al., Cambridge University Press, pp 41-48: Gaussian Elimination with Backsubstitution and LU Decomposition and Its Applications
    On the web...
  • A detailed explanation with worked examples http://mathworld.wolfram.com/GaussianElimination.html    
  • An applet for 2 x 2 systems with a geometric interpretation: http://www.freeboundaries.org/java/la_applets/GaussElim/
    • - 10/20 A3 due In section
    10 10/24 Fundamental limitations of parallel computer architectures Guest lecture by Allan Snavely
    11 10/26 Midterm review
    - 10/27 A4 due In section
    12 10/30 Midterm SSB 106, 7:00 to 8:20 PM
    13 10/31 Shared memory architecture
  • Lester, Chapter 3 (ALL). Chapter 5: pp. 151-164.
  • In Search of Clusters, 2nd Ed., by Gregory Pfister, Prentice Hall: pp. 131-5, 152-171. (Handed out in class).
  • Supplemental notes on Shared Memory
    • 14 11/2 Shared memory programming, MPI Topologies
  • Pfister Handout: pp 185-202, 235-268
  • A User's Guide to MPI, by Peter Pacheco, pp. 35-43.   pdf
    (pp. 121-8 from the hard copy version: Parallel Programming with MPI, Morgan Kauffman, 1997).
    • 15 11/7 Parallel programmming languages
    A5 part I due     		Lester, pp. 53-57 (re-read).
    Reader on data parallel programming
    16 11/9 More on UPC. Performance Programming. The best sources are the UPC Manual or presentations from the Language Tutorials page in the UPC Wiki. See UPC Application Code From the Berkeley UPC distribution
    17 11/17 Irregular problems Notes on Irregular Problems
    9.30 to 11.00 AM, Friday (special date and time)
    18 11/21 Irregular problems, continued
    Quiz #2
  • Graph Partitioning, Part I, Jim Demmel, UC Berkeley.
    Read Introduction to Graph Partitioning
  • Graph Partitioning, Part II, Jim Demmel, UC Berkeley.
    Read the last section on PARTI.
    • - 11/22 A5 (complete) due by 5:00PM
    19 11/28 The future of large scale computing
  • Reading on INstruction Level Parallelism (Wikipedia)
  • Synergistic Processing in Cell's Multicore Architecture, by M. Gschwind et al., IEEE Micro, 26(2):10-24, March-April 2006.
  • Reference material on the CELL: The Cell project at IBM Research and
    Cell Broadband Engine resource center (IBM)
    • 20 11/30 Final Exam Review
    - 12/7 Final exam
    		3pm to 6pm
     



    September 2006
    Su Mo Tu We Th Fr Sa
              1 2
    3 4 5 6 7 8 9
    10 11 12 13 14 15 16
    17 18 19 20 21 22 23
    24 25 26 27 28 29 30
    October 2006
    Su Mo Tu We Th Fr Sa
    1 2 3 4 5 6 7
    8 9 10 11 12 13 14
    15 16 17 18 19 20 21
    22 23 24 25 26 27 28
    29 30 31        
    November 2006
    Su Mo Tu We Th Fr Sa
          1 2 3 4
    5 6 7 8 9 10 11
    12 13 14 15 16 17 18
    19 20 21 22 23 24 25
    26 27 28 29 30    
    December 2006
    Su Mo Tu We Th Fr Sa
              1 2
    3 4 5 6 7 8 9
    10 11 12 13 14 15 16
    17 18 19 20 21 22 23
    24 25 26 27 28 29 30
    31            

    Scott B. Baden, UCSD © 2006

    Department of Computer Science and Engineering
    University of California, San Diego
    La Jolla, California, USA