CSE 260 Schedule (Fall 2009)

Lecture 1, 9/24/09 (Thu): Introduction

Lecture slides Homework #0 (due Monday) Today's reading: Text. Introduction to Parallel Computing, 2nd Ed. Grama, Gupta, Karypis, and Kumar, Benjamin-Cummings, 2003 Chapter 1: ALL. For background reading on memory hierarchies, including virtual memory, see John L. Hennessy and David A. Patterson, Computer Architecture, A Qualitative Approach, Ed. 3, Morgan Kaufmann, 2003, Chapter 5, esp §5.1-4 and §5.9. Other editions will have similar chapters. This book is on reserve at the S&E library (wait until Monday, 9/28)

Lecture 2, 9/29/09 (Tue): Motivating applications; multiprocessors

Lecture slides updated@12.29PM, 9/29/09 Homework #1 (due Thurs 10/8) Today's reading: Text. Chapter 2: pp. 11-21.

Lecture 3, 10/6/09 (Tue): Address space organization, performance, cache coherence and consistency

Lecture slides posted Today's reading: Text. Chapter 2: pp. 24-30, 45-53, 61-3. Chapter 5: pp. 197-203, 205-212. For additional background material on shared memory architecture, consult Hennessy and Patterson, Computer Architecture A Quantitative Approach, Morgan Kaufmann. 3rd Ed.: Chapter 6, §6.1, §6.3 or the 4th Ed., Chapter 4, §4.1-2, 4.6

Lecture 4, 10/7/09 (Weds at 11AM, EBU3B 1202): Threads Programming

Lecture slides re-posted with revisions to code examples (Code is also available on Triton in $PUB/Examples/Lec04) Today's reading: Text. Chapter 5: pp. 279-294, 307-308.

Lecture 5, 10/8/09 (Thu): A first application; OpenMP

Homework #2 (due Tues 10/20) posted Lecture slides re-posted with omp code for Jacobi3D Today's reading: On-line reader Text. Chapter 7: pp 311-333. For reference: Solving the Discrete Poisson Equation (Jim Demmel, UC Berkeley) (Read through Successive Overrelaxation, skipping The Complexity of Solving the Poisson Equation)

Lecture 6, 10/13/09 (Tue): Message passing

Lecture slides posted Today's reading: Text. Chapter 2, §2.5 (pp 53-60); Chapter 6: §6.1-6.3 (pp 233-250) A User's Guide to MPI, by Peter Pacheco, pp. 1-10.     pdf (Or Chapters 2 and 3 from Peter Pacheco's Parallel Programming with MPI. For reference: Chapter 8   from Ian Foster's Designing and Building Parallel Programs (Adison Wesley Pub.) The Ring code is available on Triton in $PUB/Examples/Ring Code examples from the Pacheco textbook are available in $Pub/Examples/Pacheco

Lecture 7, 10/15/09 (Thurs): A first application with message passing

Lecture slides posted Today's reading: Chapter 2, §2.4.2 through 2.4.4: pp. 32-44; Chapter 6: §6.6.1-6.6.3: pp. 260-263. A User's Guide to MPI, by Peter Pacheco, pp. 11-17     pdf Or: Parallel Programming with MPI, by Peter Pacheco: Chapter 4: ALL, Chapter 5: §5.1-5: pp. 65-78 Modeling the performance of an iterative method pdf For reference:Chapter 8, §8.1 to 8.3, from Ian Foster's Designing and Building Parallel Programs (Adison Wesley Pub., on-line). Nice introduction to MPI. Code Quadrature code (trapezoidal rule): available in $PUB/Pacheco/ppmpi_c/hap04

Lecture 8, 10/20/09 (Thu): Vectorization, SIMD, GPUs

Lecture slides posted Today's reading: Programming Guidelines for Vectorizing C/C++ Compilers, A. Bik et al. Dr. Dobb’s Journal, 2/1/03. (6 pp.) (Alternatively: Wikipedia article on Vectorization) Intel SSE (2 pp.) NVIDIA Tesla: A Unified Graphics and Computing Architecture, by E. Lindholm et al., IEEE Micro, March-April 2008, Vol. 28, Issue 2, pp. 39-55. (IEEE Digital Library, accessible from any campus machine. If you are off-campus, may use the UCSD Web proxy which will enable you to access restricted content from non-UCSD Internet service providers. UCSD Active Directory Login required. ) Vectorization code examples available in $PUB/Examples/Vectorization Supplemental reading - to probe further Streaming SIMD Extensions, SSE (Wikipedia) "The CRAY-1 Computer System," R. M. Russell, Comm. ACM 21(1): 63-72 (Jan. 1978.) DOI R. L. Sites, "An analysis of the Cray-1 computer." Proc. 5th Annual Symp. Computer Architecture (ISCA ’78), ACM, pp. 101-106. (Esp. sections 4 and 5) DOI Vectorizing Loops, §6.1 in Optimizing Applications on Cray X1 (TM) Series Systems, Cray. "Vector Processors," Computer Architecture: A Quantitative Approach, 4th Ed., Appendix F, by J. L. Hennessy and D. A. Patterson, Morgan Kaufmann, 2007. (45 pages) Also available as Appendix G in the 3rd Edition, if you have the CD.

Lecture 9, 10/22/09 (Thu): Performance Programming with CUDA

Homework #3 (due Friday 10/30) posted Lecture slides posted Today's reading: Scalable Parallel Programming with CUDA, J. Nickolls et al., ACM Queue, 6(2):40-53, March/April 2008. (Both PDF and HTML) CUDA, Supercomputing for the Masses, by Rob Farber, parts I and II. (14 part article, especially helpful if you are on the GPU Technology track.) To enquire further http://www.nvidia.com/object/cuda_home.html, CUDA Zone. More articles, CUDA download and documentation. If you are on the GPU Track, you’ll want to print out the Documentation, including the CUDA Programming Guide and the Best Practices Guide GPU Resources

Lecture 10, 10/26/09 (Monday at 1pm in EBU3B (CSE) Room 1202): Accelerator Programming

Guest speaker: Michael Wolfe Today's reading: The PGI Accelerator Programming Model on NVIDIA GPUs Part I and Part II

Lecture 11, 10/27/09: Memory System Performance, Collective Communication

Lecture slides posted Today's reading: Optimization of Collective Communication Operations in MPICH, by R. Thakur, R. Rabenseifner, and W. Gropp. Int’l. J. of High Performance Computing Applications, (19)1:49-66, Spring 2005 Supplemental reading: Text. Introduction to Parallel Computing, 2nd Ed. Grama et al., Benjamin-Cummings, 2003. Chapter 4: pp. 149-160,166-179 (but §4.5.2); pp 184-189. Chapter 6: pp. 260-72. Detailed discussions about MPI collective communication (For Reference). MPI: The Complete Reference, by Marc Snir et al.

Lecture 12, 10/29/09: Performance Measurement, Advanced Communication

Lecture slides posted Today's reading: R. Van de Geign and J. Watts, SUMMA: Scalable universal matrix multiplication algorithm, Concurrency: Practice and Experience, 9:255-74 (1997) (Also on Netlib) For Reference: MPI: The Complete Reference, by Marc Snir et al. Detailed reference material on collective communications. Parallel print function. PPF is the Parallel Tools consortium's parallel print facility. For more information, consult the PPF web page. The software is installed on Triton in $(PUB)/lib/PPF, examples in $(PUB)/examples/PPF (See the README file for important information about using the software).

Lecture 13, 11/3/09: Numerical Linear Algebra on the GPU

Lecture slides posted As explained in class, there will be no formal lecture (other than some background material on Gaussian Elimination), and you are expected to contribute to the class discusions. Today's reading: Benchmarking GPUs to tune dense linear algebra, by V. Volkov and J. Demmel. Proc. 2008 ACM/IEEE Conf. on Supercomputing, Austin, TX, Nov. 15 - 21, 2008. For background on Gaussian Elimination: Notes on Gaussian Elimination, "Quick review of Gaussian Elimination," Jim Demmel, UC Berkeley A detailed explanation with worked examples http://mathworld.wolfram.com/GaussianElimination.html

Lecture 14, 11/5/09: Load Balancing

Lecture slides reposted Today's reading: Reader on Data Decomposition Text. Cummings, 2003, Chapter 3: pp. 85-86, 115-142; Chapter 8: pp. 352-372. (Skim the discussions about the pipelined algorithm.) Notes on Gaussian Elimenation, "How to Layout Matrices on Distributed Memory Machines" Jim Demmel, UC Berkeley

Lecture 15, 11/10/09: More Irregular Problems; GPU applications

Lecture slides posted Jim Demmel's notes on Simulating Particle Systems. S. Sengupta et al., Scan Primitives for GPU Computing, Graphics Hardware. pp 97--106 (2007). For further reading Bell and Michael Garland, Efficient Sparse Matrix-Vector Multiplication on CUDA, NVIDIA Technical Report NVR-2008-004, Dec. 2008. M. Harris, S. Sengupta and J. D. Owens, Parallel Prefix Sum (Scan) with CUDA, CPU Gems 3, Chapter 39 (2008). S. Chatterjee, G. E. Blelloch, and M. Zagha. Scan primitives for vector computers, Proc 1990 ACM/IEEE Conf. on Supercomputing, New York, pp. 666-675. L. Nyland, M. Harris, J. Prins Fast N-Body Simulation with CUDA, GPU Gems 3, Chapter 31 (2008). For further reading on spacefilling curves Using Space-filling Curves for Multi-dimensional Indexing by J. K. Lawder and P. J. H. King, 4th July 2000. Space-filling Curves. Applet by V. B. Balayoghan, U. Texas at Austin. Plane filling curves Space-filing Curves by Zbigniew Fiedorowic, Ohio State University. Hilbert Curve (Wolfram Research). The Peano Curve and Fractal Curves. Excerpt from from the 30th Edition of the CRC Standard Mathematical Tables and Formulas, 1995, CRC (transcribed by Silvio Levy).

Lecture 16, 11/12/09: Multi-tier computing

Project Progress report due in class Lecture slides posted There will be no formal lecture and you are expected to contribute to the class discusions. Today's reading: S. B. Baden and S. J. Fink, Communication overlap in multi-tier parallel algorithms, Proc the 1998 ACM/IEEE Conference on Supercomputing, San Jose, CA, Nov 1998, pp. 1-20. M. Kistler, J. Gunnels, D. Brokenshire, and B. Benton, Petascale computing with accelerators. SIGPLAN Not. 44(4):241-250 (Feb. 2009). To read further about Cell J. Kahle et al., Introduction to the cell multiprocessor, IBM J. Res. Dev. 49(4/5): 589-604, July 2005. Kevin Krewell, Cell Moves Into the Limelight, Microprocessor Report

  Lecture 17, 11/20/09 (Fri 3:00 to 4:20, EBU3B 1202): Programming Language Support

Lecture slides posted Today's reading: M. Frigo, C. E. Leiserson, and K. H. Randall. The implementation of the Cilk-5 multithreaded language. SIGPLAN Not. 33(5):212-223 (May. 1998).     DOI (Also see HPCC06 Challenge Class 2 in Cilk) K. Fatahalian et al., "Sequoia: Programming the Memory Hierarchy," Proc. ACM/IEEE SC 2006 Conf. PDF For reference: Cilk web page

Lecture 18, 12/1: CSE 260 Symposium

Schedule

Lecture 19, 12/2 (Wed 5:00 to 7:00p, Rm. 4140 EBU3B): CSE 260 Symposium

Schedule

Lecture 20, 12/3: CSE 260 Symposium

Schedule

---

Maintained by

baden

@

ucsd.

edu

[Wed Nov 18 23:52:15 PST 2009]