CSE 160 Schedule

This schedule is subject to change, so check frequently

References to Pacheco refer to the course reader: An Introduction to Parallel Programming,
by Peter Pacheco, Morgan Kaufmann, 2011. References to Williams refer to C++ Concurrency in Action: Practical Multithreading, by Anthony Williams, Manning Publications, 2012. A tag posted will be added to indicate that lecture slides have become available.

[1] 9/26/13 (Thu) Introduction

Lecture slides
Today's reading:
- Pacheco: Chapter 1 (All), Chapter 2 (pp 15-18, 32-37,47-53). Page numbers refer to text not the reader (numbers at the top of the page)
- Williams: Chapter 1 (pp. 1-9; § 1.1-1.2)
- SUPERCOMPUTER vs your computer in bang-for-buck battle iPad 2 pwns Cray-2?, By Dan Olds, Posted in HPC, 8th March 2012.
To explore the further (optional)
- Historical Linpack benchmark results
- Apple's Swift: Visualized, posted 10/16/12 on Anandtech.
- iPad 4 GPU Performance Analyzed: PowerVR SGX 554MP4 Under the Hood posted by A. Shimpi on 11/2/12.
- Review Apple iPad 4 Tablet, posted on Notebookcheck by P. Kaboli, 11/12/12.

[2] 10/1/13 (Tue) Programming with Threads; Parallel Sorting (I)

Lecture slides
Today's reading:
- Williams: Chapter 1 (pp. 9-14; § 1.3-1.5), Chapter 2 (pp. 15-19, 23-25,28-32; § 2.1.1, 2.1.2, 2.2, 2.4-2.6), Chapter 3 (pp. 33-40).

[3] 10/3/13 (Thu) More Threads Programming; Performance Characterization

Lecture slides posted
Today's reading:
- Pacheco: Chapter 2 (pp. 48-53, 56-65, 74-76). Page numbers refer to text not the reader (numbers at the top of the page)

[4] 10/7/13 (Monday)
Special time and classroom: 1.00-2.20pm, EBU3B 1202 (CHANGED)

Lecture slides posted
Today's reading:
- Pacheco: Chapter 2 (pp. 48-53, 56-65, 74-76).
- Youtube Videos on Odd/Even Sort#1 and #2
- Video on what different sort algorithms sound like

10/8/13 (Tue) No lecture

[5] 10/10/13 (Thu) Performance Characterization, cont'd; Memory hierarchies, Cache Coherence and Consistency, false sharing

Lecture slides posted
Today's reading:
- Pacheco: Chapter 2 (pp 19-25, 43-46), Chapter 4 (pp. 190-5: §4.10, Caches, cache coherence, and false sharing). Readings in §4.10 available via moodle only
- To read further (optional): Eliminate False Sharing, by Herb Sutter (Dr. Dobb’s Journal)
For background reading on memory hierarchies, including virtual memory, see one of
- Patterson & Hennessy, Computer Organization and Design: The Hardware/Software Interface, Patterson & Hennessy, Morgan Kaufmann, 4th Edition
- What Every Programmer Should Know About Memory (Ulrich Drepper)

[6] 10/15/13 (Tue) Memory hierarchies, continued; particle methods.

Lecture slides posted
Today's reading:
- Pachecho: pp 32-35.
- What Every Programmer Should Know About Memory, Chapter 2: CPU caches, § 3.1-3.2 (Ulrich Drepper)

[7] 10/17/13 (Thu) The C++11 Memory model

Lecture slides posted
Today's reading:
- Williams: pp. 105-132.
To explore further (optional)
- Memory Models: A Case for Rethinking Parallel Languages and Hardware, by Sarita V. Adve and Hans-J. Boehm, Commun. ACM Vol. 53 No. 8, pages 90-101.
- Threads Basics, by Hans-J. Boehm.

10/22/13 (Tue) No lecture

[8] 10/24/13 (Thu) OpenMP

Lecture slides posted
Today's reading:
- Readings on OpenMP (Read one)
  - Getting Started with OpenMP, Intel.
  - OpenMP: A Portable Solution for Threading, by Shameem Akhter and Jason Roberts, Dr. Dobb's Journal (July 12, 2010). Skip Intel task queueing, Library functions and compilation [You'll need to sign up for a free subscription to view this article]

[9] 10/29/13 (Tue) Software design issues;
Load Balancing and Partitioning; NUMA

Lecture slides posted
Today's reading:
- Reader on Load Balancing and Partitioning
- OpenMP: A Portable Solution for Threading, by S. Akhter and J. Roberts, Dr. Dobb's Journal (July 12, 2010), Loop Scheduling and Partitioning [You may need to sign up for a free subscription to view this article]
- Reader on shared memory and NUMA
To explore further (optional)
- "The SGI Origin: a ccNUMA highly scalable server," J. Laudon and D. Lenoski, Proc. 24th ISCA, pp 241-251, 1997. DOI
- Hot Chips Presentation on Magny Cours (2009)
- Resources for OpenMP (including tutorials)

[10] 10/31/13 (Thu) Midterm review

Lecture slides posted (includes study problems)
Today's reading:
- Williams. § 8.2-8.3 (pp. 233-243); § 8.4.2 (pp 250-251)

[11] 11/1/13 (Friday) SDSC Tour

We'll meet at SDSC in the the main lobby outside the data center.
See this map.
SDSC is just south of RIMAC, but you will be entering from the back, which is at street level.
From the CSE Building, follow Voigt Lane up the hill.
Turn right at the parking structure. SDSC is on your left, and is shown in the picture on the map.
Walk up the stairs into the building, head up the next set of stairs, then keep walking straight. You will enter the main lobby by the elevators and the data center will be on your left.

[12] 11/5/13 (Tuesday) Midterm

The midterm will be held in our usual classroom.

[13] 11/7/13 (Thursday) Midterm Exam Return
Gaussian Elimination

Lecture slides posted
Today's reading:
- Notes on Gaussian Elimination, Jim Demmel, UC Berkeley. Read "Quick review of Gaussian Elimination" and "Pivoting to avoid tiny A(i,i)."
To probe further into Gaussian Elimination
- Numerical Recipies in C. 2nd Ed., by Press et al., Cambridge University Press, Chapter 2, pp. 32-33, §2.3 (pp. 41-48). Also On line.
- Applet for 3x3 systems. Choose "Gaussian Elimination."

[14] 11/12/13 (Tuesday) Condition Variables

Lecture slides posted
Today's reading:
- Williams. §3.2.6; (pp 54-55); §4.1-4.1.1 (pp 67-71). §10.1-10.2.3 through Brute-Force Testing.

[15] 11/14/13 (Thursday) Message passing: concepts, a programming interface (MPI), and a first application

Lecture slides posted
Today's reading:
- Pacheco. Chapter 2: §2.3.2 (pp. 32-35), §2.4.4 (pp. 53-54).
- Pacheco. Chapter 3: §3.1-3.2 (pp. 83-97), §3.4-3.4.4 (pp. 101-106)
- Tutorial on non-blocking communication(LLNL)
- MPI: The Joys of Asynchronous Communication, by Jeff Squyres.
Tutorial material about numerical integration in MPI (To explore further, optional)
- Using point-to-point communication
- Using collective communication

[16] 11/19/13 (Tuesday) Immediate mode communication; performance; Stencil methods.

Lecture slides posted
Today's reading:
- A User’s Guide to MPI, by Peter Pacheco, pp. 1-15. A partial draft of Pacheco's MPI text Parallel Programming with MPI (Morgan Kaufmann Pub., 1997).
- Pacheco. Chapter 3: §3.6 (pp. 123-127)

[17] 11/21/13 (Thursday) Stencil Methods (continued), Hypercube Algorithm, Gather/Scatter.

Lecture slides posted
Today's reading:
- Pacheco. Chapter 3: §3.4.5 (pp. 106-108), §3.4.7-3.4.8 (pp. 110-113)
- A User’s Guide to MPI, by Peter Pacheco, pp. 16-17.
- Gather and Scatter
- Gather and Scatter Examples
Parallel print function. PPF is the Parallel Tools consortium's parallel print facility. For more information, consult the PPF web page. The software is installed on in $(PUB)/lib/PPF, examples in $(PUB)/Examples/MPI/PPF (See the README file for important information about using the software).

[18] 11/26/13 (Tuesday) Cannon's Matrix Multiplication Algorithm;
Working with Communicators

Lecture slides posted
Today's reading:
- Pacheco. Chapter 3: §3.6-end (pp. 119-140).
- Lectures Notes on Parallel Matrix Multiplication, by Jim Demmel, UC Berkeley. Read the Introduction and Cannon’s algorithm on a 2D mesh.
- A User’s Guide to MPI, by Peter Pacheco, pp. 29-32, 35-36. A partial draft of Pacheco's MPI text Parallel Programming with MPI (Morgan Kaufmann Pub., 1997).

[19] 12/3/13 (Tuesday)
Floating Point Arithmetic; Supercomputing

Lecture slides posted

Today's Reading:
Programming Massively Parallel Processors: A Hands-on Approach, by David Kirk and Wen-mei Hwu, Morgan Kaufmann Publishers (2010), Chapter 7 (Floating Point). (9 pp). Available in Draft Form.

To read further about IEEE Floating Point Arithmetic
- Dealing with the Unexpected: William Kahan and the Need for Graceful Exits, M. Pössel, Scientific American, Oct 2013.
- IEEE 754: Standard for Binary Floating-Point Arithmetic, a web page with more resources.
- "What Every Computer Scientists should know about Computer Arithmetic," by David Goldberg (2001).
  HTML DOI PDF
To read about supercomputer technology
- Tianhe-2 Report, Jack Dongarra, June 2013.
- Overview of the IBM Blue Gene/P project, IBM J. of Research and Development, Vol. 52, Isssue 1/2, January 2008.
- Also see Wikipedia article on Blue Gene. Gives a longer term history of the project.
- Today's reading: The future of microprocessors, by Shekhar Borkar and Andrew A. Chien, Commun. ACM 54(5):67-77, May 2011.
- Exascale computing study: technology challenges in achieving exascale systems, by P. Kogge et al, September 2008, DARPA, 297 pp. The "Kogge report" is an encyclopedic source of information with many references.
- Exascale.org web site.
- "Software challenges in extreme scale systems," V. Sarkar, W. Harrod and A. E. Snavely. J. of Physics: Conference Series, 180(1):012045 (2009). IOP Publishing. A good overview of the software issues. For more details see the report, the next reading.
- International Technology Roadmap for Semiconductors web site

12/5 (Thursday) Final Exam Review

Lecture slides posted

12/11/13 (Wed) Final Exam

The final exam will be held from 11.30a to 2.30p.
The Exam location will become available on the UCSD Schedule of Classes later in the quarter.