Lectures and Readings : Parallel Programming :: Fall 2019 (original) (raw)

Stanford CS149, Fall 2019

PARALLEL COMPUTING

This page contains lecture slides and recommended readings for the Fall 2019 offering of CS149. Lecture videos are available via SCPD.

(forms of parallelism: multicore, SIMD, threading + understanding latency and bandwidth)

Further Reading:

• CPU DB: Recording Microprocessor History. A. Danowitz, K. Kelley, J. Mao, J.P. Stevenson, M. Horowitz, ACM Queue 2005. (You can also take a peak at the CPU DB website)
• The Compute Architecture of Intel Processor Graphics. Intel Technical Report, 2015 (a very nice description of a modern throughput processor)
• Intel's Haswell CPU Microarchitecture. D. Kanter, 2013 (realworldtech.com article)
• NVIDIA GV100 (Volta) Whitepaper. NVIDIA Technical Report 2017

(ways of thinking about parallel programs, and their corresponding hardware implementations)

(the thought process of parallelizing a program)

(message passing, async vs. blocking sends/receives, pipelining, increasing arithmetic intensity, avoiding contention)

(map, reduce, fold, scan, gather/scatter. Parallel implementations of scan. Data-parallel algorithm design.)