1 Motivation and History 2 Parallel Architectures 3 Parallel Algorithm Design 4 Message-Passing Programming 5 The Sieve of Eratosthenes 6 Floyd's Algorithm 7 Performance Analysis 8 Matrix-Vector Multiplication 9 Document Classification 10 Monte Carlo Methods 11 Matrix Multiplication 12 Solving Linear Systems 13 Finite Difference Methods 14 Sorting 15 The Fast Fourier Transform 16 Combinatorial Search 17 Shared-memory Programming 18 Combining MPI and OpenMP Appendix A MPI Functions Appendix B Utility Functions Appendix C Debugging MPI Programs Appendix D Review of Complex Numbers Appendix E OpenMP Functions