Introduction to Parallel Programming

Introduction to Parallel Programming. Instructor: Prof. Yogish Sabharwal, Department of Computer Science and Engineering, IIT Delhi. This course focuses on the shared memory programming paradigm. It covers concepts and programming principles involved in developing scalable parallel applications. Assignments focus on writing scalable programs for multi-core architectures using OpenMP and C. This is an introductory course in shared memory parallel programming suitable for computer science as well as non-computer science students working on parallel/HPC applications and interested in parallel programming. (from

Lecture 08 - Intuition for Shared and Distributed Memory Architectures

Go to the Course Home or watch other lectures:

Lecture 01 - Introduction
Lecture 02 - Parallel Architectures and Programming Models
Lecture 03 - Pipelining
Lecture 04 - Super-pipelining and VLIW
Lecture 05 - Memory Latency
Lecture 06 - Cache and Temporal Locality
Lecture 07 - Cache, Memory Bandwidth and Spatial Locality
Lecture 08 - Intuition for Shared and Distributed Memory Architectures
Lecture 09 - Shared and Distributed Memory Architectures
Lecture 10 - Interconnection Networks in Distributed Memory Architectures
Lecture 11 - OpenMP: A Parallel Hello World Program
Lecture 12 - Program with Single Thread
Lecture 13 - Program Memory with Multiple Threads and Multitasking
Lecture 14 - Context Switching
Lecture 15 - OpenMP: Basic Thread Functions
Lecture 16 - OpenMP: About OpenMP
Lecture 17 - Shared Memory Consistency Models and the Sequential Consistency Model
Lecture 18 - Race Conditions
Lecture 19 - OpenMP: Scoping Variables and Some Race Conditions
Lecture 20 - OpenMP: Thread Private Variables and More Constructs
Lecture 21 - Computing Sum: First Attempt at Parallelization
Lecture 22 - Manual Distribution of Work and Critical Sections
Lecture 23 - Distributing for Loops and Reduction
Lecture 24 - Vector-Vector Operations (Dot Product)
Lecture 25 - Matrix-Vector Operations (Matrix-Vector Multiply)
Lecture 26 - Matrix-Matrix Operations (Matrix-Matrix Multiply)
Lecture 27 - Introduction to Tasks
Lecture 28 - Task Queues and Task Execution
Lecture 29 - Accessing Variables in Tasks
Lecture 30 - Completion of Tasks and Scoping Variables in Tasks
Lecture 31 - Recursive Task Spawning and Pitfalls
Lecture 32 - Understanding LU Factorization
Lecture 33 - Parallel LU Factorization
Lecture 34 - Locks
Lecture 35 - Advanced Task Handling
Lecture 36 - Matrix Multiplication using Tasks
Lecture 37 - The OpenMP Shared Memory Consistency Model