Parallel Programming (3:1)

The objective of this course is to give you some level of confidence in parallel programming techniques, algorithms and tools. At the end of the course, you would (we hope) be in a position to deal with real-life parallel  problems and to explore new avenues of research in the area of parallel programming.


Girish, Jitender, Sunil Bishoi, Rajul Biswas


Sathish Vadhiyar

Reading Materials

Additional Reading


    1. Introduction
      1. Motivations for parallel programming
      2. Challenges
      3. Metrics - speedup, utilization, efficiency, scalability
      4. Amdahl's law and Gustaffson's law
      5. Parallel architecture models , physical organization of parallel platforms
      6. Parallel programming paradigms, algorithm models
      7. Communication network topology in parallel machines
    2.  Programming using the Message passing paradigm MPI
      1. MPI basics
      2. Collective communication implementations
      3. MPI-2 overview
      4. An extended session on MPI I/O
    3. A Few Parallel Programs
      1. Vibrating string, Laplace equation, Molecular dynamics, WaTor
    1. Parallel Dense Linear algebra
      1. Gaussian elimination and others
    2. Parallel Sparse Linear algebra
      1. Sparse Linear Algebra - basics and sequential solutions
      2. parallel steps for ordering, symbolic factorization, numerical factorization, triangular solve
      3. Parallel Kernighan-Lin, Nested dissection
      4. sparse iterative methods
      5. parallel graph coloring
    3. Programming shared address space platforms
      1. OpenMP: A standard for directive based parallel programming
    4. Other Topics
      1. FFT
      2. Bioinformatics
      3. Scheduling
      4. Checkpointing

Mid-Term Exams


  1. Implementation of Collective communications

  2. Parallel Direct Solution of Linear Systems

  3. Sparse Matrix Triangular Solve