SE 292: High Performance Computing (3:0)

Overview

This is is an introductory course in understanding high performance computing (HPC) organization and the basics of parallel computing. The course is divided into two logical but related parts. (1) Introduction to Computer Systems discusses how a single computer is organized from a programmer's perspective, and the performance implications when developing computational applications; and (2) Parallel computing introduces parallel architectures, and how parallel applications can be designed on such systems. Specific topics that will be covered are:

• Introduction to Computer Systems: Processors, Memory, I/O Devices; Cost, timing and scale (size) models; Program Execution: Process; Virtual Memory; System Calls; Dynamic Memory Allocation. Machine level view of a program; typical RISC instruction set and execution; Pipelining. Performance issues and Techniques; Cost and frequency models for I/O, paging and caching. Temporal and spatial locality. Typical compiler optimizations. Identifying program bottlenecks - profiling, tracing. Simple high level language optimizations - locality enhancement, memory disambiguation. Choosing appropriate computing platforms: benchmarking, cost-performance issues.
• Parallel computing: Parallel Computing: Introduction to parallel architectures and interconnection networks, communication latencies. Program parallelization; task partitioning and mapping; data distribution; message passing; synchronization and deadlocks. Distributed memory programming using MPI/PVM. Shared memory parallel programming. Multithreading.

Govind will be teaching the first part of the course (8-10 weeks) while Yogesh will cover the second part (6-8 weeks).

Intended Learning Objectives

At the end of the course, students should have learned the following concepts:

• Understand computer systems, such as processor architecture, memory subsystem and caching, from a programmer's viewpoint to maximize performance of his/her application.
• Understand underlying Operating Systems concepts from a programmer's viewpoint, including process, memory and file system management.
• Understand parallel processing systems, from a programmer's viewpoint, and develop parallel programs.

Collectively, these should help you design and develop high performance applications, and also improve applications' performance.

Teaching and Learning Activities

• Lectures: Lectures will form the primary teaching activity, the tentative schedule for which is outlined below. Lecture material will be posted online, and serve as the primary resource. The course text books will supplement the lecture slides. A mailing list is be available for discussions and queries outside the classroom, between students and with the faculty. All students should sign up on the list for official announcements.
• Homeworks: Homework assignments serve the dual purposes of forming a learning activity as well as a means for assessment. These count toward 25% of the total assessment score. Homework will include short answer questions, problem solving and some programming. The students are expected to understand and apply concepts learnt from class lectures and the text book, as well as use online resources, to complete these assignments.
• Exam: There will be three mid-term exams and a final exam for the course. These collectively count toward 75% of the total assessment score.

Assessment

The total assessment score for the course is based on a 1000 point scale. Of this, the weightage to different activities will be as follows:

Academic Integrity

Students must uphold IISc's Academic Integrity guidelines. While these are common sense, it is helpful to review them since failure to follow them will lead to sanctions and penalties. This includes a reduced or failing grade in the course. Severe cases of academic violations will be reported to the Institute and may lead to an expulsion.

Learning takes place both within and outside the class. Hence, discussions between students and reference to online material is encouraged as part of the course to achieve the intended learning objectives. However, while you may learn from any valid source, you must form your own ideas and complete problems and assignments by yourself. All works submitted by the student as part of their academic assessment must be their own. No group work is allowed unless explicitly stated in the assignment.

Plagiarism

Verbatim reproduction of material from external sources (web pages, books, papers, etc.) is not acceptable. If you are paraphrasing external content (or even your own prior work) or were otherwise influenced by them while completing your assignments, projects or exams, you must clearly acknowledge them. When in doubt, add a citation!

Cheating

While you may discuss lecture topics and broad outlines of homework problems with others, you cannot collaborate in completing the assignments, copy someone else's solution or falsify results. You cannot use notes or unauthorized resources during exams, or copy from others.

Classroom Behavior

Ensure that the course atmosphere, both in the class, outside and on the mailing list, is conducive for learning. Participate in discussions but do not dominate or be abusive. There are no "stupid" questions. Be considerate of your fellow students and avoid disruptive behavior.

Resources

Teaching & Office Hours

Tentative Schedule

Schedule is based on two 1.5 hour lectures on TTh 8AM-930AM each week.

Assignments

Homework

1. Assignment 1 has been posted. Due Date: Sep 25, 2014
2. Assignment 2 has been posted. Due Date: Oct 17, 2014
3. Assignment 3 with Pthreads source code has been posted. Due Date: 7AM Tue Nov 18, 2014
4. Assignment 4 on MPI programming has been posted. Due Date: Midnight Mon Dec 1, 2014

Acknowledgement

This course includes material prepared by Matthew Jacobs and Sathish Vadhiyar from SERC, IISc. Material used from other sources are acknowledged in the slides.