CS/SE 4F03 Distributed Computer Systems

Ned Nedialkov Department of Computing and Software McMaster University Winter 2014
Assignment 1 This is a good online book

Introduction

This course is devoted to studying distributed parallel architectures, parallel algorithms and programming. After covering basic concepts, like parallel programming platforms and models, we shall study parallel algorithm design and the Message Passing Interface (MPI). Our focus will be on designing and implementing ecient distributed parallel programs. We shall also consider tuning and debugging MPI programs.

Objectives

At the end of this course, students should have thorough understanding of the concepts behind distributed systems and programming. Students should understand the bottlenecks in distributed parallel computing and should know how to design and implement ecient distributed algorithms. In particular, students should be able to write non-trivial applications using MPI on distributed machines.

Room/time
Tuesday, Thursday, Friday 8:309:20 MDCL/1309 Tuesday 11:30-12:20 BSB/105 Monday 13:30-14:20 BSB/105 Friday 10:3011:30

Lectures: Tutorial: Oce hours

Teaching assistants

Thomas Gwosdz (email: gwosdzto) Jamie Turner (email: turnerjr)

Text

Peter Pacheco, An Introduction to Parallel Programming

Lectures
1. Introduction to MPI Code Code 2. Collective communications I 3. Caching Code

4. Interconnection networks

5. Communication cost 6. Scalability 7. Collective communications Code 8. Parallel program design. Tasks, critical path 9. Nonblocking communications 10. Understanding communications 11. Examples 12. Data decomposition techniques 13. Array distribution schemes 14. Communicators 15. Topologies Code Code Code Code

16. Communicators and topologies: Foxs Algorithm 17. The Traveling salesman problem. Parallel distributed tree search 18. Distributed shortest paths 19. Advanced point-to-point communications 20. OpenMP Code

MPI on CAS machines

We have 10 servers set up for MPI; all running RHEL 6.5. Hostnames are mpihost01 to mpihost10. mpihost01 is the head node. You should ssh to it. Then set up passwordless ssh: ssh-keygen -t rsa cp .ssh/id_rsa.pub .ssh/authorized_keys The machines and number of cores are # Intel Xeon E7-L8867 mpihost01 slots=16 2.13GHz

# Intel Xeon E5-2960 2.6GHz mpihost02 slots=4 mpihost03 slots=4 # AMD Opteron 8218 2.6GHz mpihost04 slots=4 # AMD Opteron 8356 2.3GHz mpihost05 slots=8 mpihost06 slots=8 # Intel Xeon X5482 3.2GHz mpihost07 slots=4 mpihost08 slots=4 mpihost09 slots=4 mpihost10 slots=4

Resources
MPI standard MPI: The Complete Reference MPI tutorial Performance topics Performance analysis tools

Grading
8% 12% 15% 20% 45%

Assignment 1 Assignment 2 Assignment 3 Midterm Project

10

Tentative Schedule
21 31 13 25 13 Jan 31 Jan Jan 13 Feb March Feb 11 March March 8 April

Assignment 1 Assignment 2 Midterm Assignment 3 Project

11

Course Policy

Course-related announcement will be at the course web site. You are responsible for checking it regularly. The assignments will be due at the beginning of the lectures. No assignments will be accepted after the end of the lecture on the date when they are due. Requests for remarking of an assignment or the midterm exam must be made within one week after the marked assignment/midterm is returned. Requests that are later than a week will not be accommodated. The assignments will be marked by the TAs. Any request for remarking must be rst directed to the TA that has marked your assignment problem(s). After you have talked to your TA, and you still believe that you deserve a higher mark, then you can contact me. You are allowed to discuss the problems from the assignments. However, you must submit your own work. Assignments that are very similar may lose half of their marks. Identical solutions to the same problem will receive zero marks.

12

Academic Dishonesty/Ethics

Students are reminded that they should read and comply with the Statement on Academic Ethics and the Senate Resolutions on Academic Dishonesty as found in the Senate Policy Statements distributed at registration and available in the Senate Oce. These policies are also available at http://www.mcmaster.ca/policy/ac_ethics.htm. 4

13

Faculty Notices

The Faculty of Engineering is concerned with ensuring an environment that is free of all discrimination. If there is a problem, individuals are reminded that they should contact the Department Chair, the Sexual Harassment Ocer or the Human Rights Consultant, as the problem occurs.