CS/CMPE 524 HighPerformance Computing

Outline

Description (1)

Introduction to high-performance computing architectures and programming environments

Performance

enhancements for single-processor computers Data locality, cache optimizations, data dependences, loop optimizations Parallel computer architectures Parallel computing performance metrics Message-passing programming using MPI Shared-memory programming using OpenMP and P-threads Dense matrix algorithms Sorting algorithms

CS 524 - High-Performance Computing (Au 2005-2006) - Asim Karim @ LUMS

Description (2)

Emphasis on code optimizations and programming techniques

This

is not a hardware design course

Programming in C/C++ only Programming under Unix/Linux environment

CS 524 - High-Performance Computing (Au 2005-2006) - Asim Karim @ LUMS

Goals

Mastery of fundamental performance issues for parallel and serial computing Familiarity with shared-memory and message passing architectures and models for parallel program development Hands-on experience in designing and implementing efficient algorithms for high-performance computers Familiarity with the current state-of-art in parallel programming environments, portable software libraries, and program development

CS 524 - High-Performance Computing (Au 2005-2006) - Asim Karim @ LUMS

Before Taking This Course

You should be comfortable with Data structures and algorithms
CS213

is a prerequisite You should be comfortable with algorithm descriptions and implementations in a high-level programming language

Computer organization
CS223

is a prerequisite CS423 is NOT a prerequisite

Programming in C/C++
Majority of Matrix

work under Unix/Linux environment

Linear algebra
operations, systems of linear equations
5

CS 524 - High-Performance Computing (Au 2005-2006) - Asim Karim @ LUMS

After Taking this Course

You should be able to understand the basic architectures of single- and multiprocessor computers develop and implement programs for distributedmemory and shared-memory parallel computers evaluate and enhance performance on single- and multi-processor computers develop programs in MPI and OpenMP

CS 524 - High-Performance Computing (Au 2005-2006) - Asim Karim @ LUMS

Grading

Points distribution

Assignments (~ 6) Quizzes (~ 5) Midterm exam Final exam (comprehensive)

35% 10% 25% 30%

CS 524 - High-Performance Computing (Au 2005-2006) - Asim Karim @ LUMS

Policies (1)

Quizzes
Quizzes

can be announced 1 to 2 days in advance or unannounced

Sharing
No

copying is allowed for assignments. Discussions are encouraged; however, you must submit your own work Violators can face mark reduction and/or reported to Disciplinary Committee for action

Plagiarism
Do

NOT pass someone elses work as yours! Write in your words and cite the reference. This applies to code as well. Violators can face mark reduction and/or reported to Disciplinary Committee for action
CS 524 - High-Performance Computing (Au 2005-2006) - Asim Karim @ LUMS 8

Policies (2)

Submission policy
Submissions

are due at the day and time specified Late penalties: 1 day = 10%; 2 day late = 20%; not accepted after 2 days An extension will be granted only if there is a need and when requested several days in advance.

Classroom behavior
Maintain classroom sanctity by

remaining quiet and attentive If you have a need to talk and gossip, please leave the classroom so as not to disturb others Dozing is allowed provided you do not snore loud

CS 524 - High-Performance Computing (Au 2005-2006) - Asim Karim @ LUMS

Summarized Course Contents

Introduction to single- and multi-processor computing models and architectures Cache tuning and characterization Data locality, data dependences, loop optimization Parallel computer performance metrics, interconnection networks Message-passage programming in MPI Shared-memory programming in OpenMP and PThreads Parallel dense matrix and sorting algorithms
10

CS 524 - High-Performance Computing (Au 2005-2006) - Asim Karim @ LUMS

Course Material

Text

Introduction to Parallel Computing, A. Grama, A. Gupta, G. Karypis, and V. Kumar, 2nd edition, Addison Wesley, 2003.

Reference texts

Software Optimization for High Performance Computing, K.R. Wadleigh and I.L. Crawford, Prentice Hall, 2000. The Sourcebook of Parallel Computing, W. Gropp et al., Mrogan Kaufmann, 2003. Using MPI Portable Parallel Programming with the Message-Passing Interface, 2nd Ed., W. Gropp, E. Lusk, A. Skjellum, MIT Press, 1999. Parallel Programming in OpenMP, R. Chandra et al., Academic Press, 2001.
CS 524 - High-Performance Computing (Au 2005-2006) - Asim Karim @ LUMS 11

Course Web Site

For announcements, lecture slides, handouts, assignments, solutions, web resources: http://suraj.lums.edu.pk/~cs524a05/

The resource page has links to information available on the Web. It is basically a meta-list for finding further information.

CS 524 - High-Performance Computing (Au 2005-2006) - Asim Karim @ LUMS

12

Other Stuff

How to contact me?

Office

hours: 11.40 to 13.00 TR (office: 429) E-mail: akarim@lums.edu.pk Please see me during my office hours only. If you need to see me at another time, send me an e-mail for an appointment.

Philosophy
Knowledge

cannot be taught; it is learned. Be excited. That is the best way to learn. I cannot teach everything in class. Develop an inquisitive mind, ask questions, and go beyond what is required. I dont believe in strict grading. But there has to be a way of rewarding performance.
CS 524 - High-Performance Computing (Au 2005-2006) - Asim Karim @ LUMS 13

Reference Books in LUMS Library

High Performance Cluster Computing: Programming and Applications, Volume 2, Rajkumar Buyya (Editor), 1999. Parallel Programming Techniques and Applications Using Networked Workstations and Parallel Computers, B. Wilkinson and M. Allen, 1999, 5.3W686P. Introduction to Parallel Processing, Sasikumar et al., Prentice Hall of India, 2000. Parallel Computer Architecture: A Hardware/Software Approach, D.E. Culler et al., 1999. Computer Organization, Hamacher et al., 2002. Computer Organization and Architectural Design for Performance, W. Stalling, 2000. Using MPI-2: Advanced Features of the Message Passing Interface, Gropp et al., 1999. High Performance Linux Clusters: With OSCAR, Rocks, OpenMosix, and MPI, J.D. Sloan, Sebastopol, 2005. How to Build a Beowulf, T.L. Sterling et al., 1999. Beowulf Cluster Computing With Linux/Windows, T.L. Sterling (ed.), 2002, 5.43 B481. Parallel I/O for High Performance Computing, J.M. May, 2001.

CS 524 - High-Performance Computing (Au 2005-2006) - Asim Karim @ LUMS

14