Title:

Practical Parallel Programming

Code:PPP
Ac.Year:ukončen 2002/2003
Term:Summer
Study plans:
ProgramBranchYearDuty
EI-BC-3VTB2nd Stage/1st YearElective
EI-MSC-3VTN2ndElective
EI-MSC-5VTI2nd Stage/2nd YearElective
Language:Czech
Public info:http://www.fit.vutbr.cz/study/courses/PPP/public/
Credits:6
Completion:examination (written)
Type of
instruction:
Hour/semLecturesSem. ExercisesLab. exercisesComp. exercisesOther
Hours:3900260
 ExaminationTestsExercisesLaboratoriesOther
Points:60120028
Guarantee:Dvořák Václav, prof. Ing., DrSc., DCSY
Lecturer:Dvořák Václav, prof. Ing., DrSc., DCSY
Faculty:Faculty of Information Technology BUT
Prerequisites: 
Operating Systems 1 (OS1), FIT
Follow-ups:
Parallel and Distributed Algorithms (PDA), FIT
 
Learning objectives:
To get familiar with programming of various parallel systems, especially symmetrical multiprocessors and clusters of workstations. Learn how to write portable programs using standardized interfaces and languages, specify parallelism and process communication. Be able to predict performance of parallel applications.
Description:
Architectures and components of parallel systems. Programming models, concurrent processes, threads, communication. Design of parallel programs. Message passing: channel communication, routing algorithms, pipeline and farm model, prototyping parallel applications. Shared memory: exclussive access, support in programming languages, thread synchronization. Data parallelism: languages, vector and matrix operations, application examples. Case studies of parallel applications.
Learning outcomes and competences:
Survey of parallelization techniques of fundamental scientific problems, knowledge of parallel programming in MPI and OpenMP.
Syllabus of lectures:
  1. Parallel systems architectures, programming models, performance.
  2. Parallel processes, threads and communications, efficiency and overhead of parallel processing.
  3. Message passing programming, prototyping algorithms in TRANSIM.
  4. Programming in MPI, PVM, cluster computations.
  5. Designing parallel programs, pipeline computations, dynamic load balancing (farms).
  6. Shared variable programming, atomicity, exclusive access, synchronization constructs.
  7. Programming in OpenMP.
  8. Partitioning, work-pool and divide and conquer strategies.
  9. Data parallelism: languages (HPF), operations on vectors and matrices, application examples.
  10. Case studies of parallel applications - parallel sorting.
  11. Parallel FFT and image processing on different parallel platforms
  12. Parallel linear algebra, solution of large systems of linear equations.
  13. Searching and discrete optimization, parallel GA and ANN.
Syllabus of laboratory exercises:
  1. Parallel search of a global minimum.
  2. Parallel sorting, performance prediction (Transim).
  3. A solution of a large system of linear equations (MPI).
  4. Parallel FFT on a SMP (OpenMP, simulation).
Fundamental literature:
  1. Wilkinson, B. - Allen, M.: Parallel Programming: Techniques and Applications Using Networked Workstations and Parallel Computers. Prentice Hall, 1999, ISBN 0-13-671710-1.
  2. Andrews, G.R.: Foundations of Multithreaded, Parallel, and Distributed Programming. Addison Wesley, 2000, ISBN 0-201-35752-6.
Study literature:
  1. Dvořák, V.: Architecture and programming of parallel systems. Lecture Notes FIT VUT v Brně, a manuscript under preparation.
  2. 300+ Powerpoint frames available to students.
Progress assessment:
Assessment of four small projects, a midterm examination.