Title:

Practical Parallel Programming

Code:PPP
Ac.Year:2019/2020
Sem:Summer
Curriculums:
ProgrammeField/
Specialization
YearDuty
IT-MSC-2MBI-Compulsory-Elective - group C
IT-MSC-2MBS-Elective
IT-MSC-2MGM-Compulsory-Elective - group C
IT-MSC-2MIN-Elective
IT-MSC-2MIS-Elective
IT-MSC-2MMM-Elective
IT-MSC-2MPV1stCompulsory
IT-MSC-2MSK1stCompulsory
MITAINADE-Elective
MITAINBIO-Compulsory
MITAINCPS-Elective
MITAINEMB2ndCompulsory
MITAINGRI-Elective
MITAINHPC1stCompulsory
MITAINIDE-Elective
MITAINISD-Elective
MITAINISY-Elective
MITAINMAL-Elective
MITAINMAT-Elective
MITAINNET-Elective
MITAINSEC-Elective
MITAINSEN-Elective
MITAINSPE-Elective
MITAINVER-Elective
MITAINVIZ-Elective
Language of Instruction:Czech
Public info:http://www.fit.vutbr.cz/study/courses/PPP/public/
Credits:5
Completion:credit+exam (written)
Type of
instruction:
Hour/semLecturesSeminar
Exercises
Laboratory
Exercises
Computer
Exercises
Other
Hours:26001610
 ExamsTestsExercisesLaboratoriesOther
Points:60200020
Guarantor:Jaroš Jiří, doc. Ing., Ph.D. (DCSY)
Deputy guarantor:Kořenek Jan, doc. Ing., Ph.D. (DCSY)
Lecturer:Jaroš Jiří, doc. Ing., Ph.D. (DCSY)
Instructor:Kadlubiak Kristián, Ing. (DCSY)
Kukliš Filip, Ing. (DCSY)
Faculty:Faculty of Information Technology BUT
Department:Department of Computer Systems FIT BUT
Schedule:
DayLessonWeekRoomStartEndLect.Gr.Groups
MonlecturelecturesD0207 08:0009:501MIT 2MIT MPV - MSK xx
Moncomp.lablecturesO204 10:0011:501MIT 2MIT xx
Moncomp.lablecturesO204 15:0016:501MIT 2MIT xx
Wedcomp.lablecturesO204 08:0009:501MIT 2MIT xx
Fricomp.lablecturesO204 08:0009:501MIT 2MIT xx
Fricomp.lablecturesO204 10:0011:501MIT 2MIT xx
 
Learning objectives:
  To get familiar with the architecture of distributed supercomputing systems, their interconnection networks and storage. To orientate oneself in parallel systems on the market, be able to assess communication and computing possibilities of a particular architecture and to predict the performance of parallel applications. Learn how to write portable programs using standardized interfaces and languages, specify parallelism and process communication. To learn how to practically use supercoputer for solving complex engineering problems.
Description:
  The course covers architecture and programming of parallel systems with functional and data parallelism. First, the parallel system theory and program parallelization are discussed. The detailed description of most proliferated supercomputing systems, interconnection network typologies and routing algorithms is followed by the architecture of parallel and distributed storage systems. The course goes on in message passing programming in standardized interface MPI. Consequently, techniques for parallel debugging and profiling are discussed. Last part of the course is devoted to the description of parallel programming patterns and case studies from the are of linear algebra, physical systems described by partial differential equations, N-Body systems and Monte-Carlo methods. 
Knowledge and skills required for the course:
  Von-Neumann computer architecture, computer memory hierarchy, cache memories and their organization, programming in assembly and in C/C++. Knowledge gained in courses PRL and AVS.
Subject specific learning outcomes and competencies:
  Overview of principles of current parallel system design and of interconnection networks, communication techniques and algorithms. Survey of parallelization techniques of fundamental scientific problems, knowledge of parallel programming in MPI. Knowledge of basic parallel programming patterns. Practical experience with the work on supercomputers, ability to identify performance issues and propose their solution.
Generic learning outcomes and competencies:
  Knowledge of capabilities and limitations of parallel processing, ability to estimate performance of parallel applications. Language means for process/thread communication and synchronization. Competence in hardware-software platforms for high-performance computing and simulations.
Why is the course taught:
  This course will take you into the area of high performance computing where a single computer is far from being powerful enough to satisfy application demands. The only solution in such cases is to distribute computation across a supercomputing cluster. This course first examines the architecture of top machines, and then focuses on their software equipment. We will learn the MPI library which is an industry standard in high performance computing. Finally, we will introduce a few typical applications ranging such as physical simulation of heat distribution, fluid dynamics, n-body gravitational and Coulomb systems of galaxies and molecules or Monte-Carlo methods. 
Syllabus of lectures:
 
  1. Introduction to parallel processing.
  2. Architectures with distributed memory,
  3. Interconnection networks: topology and routing algorithms, switching, flow control.
  4. Technologies of interconnection networks (Infiniband).
  5. Distributed file systems (Lustre, HPFS).
  6. Message passing interface, pair-wise communications, data types
  7. Collective communications and communicators,
  8. Hybrid programming OpenMP/MPI and one-sided communications.
  9. Parallel code debugging, profiling and tracing. 
  10. Programming patterns for parallel programming.
  11. Case studies: matrix calculations, linear equation systems 
  12. Case studies: solution of PDE systems, finite difference, spectral methods
  13. Case studies: Fluid dynamics, N-Body systems, Monte-Carlo.
Syllabus of computer exercises:
 
  1. MPI: Point-to-point communications
  2. MPI: Collective communications
  3. MPI: Communicators
  4. MPI: Data types, reduction
  5. MPI: Parallel input and output
  6. Profiling and tracing of parallel applications
  7. Matrix calculations.
  8. Finite difference methods.
Syllabus - others, projects and individual work of students:
 
  • A parallel program in MPI on the supercomputer.
Fundamental literature:
 
  • Pacecho, P.: Introduction to Parallel Programming. Morgan Kaufman Publishers, 2011, 392 s., ISBN: 9780123742605 
  • Hennessy, J.L., Patterson, D.A.: Computer Architecture - A Quantitative Approach. 5. vydání, Morgan Kaufman Publishers, Inc., 2012, 856 s., ISBN: 9780123838728
Study literature:
 
Controlled instruction:
  
  • Missed labs can be substituted in alternative dates (monday or friday)
  • There will be a place for missed labs in the last week of the semester.
Progress assessment:
  Assessment of a project, 10 hours in total and a midterm examination.
Exam prerequisites:
  To get 20 out of 40 points for projects and midterm examination.
 

Your IPv4 address: 54.211.135.32