Tutorials

|
E
x
e
r
c
i
s
e
s
|
A
b
s
t
r
a
c
t
s
|
L
C

W
o
r
k
s
h
o
p
s
|
C
o
m
m
e
n
t
s
|
S
e
a
r
c
h
|
P
r
i
v
a
c
y

&

L
e
g
a
l
N
o
t
i
c
e
Tutorials | Exercises | Abstracts | LC Workshops | Comments | Search | Privacy & Legal Notice
Message Passing Interface (MPI)
Blaise Barney, Lawrence Livermore National Laboratory
UCRL-MI-133316
Table of Contents
1. Abstract
2. What is MPI?
3. Getting Started
. !n"ir#n$ent Manage$ent R#%tines
&. P#int t# P#int C#$$%nicati#n R#%tines
1. Genera' C#nce(ts
2. MPI Message Passing R#%tine Arg%$ents
3. )'#c*ing Message Passing R#%tines
4. +#n-)'#c*ing Message Passing R#%tines
6. C#''ecti"e C#$$%nicati#n R#%tines
,. -eri"ed -ata ./(es
0. Gr#%( and C#$$%nicat#r Manage$ent R#%tines
1. 2irt%a' .#(#'#gies
13. A )rie4 W#rd #n MPI-2
11. LL+L S(eci4ic In4#r$ati#n and Rec#$$endati#ns
12. Re4erences and M#re In4#r$ati#n
13. A((endi5 A6 MPI-1 R#%tine Inde5
1. !5ercise
Abstract
.he Message Passing Inter4ace Standard 7MPI8 is a $essage (assing 'ibrar/ standard based #n
the c#nsens%s #4 the MPI 9#r%$: ;hich has #"er 3 (artici(ating #rgani<ati#ns: inc'%ding
"end#rs: researchers: s#4t;are 'ibrar/ de"e'#(ers: and %sers. .he g#a' #4 the Message Passing
Inter4ace is t# estab'ish a (#rtab'e: e44icient: and 4'e5ib'e standard 4#r $essage (assing that ;i''
be ;ide'/ %sed 4#r ;riting $essage (assing (r#gra$s. As s%ch: MPI is the 4irst standardi<ed:
"end#r inde(endent: $essage (assing 'ibrar/. .he ad"antages #4 de"e'#(ing $essage (assing
s#4t;are %sing MPI c'#se'/ $atch the design g#a's #4 (#rtabi'it/: e44icienc/: and 4'e5ibi'it/. MPI
is n#t an I!!! #r IS= standard: b%t has in 4act: bec#$e the >ind%str/ standard> 4#r ;riting
$essage (assing (r#gra$s #n ?PC ('at4#r$s.
.he g#a' #4 this t%t#ria' is t# teach th#se %n4a$i'iar ;ith MPI h#; t# de"e'#( and r%n (ara''e'
(r#gra$s acc#rding t# the MPI standard. .he (ri$ar/ t#(ics that are (resented 4#c%s #n th#se
;hich are the $#st %se4%' 4#r ne; MPI (r#gra$$ers. .he t%t#ria' begins ;ith an intr#d%cti#n:
bac*gr#%nd: and basic in4#r$ati#n 4#r getting started ;ith MPI. .his is 4#''#;ed b/ a detai'ed
'##* at the MPI r#%tines that are $#st %se4%' 4#r ne; MPI (r#gra$$ers: inc'%ding MPI
!n"ir#n$ent Manage$ent: P#int-t#-P#int C#$$%nicati#ns: and C#''ecti"e C#$$%nicati#ns
r#%tines. +%$er#%s e5a$('es in b#th C and 9#rtran are (r#"ided: as ;e'' as a 'ab e5ercise.
.he t%t#ria' $ateria's a's# inc'%de $#re ad"anced t#(ics s%ch as -eri"ed -ata ./(es: Gr#%( and
C#$$%nicat#r Manage$ent R#%tines: and 2irt%a' .#(#'#gies. ?#;e"er: these are n#t act%a''/
(resented d%ring the 'ect%re: b%t are $eant t# ser"e as >4%rther reading> 4#r th#se ;h# are
interested.
Le"e'@PrereA%isites6 Idea' 4#r th#se ;h# are ne; t# (ara''e' (r#gra$$ing ;ith MPI. A basic
%nderstanding #4 (ara''e' (r#gra$$ing in C #r 9#rtran is ass%$ed. 9#r th#se ;h# are %n4a$i'iar
;ith Para''e' Pr#gra$$ing in genera': the $ateria' c#"ered in !C3&336 Intr#d%cti#n .# Para''e'
C#$(%ting ;#%'d be he'(4%'.
What is MPI?
An Inter4ace S(eci4icati#n6
M P I B Message Passing Inter4ace
MPI is a s(eci4icati#n 4#r the de"e'#(ers and %sers #4 $essage (assing 'ibraries. )/ itse'4:
it is +=. a 'ibrar/ - b%t rather the s(eci4icati#n #4 ;hat s%ch a 'ibrar/ sh#%'d be.
Si$('/ stated: the g#a' #4 the Message Passing Inter4ace is t# (r#"ide a ;ide'/ %sed
standard 4#r ;riting $essage (assing (r#gra$s. .he inter4ace atte$(ts t# be
o (ractica'
o (#rtab'e
o e44icient
o 4'e5ib'e
Inter4ace s(eci4icati#ns ha"e been de4ined 4#r C@CCC and 9#rtran (r#gra$s.
?ist#r/ and !"#'%ti#n6
MPI res%'ted 4r#$ the e44#rts #4 n%$er#%s indi"id%a's and gr#%(s #"er the c#%rse #4 a 2
/ear (eri#d bet;een 1112 and 111. S#$e hist#r/6
1103s - ear'/ 1113s6 -istrib%ted $e$#r/: (ara''e' c#$(%ting de"e'#(s: as d# a n%$ber #4
inc#$(atib'e s#4t;are t##'s 4#r ;riting s%ch (r#gra$s - %s%a''/ ;ith trade#44s bet;een
(#rtabi'it/: (er4#r$ance: 4%ncti#na'it/ and (rice. Rec#gniti#n #4 the need 4#r a standard
ar#se.
A(ri': 11126 W#r*sh#( #n Standards 4#r Message Passing in a -istrib%ted Me$#r/
!n"ir#n$ent: s(#ns#red b/ the Center 4#r Research #n Para''e' C#$(%ting:
Wi''ia$sb%rg: 2irginia. .he basic 4eat%res essentia' t# a standard $essage (assing
inter4ace ;ere disc%ssed: and a ;#r*ing gr#%( estab'ished t# c#ntin%e the standardi<ati#n
(r#cess. Pre'i$inar/ dra4t (r#(#sa' de"e'#(ed s%bseA%ent'/.
+#"e$ber 11126 - W#r*ing gr#%( $eets in Minnea(#'is. MPI dra4t (r#(#sa' 7MPI18 4r#$
=R+L (resented. Gr#%( ad#(ts (r#ced%res and #rgani<ati#n t# 4#r$ the MPI 9#r%$.
MPI9 e"ent%a''/ c#$(rised #4 ab#%t 1,& indi"id%a's 4r#$ 3 #rgani<ati#ns inc'%ding
(ara''e' c#$(%ter "end#rs: s#4t;are ;riters: acade$ia and a(('icati#n scientists.
+#"e$ber 11136 S%(erc#$(%ting 13 c#n4erence - dra4t MPI standard (resented.
9ina' "ersi#n #4 dra4t re'eased in Ma/: 111 - a"ai'ab'e #n the at6 htt(6@@;;;-
%ni5.$cs.an'.g#"@$(i.
MPI-2 (ic*ed %( ;here the 4irst MPI s(eci4icati#n 'e4t #44: and addressed t#(ics ;hich g#
be/#nd the 4irst MPI s(eci4icati#n. .he #rigina' MPI then beca$e *n#;n as MPI-1. MPI-
2 is brie4'/ c#"ered 'ater. Was 4ina'i<ed in 1116.
.#da/: MPI i$('e$entati#ns are a c#$binati#n #4 MPI-1 and MPI-2. A 4e;
i$('e$entati#ns inc'%de the 4%'' 4%ncti#na'it/ #4 b#th.
Reas#ns 4#r Using MPI6
Standardization - MPI is the #n'/ $essage (assing 'ibrar/ ;hich can be c#nsidered a
standard. It is s%((#rted #n "irt%a''/ a'' ?PC ('at4#r$s. Practica''/: it has re('aced a''
(re"i#%s $essage (assing 'ibraries.
Portability - .here is n# need t# $#di4/ /#%r s#%rce c#de ;hen /#% (#rt /#%r
a(('icati#n t# a di44erent ('at4#r$ that s%((#rts 7and is c#$('iant ;ith8 the MPI standard.
Performance Opportunities - 2end#r i$('e$entati#ns sh#%'d be ab'e t# e5('#it nati"e
hard;are 4eat%res t# #(ti$i<e (er4#r$ance. 9#r $#re in4#r$ati#n ab#%t MPI
(er4#r$ance see the MPI Per4#r$ance .#(ics t%t#ria'.
Functionality - ="er 11& r#%tines are de4ined in MPI-1 a'#ne.
!ailability - A "ariet/ #4 i$('e$entati#ns are a"ai'ab'e: b#th "end#r and (%b'ic d#$ain.
Pr#gra$$ing M#de'6
MPI 'ends itse'4 t# "irt%a''/ an/ distrib%ted $e$#r/ (ara''e' (r#gra$$ing $#de'. In
additi#n: MPI is c#$$#n'/ %sed t# i$('e$ent 7behind the scenes8 s#$e shared $e$#r/
$#de's: s%ch as -ata Para''e': #n distrib%ted $e$#r/ architect%res.
?ard;are ('at4#r$s6
o -istrib%ted Me$#r/6 =rigina''/: MPI ;as targeted 4#r distrib%ted $e$#r/
s/ste$s.
o Shared Me$#r/6 As shared $e$#r/ s/ste$s beca$e $#re (#(%'ar: (artic%'ar'/
SMP @ +UMA architect%res: MPI i$('e$entati#ns 4#r these ('at4#r$s a((eared.
o ?/brid6 MPI is n#; %sed #n D%st ab#%t an/ c#$$#n (ara''e' architect%re
inc'%ding $assi"e'/ (ara''e' $achines: SMP c'%sters: ;#r*stati#n c'%sters and
heter#gene#%s net;#r*s.
A'' (ara''e'is$ is e5('icit6 the (r#gra$$er is res(#nsib'e 4#r c#rrect'/ identi4/ing
(ara''e'is$ and i$('e$enting (ara''e' a'g#rith$s %sing MPI c#nstr%cts.
.he n%$ber #4 tas*s dedicated t# r%n a (ara''e' (r#gra$ is static. +e; tas*s can n#t be
d/na$ica''/ s(a;ned d%ring r%n ti$e. 7MPI-2 addresses this iss%e8.
Getting Started
?eader 9i'e6
ReA%ired 4#r a'' (r#gra$s that $a*e MPI 'ibrar/ ca''s.
C include file Fortran include file
#include "mpi.h" include 'mpif.h'
9#r$at #4 MPI Ca''s6
C na$es are case sensiti"eE 9#rtran na$es are n#t.
C "inding
Format#
rc = MPI_Xxxxx(parameter, ... )
$%ample#
rc = MPI_Bsend(!uf,c"unt,t#pe,dest,ta$,c"mm)
$rror code# Ret%rned as >rc>. MPIFSUCC!SS i4 s%ccess4%'
Fortran "inding
Format#
%&'' MPI_XXXXX(parameter,..., ierr)
call mpi_xxxxx(parameter,..., ierr)
$%ample#
%&'' MPI_B()*+(!uf,c"unt,t#pe,dest,ta$,c"mm,ierr)
$rror code# Ret%rned as >ierr> (ara$eter. MPIFSUCC!SS i4 s%ccess4%'
Genera' MPI Pr#gra$ Str%ct%re6
C#$$%nicat#rs and Gr#%(s6
MPI %ses #bDects ca''ed c#$$%nicat#rs and gr#%(s t# de4ine ;hich c#''ecti#n #4
(r#cesses $a/ c#$$%nicate ;ith each #ther.
M#st MPI r#%tines reA%ire /#% t# s(eci4/ a c#$$%nicat#r as an arg%$ent.
C#$$%nicat#rs and gr#%(s ;i'' be c#"ered in $#re detai' 'ater. 9#r n#;: si$('/ %se
MPI&COMM&'O()* ;hene"er a c#$$%nicat#r is reA%ired - it is the (rede4ined
c#$$%nicat#r that inc'%des a'' #4 /#%r MPI (r#cesses.
Ran*6
Within a c#$$%nicat#r: e"er/ (r#cess has its #;n %niA%e: integer identi4ier assigned b/
the s/ste$ ;hen the (r#cess initia'i<es. A ran* is s#$eti$es a's# ca''ed a >tas* I->.
Ran*s are c#ntig%#%s and begin at <er#.
Used b/ the (r#gra$$er t# s(eci4/ the s#%rce and destinati#n #4 $essages. =4ten %sed
c#nditi#na''/ b/ the a(('icati#n t# c#ntr#' (r#gra$ e5ec%ti#n 7i4 ran*B3 d# this @ i4
ran*B1 d# that8.
!rr#r ?and'ing6
M#st MPI r#%tines inc'%de a ret%rn@err#r c#de (ara$eter: as described in the >9#r$at #4
MPI Ca''s> secti#n ab#"e.
?#;e"er: acc#rding t# the MPI standard: the de4a%'t beha"i#r #4 an MPI ca'' is t# ab#rt i4
there is an err#r. .his $eans /#% ;i'' (r#bab'/ n#t be ab'e t# ca(t%re a ret%rn@err#r c#de
#ther than MPIFSUCC!SS 7<er#8.
.he standard d#es (r#"ide a $eans t# #"erride this de4a%'t err#r hand'er. A disc%ssi#n #n
h#; t# d# this is a"ai'ab'e ?!R!. G#% can a's# c#ns%'t the err#r hand'ing secti#n #4 the
MPI Standard '#cated at htt(6@@;;;.$(i-4#r%$.#rg@d#cs@$(i-11-ht$'@n#de10.ht$'.
.he t/(es #4 err#rs dis('a/ed t# the %ser are i$('e$entati#n de(endent.
!n"ir#n$ent Manage$ent R#%tines
MPI en"ir#n$ent $anage$ent r#%tines are %sed 4#r an ass#rt$ent #4 (%r(#ses: s%ch as
initia'i<ing and ter$inating the MPI en"ir#n$ent: A%er/ing the en"ir#n$ent and identit/: etc.
M#st #4 the c#$$#n'/ %sed #nes are described be'#;.
MPI&Init
Initia'i<es the MPI e5ec%ti#n en"ir#n$ent. .his 4%ncti#n $%st be ca''ed in e"er/ MPI
(r#gra$: $%st be ca''ed be4#re an/ #ther MPI 4%ncti#ns and $%st be ca''ed #n'/ #nce in
an MPI (r#gra$. 9#r C (r#gra$s: MPIFInit $a/ be %sed t# (ass the c#$$and 'ine
arg%$ents t# a'' (r#cesses: a'th#%gh this is n#t reA%ired b/ the standard and is
i$('e$entati#n de(endent.
MPI_Init (ar$c,ar$,)
MPI_I*I- (ierr)
MPI&Comm&size
-eter$ines the n%$ber #4 (r#cesses in the gr#%( ass#ciated ;ith a c#$$%nicat#r.
Genera''/ %sed ;ithin the c#$$%nicat#r MPIFC=MMFW=RL- t# deter$ine the
n%$ber #4 (r#cesses being %sed b/ /#%r a(('icati#n.
MPI_%"mm_si.e (c"mm,si.e)
MPI_%/MM_(I0) (c"mm,si.e,ierr)
MPI&Comm&ran+
-eter$ines the ran* #4 the ca''ing (r#cess ;ithin the c#$$%nicat#r. Initia''/: each
(r#cess ;i'' be assigned a %niA%e integer ran* bet;een 3 and n%$ber #4 (r#cess#rs - 1
;ithin the c#$$%nicat#r MPIFC=MMFW=RL-. .his ran* is #4ten re4erred t# as a tas*
I-. I4 a (r#cess bec#$es ass#ciated ;ith #ther c#$$%nicat#rs: it ;i'' ha"e a %niA%e ran*
;ithin each #4 these as ;e''.
MPI_%"mm_ran1 (c"mm,ran1)
MPI_%/MM_2&*3 (c"mm,ran1,ierr)
MPI&bort
.er$inates a'' MPI (r#cesses ass#ciated ;ith the c#$$%nicat#r. In $#st MPI
i$('e$entati#ns it ter$inates ALL (r#cesses regard'ess #4 the c#$$%nicat#r s(eci4ied.
MPI_&!"rt (c"mm,err"rc"de)
MPI_&B/2- (c"mm,err"rc"de,ierr)
MPI&,et&processor&name
Ret%rns the (r#cess#r na$e. A's# ret%rns the 'ength #4 the na$e. .he b%44er 4#r >na$e>
$%st be at 'east MPIFMAHFPR=C!SS=RF+AM! characters in si<e. What is ret%rned
int# >na$e> is i$('e$entati#n de(endent - $a/ n#t be the sa$e as the #%t(%t #4 the
>h#stna$e> #r >h#st> she'' c#$$ands.
MPI_4et_pr"cess"r_name (name,resultlen$th)
MPI_4)-_P2/%)((/2_*&M) (name,resultlen$th,ierr)
MPI&Initialized
Indicates ;hether MPIFInit has been ca''ed - ret%rns 4'ag as either '#gica' tr%e 718 #r
4a'se738. MPI reA%ires that MPIFInit be ca''ed #nce and #n'/ #nce b/ each (r#cess. .his
$a/ (#se a (r#b'e$ 4#r $#d%'es that ;ant t# %se MPI and are (re(ared t# ca'' MPIFInit
i4 necessar/. MPIFInitia'i<ed s#'"es this (r#b'e$.
MPI_Initiali.ed (fla$)
MPI_I*I-I&'I0)+ (fla$,ierr)
MPI&'time
Ret%rns an e'a(sed ;a'' c'#c* ti$e in sec#nds 7d#%b'e (recisi#n8 #n the ca''ing (r#cess#r.
MPI_5time ()
MPI_5-IM) ()
MPI&'tic+
Ret%rns the res#'%ti#n in sec#nds 7d#%b'e (recisi#n8 #4 MPIFWti$e.
MPI_5tic1 ()
MPI_5-I%3 ()
MPI&Finalize
.er$inates the MPI e5ec%ti#n en"ir#n$ent. .his 4%ncti#n sh#%'d be the 'ast MPI r#%tine
ca''ed in e"er/ MPI (r#gra$ - n# #ther MPI r#%tines $a/ be ca''ed a4ter it.
MPI_6inali.e ()
MPI_6I*&'I0) (ierr)
$%amples# $n!ironment Management (outines
C Lang%age - !n"ir#n$ent Manage$ent R#%tines !5a$('e
#include "mpi.h"
#include <stdio.h>
int main(argc,argv)
int argc;
char *argv[]; {
int numtass, ran, rc;
rc ! "#$%$nit(&argc,&argv);
i' (rc (! "#$%)*++,))) {
print' (",rror starting "#$ program. -erminating..n");
"#$%/0ort("#$%+1""%21345, rc);
6
"#$%+omm%si7e("#$%+1""%21345,&numtass);
"#$%+omm%ran("#$%+1""%21345,&ran);
print' ("8um0er o' tass! 9d ": ran! 9d.n", numtass,ran);
;******* do some <or *******;
"#$%=inali7e();
6
9#rtran - !n"ir#n$ent Manage$ent R#%tines !5a$('e
program simple
include >mpi'.h>
integer numtass, ran, ierr, rc
call "#$%$8$-(ierr)
i' (ierr .ne. "#$%)*++,))) then
print *,>,rror starting "#$ program. -erminating.>
call "#$%/?13-("#$%+1""%21345, rc, ierr)
end i'
call "#$%+1""%3/8@("#$%+1""%21345, ran, ierr)
call "#$%+1""%)$A,("#$%+1""%21345, numtass, ierr)
print *, >8um0er o' tass!>,numtass,> ": ran!>,ran
+ ****** do some <or ******
call "#$%=$8/4$A,(ierr)
end
P#int t# P#int C#$$%nicati#n R#%tines
,eneral Concepts
./(es #4 P#int-t#-P#int =(erati#ns6
MPI (#int-t#-(#int #(erati#ns t/(ica''/ in"#'"e $essage (assing bet;een t;#: and #n'/
t;#: di44erent MPI tas*s. =ne tas* is (er4#r$ing a send #(erati#n and the #ther tas* is
(er4#r$ing a $atching recei"e #(erati#n.
.here are di44erent t/(es #4 send and recei"e r#%tines %sed 4#r di44erent (%r(#ses. 9#r
e5a$('e6
o S/nchr#n#%s send
o )'#c*ing send @ b'#c*ing recei"e
o +#n-b'#c*ing send @ n#n-b'#c*ing recei"e
o )%44ered send
o C#$bined send@recei"e
o >Read/> send
An/ t/(e #4 send r#%tine can be (aired ;ith an/ t/(e #4 recei"e r#%tine.
MPI a's# (r#"ides se"era' r#%tines ass#ciated ;ith send - recei"e #(erati#ns: s%ch as
th#se %sed t# ;ait 4#r a $essageIs arri"a' #r (r#be t# 4ind #%t i4 a $essage has arri"ed.
)%44ering6
In a (er4ect ;#r'd: e"er/ send #(erati#n ;#%'d be (er4ect'/ s/nchr#ni<ed ;ith its
$atching recei"e. .his is rare'/ the case. S#$eh#; #r #ther: the MPI i$('e$entati#n
$%st be ab'e t# dea' ;ith st#ring data ;hen the t;# tas*s are #%t #4 s/nc.
C#nsider the 4#''#;ing t;# cases6
o A send #(erati#n #cc%rs & sec#nds be4#re the recei"e is read/ - ;here is the
$essage ;hi'e the recei"e is (ending?
o M%'ti('e sends arri"e at the sa$e recei"ing tas* ;hich can #n'/ acce(t #ne send at
a ti$e - ;hat ha((ens t# the $essages that are >bac*ing %(>?
.he MPI i$('e$entati#n 7n#t the MPI standard8 decides ;hat ha((ens t# data in these
t/(es #4 cases. ./(ica''/: a system buffer area is reser"ed t# h#'d data in transit. 9#r
e5a$('e6
S/ste$ b%44er s(ace is6
o =(aA%e t# the (r#gra$$er and $anaged entire'/ b/ the MPI 'ibrar/
o A 4inite res#%rce that can be eas/ t# e5ha%st
o =4ten $/steri#%s and n#t ;e'' d#c%$ented
o Ab'e t# e5ist #n the sending side: the recei"ing side: #r b#th
o S#$ething that $a/ i$(r#"e (r#gra$ (er4#r$ance beca%se it a''#;s send -
recei"e #(erati#ns t# be as/nchr#n#%s.
User $anaged address s(ace 7i.e. /#%r (r#gra$ "ariab'es8 is ca''ed the application
buffer. MPI a's# (r#"ides 4#r a %ser $anaged send b%44er.
)'#c*ing "s. +#n-b'#c*ing6
M#st #4 the MPI (#int-t#-(#int r#%tines can be %sed in either b'#c*ing #r n#n-b'#c*ing
$#de.
)'#c*ing6
o A b'#c*ing send r#%tine ;i'' #n'/ >ret%rn> a4ter it is sa4e t# $#di4/ the a(('icati#n
b%44er 7/#%r send data8 4#r re%se. Sa4e $eans that $#di4icati#ns ;i'' n#t a44ect the
data intended 4#r the recei"e tas*. Sa4e d#es n#t i$('/ that the data ;as act%a''/
recei"ed - it $a/ "er/ ;e'' be sitting in a s/ste$ b%44er.
o A b'#c*ing send can be s/nchr#n#%s ;hich $eans there is handsha*ing #cc%rring
;ith the recei"e tas* t# c#n4ir$ a sa4e send.
o A b'#c*ing send can be as/nchr#n#%s i4 a s/ste$ b%44er is %sed t# h#'d the data
4#r e"ent%a' de'i"er/ t# the recei"e.
o A b'#c*ing recei"e #n'/ >ret%rns> a4ter the data has arri"ed and is read/ 4#r %se b/
the (r#gra$.
+#n-b'#c*ing6
o +#n-b'#c*ing send and recei"e r#%tines beha"e si$i'ar'/ - the/ ;i'' ret%rn a'$#st
i$$ediate'/. .he/ d# n#t ;ait 4#r an/ c#$$%nicati#n e"ents t# c#$('ete: s%ch as
$essage c#(/ing 4r#$ %ser $e$#r/ t# s/ste$ b%44er s(ace #r the act%a' arri"a' #4
$essage.
o +#n-b'#c*ing #(erati#ns si$('/ >reA%est> the MPI 'ibrar/ t# (er4#r$ the
#(erati#n ;hen it is ab'e. .he %ser can n#t (redict ;hen that ;i'' ha((en.
o It is %nsa4e t# $#di4/ the a(('icati#n b%44er 7/#%r "ariab'e s(ace8 %nti' /#% *n#;
4#r a 4act the reA%ested n#n-b'#c*ing #(erati#n ;as act%a''/ (er4#r$ed b/ the
'ibrar/. .here are >;ait> r#%tines %sed t# d# this.
o +#n-b'#c*ing c#$$%nicati#ns are (ri$ari'/ %sed t# #"er'a( c#$(%tati#n ;ith
c#$$%nicati#n and e5('#it (#ssib'e (er4#r$ance gains.
=rder and 9airness6
=rder6
o MPI g%arantees that $essages ;i'' n#t #"erta*e each #ther.
o I4 a sender sends t;# $essages 7Message 1 and Message 28 in s%ccessi#n t# the
sa$e destinati#n: and b#th $atch the sa$e recei"e: the recei"e #(erati#n ;i''
recei"e Message 1 be4#re Message 2.
o I4 a recei"er (#sts t;# recei"es 7Recei"e 1 and Recei"e 28: in s%ccessi#n: and b#th
are '##*ing 4#r the sa$e $essage: Recei"e 1 ;i'' recei"e the $essage be4#re
Recei"e 2.
o =rder r%'es d# n#t a(('/ i4 there are $%'ti('e threads (artici(ating in the
c#$$%nicati#n #(erati#ns.
9airness6
o MPI d#es n#t g%arantee 4airness - itIs %( t# the (r#gra$$er t# (re"ent >#(erati#n
star"ati#n>.
o !5a$('e6 tas* 3 sends a $essage t# tas* 2. ?#;e"er: tas* 1 sends a c#$(eting
$essage that $atches tas* 2Is recei"e. =n'/ #ne #4 the sends ;i'' c#$('ete.
P#int t# P#int C#$$%nicati#n R#%tines
MPI Message Passing (outine rguments
MPI (#int-t#-(#int c#$$%nicati#n r#%tines genera''/ ha"e an arg%$ent 'ist that ta*es #ne #4 the
4#''#;ing 4#r$ats6
"loc+ing sends
MPI_(end(!uffer,c"unt,t#pe,dest,ta$,c"mm)
-on.bloc+ing sends
MPI_Isend(!uffer,c"unt,t#pe,dest,ta$,c"mm,re7uest)
"loc+ing recei!e
MPI_2ec,(!uffer,c"unt,t#pe,s"urce,ta$,c"mm,status)
-on.bloc+ing
recei!e
MPI_Irec,(!uffer,c"unt,t#pe,s"urce,ta$,c"mm,re7uest)
"uffer
Pr#gra$ 7a(('icati#n8 address s(ace that re4erences the data that is t# be sent #r recei"ed.
In $#st cases: this is si$('/ the "ariab'e na$e that is be sent@recei"ed. 9#r C (r#gra$s:
this arg%$ent is (assed b/ re4erence and %s%a''/ $%st be (re(ended ;ith an a$(ersand6
,ar8
*ata Count
Indicates the n%$ber #4 data e'e$ents #4 a (artic%'ar t/(e t# be sent.
*ata Type
9#r reas#ns #4 (#rtabi'it/: MPI (rede4ines its e'e$entar/ data t/(es. .he tab'e be'#; 'ists
th#se reA%ired b/ the standard.
C *ata Types Fortran *ata Types
MPI_%9&2
signed char
MPI_%9&2&%-)2
character718
MPI_(9/2-
signed sh#rt int
MPI_I*-
signed int
MPI_I*-)4)2
integer
MPI_'/*4
signed '#ng int
MPI_:*(I4*)+_%9&2
%nsigned char
MPI_:*(I4*)+_(9/2-
%nsigned sh#rt
int

MPI_:*(I4*)+
%nsigned int
MPI_:*(I4*)+_'/*4
%nsigned '#ng int
MPI_6'/&-
4'#at
MPI_2)&'
rea'
MPI_+/:B')
d#%b'e
MPI_+/:B')_P2)%I(I/*
d#%b'e (recisi#n
MPI_'/*4_+/:B')
'#ng d#%b'e

MPI_%/MP')X
c#$('e5

MPI_+/:B')_%/MP')X
d#%b'e c#$('e5

MPI_'/4I%&'
'#gica'
MPI_B;-)
0 binar/ digits
MPI_B;-)
0 binar/ digits
MPI_P&%3)+
data (ac*ed #r
%n(ac*ed ;ith
MPIFPac*78@
MPIFUn(ac*
MPI_P&%3)+
data (ac*ed #r
%n(ac*ed ;ith
MPIFPac*78@
MPIFUn(ac*
-otes#
Pr#gra$$ers $a/ a's# create their #;n data t/(es 7see -eri"ed -ata ./(es8.
MPIF)G.! and MPIFPACJ!- d# n#t c#rres(#nd t# standard C #r 9#rtran t/(es.
.he MPI standard inc'%des the 4#''#;ing #(ti#na' data t/(es6
o C6 MPIFL=+GFL=+GFI+.
o 9#rtran6 MPIFI+.!G!R1: MPIFI+.!G!R2: MPIFI+.!G!R:
MPIFR!AL2: MPIFR!AL: MPIFR!AL0
S#$e i$('e$entati#ns $a/ inc'%de additi#na' e'e$entar/ data t/(es
7MPIFL=GICAL2: MPIFC=MPL!H32: etc.8. Chec* the MPI header 4i'e.
*estination
An arg%$ent t# send r#%tines that indicates the (r#cess ;here a $essage sh#%'d be
de'i"ered. S(eci4ied as the ran* #4 the recei"ing (r#cess.
Source
An arg%$ent t# recei"e r#%tines that indicates the #riginating (r#cess #4 the $essage.
S(eci4ied as the ran* #4 the sending (r#cess. .his $a/ be set t# the ;i'd card
MPIFA+GFS=URC! t# recei"e a $essage 4r#$ an/ tas*.
Tag
Arbitrar/ n#n-negati"e integer assigned b/ the (r#gra$$er t# %niA%e'/ identi4/ a
$essage. Send and recei"e #(erati#ns sh#%'d $atch $essage tags. 9#r a recei"e
#(erati#n: the ;i'd card MPIFA+GF.AG can be %sed t# recei"e an/ $essage regard'ess
#4 its tag. .he MPI standard g%arantees that integers 3-32,6, can be %sed as tags: b%t
$#st i$('e$entati#ns a''#; a $%ch 'arger range than this.
Communicator
Indicates the c#$$%nicati#n c#nte5t: #r set #4 (r#cesses 4#r ;hich the s#%rce #r
destinati#n 4ie'ds are "a'id. Un'ess the (r#gra$$er is e5('icit'/ creating ne;
c#$$%nicat#rs: the (rede4ined c#$$%nicat#r MPIFC=MMFW=RL- is %s%a''/ %sed.
Status
9#r a recei"e #(erati#n: indicates the s#%rce #4 the $essage and the tag #4 the $essage. In
C: this arg%$ent is a (#inter t# a (rede4ined str%ct%re MPIFStat%s 7e5.
stat.MPIFS=URC! stat.MPIF.AG8. In 9#rtran: it is an integer arra/ #4 si<e
MPIFS.A.USFSIK! 7e5. stat7MPIFS=URC!8 stat7MPIF.AG88. Additi#na''/: the act%a'
n%$ber #4 b/tes recei"ed are #btainab'e 4r#$ Stat%s "ia the MPIFGetFc#%nt r#%tine.
(e/uest
Used b/ n#n-b'#c*ing send and recei"e #(erati#ns. Since n#n-b'#c*ing #(erati#ns $a/
ret%rn be4#re the reA%ested s/ste$ b%44er s(ace is #btained: the s/ste$ iss%es a %niA%e
>reA%est n%$ber>. .he (r#gra$$er %ses this s/ste$ assigned >hand'e> 'ater 7in a WAI.
t/(e r#%tine8 t# deter$ine c#$('eti#n #4 the n#n-b'#c*ing #(erati#n. In C: this arg%$ent
is a (#inter t# a (rede4ined str%ct%re MPIFReA%est. In 9#rtran: it is an integer.
P#int t# P#int C#$$%nicati#n R#%tines
"loc+ing Message Passing (outines
.he $#re c#$$#n'/ %sed MPI b'#c*ing $essage (assing r#%tines are described be'#;.
MPI&Send
)asic b'#c*ing send #(erati#n. R#%tine ret%rns #n'/ a4ter the a(('icati#n b%44er in the
sending tas* is 4ree 4#r re%se. +#te that this r#%tine $a/ be i$('e$ented di44erent'/ #n
di44erent s/ste$s. .he MPI standard (er$its the %se #4 a s/ste$ b%44er b%t d#es n#t
reA%ire it. S#$e i$('e$entati#ns $a/ act%a''/ %se a s/nchr#n#%s send 7disc%ssed be'#;8
t# i$('e$ent the basic b'#c*ing send.
MPI_(end (!uf,c"unt,datat#pe,dest,ta$,c"mm)
MPI_()*+ (!uf,c"unt,datat#pe,dest,ta$,c"mm,ierr)
MPI&(ec!
Recei"e a $essage and b'#c* %nti' the reA%ested data is a"ai'ab'e in the a(('icati#n b%44er
in the recei"ing tas*.
MPI_2ec, (!uf,c"unt,datat#pe,s"urce,ta$,c"mm,status)
MPI_2)%< (!uf,c"unt,datat#pe,s"urce,ta$,c"mm,status,ierr)
MPI&Ssend
S/nchr#n#%s b'#c*ing send6 Send a $essage and b'#c* %nti' the a(('icati#n b%44er in the
sending tas* is 4ree 4#r re%se and the destinati#n (r#cess has started t# recei"e the
$essage.
MPI_(send (!uf,c"unt,datat#pe,dest,ta$,c"mm)
MPI_(()*+ (!uf,c"unt,datat#pe,dest,ta$,c"mm,ierr)
MPI&"send
)%44ered b'#c*ing send6 (er$its the (r#gra$$er t# a''#cate the reA%ired a$#%nt #4
b%44er s(ace int# ;hich data can be c#(ied %nti' it is de'i"ered. Ins%'ates against the
(r#b'e$s ass#ciated ;ith ins%44icient s/ste$ b%44er s(ace. R#%tine ret%rns a4ter the data
has been c#(ied 4r#$ a(('icati#n b%44er s(ace t# the a''#cated send b%44er. M%st be %sed
;ith the MPIF)%44erFattach r#%tine.
MPI_Bsend (!uf,c"unt,datat#pe,dest,ta$,c"mm)
MPI_B()*+ (!uf,c"unt,datat#pe,dest,ta$,c"mm,ierr)
MPI&"uffer&attac0
MPI&"uffer&detac0
Used b/ (r#gra$$er t# a''#cate@dea''#cate $essage b%44er s(ace t# be %sed b/ the
MPIF)send r#%tine. .he si<e arg%$ent is s(eci4ied in act%a' data b/tes - n#t a c#%nt #4
data e'e$ents. =n'/ #ne b%44er can be attached t# a (r#cess at a ti$e. +#te that the I)M
i$('e$entati#n %ses MPIF)S!+-F=2!R?!A- b/tes #4 the a''#cated b%44er 4#r
#"erhead.
MPI_Buffer_attach (!uffer,si.e)
MPI_Buffer_detach (!uffer,si.e)
MPI_B:66)2_&--&%9 (!uffer,si.e,ierr)
MPI_B:66)2_+)-&%9 (!uffer,si.e,ierr)
MPI&(send
)'#c*ing read/ send. Sh#%'d #n'/ be %sed i4 the (r#gra$$er is certain that the $atching
recei"e has a'read/ been (#sted.
MPI_2send (!uf,c"unt,datat#pe,dest,ta$,c"mm)
MPI_2()*+ (!uf,c"unt,datat#pe,dest,ta$,c"mm,ierr)
MPI&Sendrec!
Send a $essage and (#st a recei"e be4#re b'#c*ing. Wi'' b'#c* %nti' the sending
a(('icati#n b%44er is 4ree 4#r re%se and %nti' the recei"ing a(('icati#n b%44er c#ntains the
recei"ed $essage.
MPI_(endrec, (send!uf,sendc"unt,sendt#pe,dest,sendta$,
...... rec,!uf,rec,c"unt,rec,t#pe,s"urce,rec,ta$,
...... c"mm,status)
MPI_()*+2)%< (send!uf,sendc"unt,sendt#pe,dest,sendta$,
...... rec,!uf,rec,c"unt,rec,t#pe,s"urce,rec,ta$,
...... c"mm,status,ierr)
MPI&'ait
MPI&'aitany
MPI&'aitall
MPI&'aitsome
MPIFWait b'#c*s %nti' a s(eci4ied n#n-b'#c*ing send #r recei"e #(erati#n has c#$('eted.
9#r $%'ti('e n#n-b'#c*ing #(erati#ns: the (r#gra$$er can s(eci4/ an/: a'' #r s#$e
c#$('eti#ns.
MPI_5ait (re7uest,status)
MPI_5aitan# (c"unt,arra#_"f_re7uests,index,status)
MPI_5aitall (c"unt,arra#_"f_re7uests,arra#_"f_statuses)
MPI_5aits"me (inc"unt,arra#_"f_re7uests,"utc"unt,
...... arra#_"f_"ffsets, arra#_"f_statuses)
MPI_5&I- (re7uest,status,ierr)
MPI_5&I-&*; (c"unt,arra#_"f_re7uests,index,status,ierr)
MPI_5&I-&'' (c"unt,arra#_"f_re7uests,arra#_"f_statuses,
...... ierr)
MPI_5&I-(/M) (inc"unt,arra#_"f_re7uests,"utc"unt,
...... arra#_"f_"ffsets, arra#_"f_statuses,ierr)
MPI&Probe
Per4#r$s a b'#c*ing test 4#r a $essage. .he >;i'dcards> MPIFA+GFS=URC! and
MPIFA+GF.AG $a/ be %sed t# test 4#r a $essage 4r#$ an/ s#%rce #r ;ith an/ tag. 9#r
the C r#%tine: the act%a' s#%rce and tag ;i'' be ret%rned in the stat%s str%ct%re as
stat%s.MPIFS=URC! and stat%s.MPIF.AG. 9#r the 9#rtran r#%tine: the/ ;i'' be ret%rned
in the integer arra/ stat%s7MPIFS=URC!8 and stat%s7MPIF.AG8.
MPI_Pr"!e (s"urce,ta$,c"mm,status)
MPI_P2/B) (s"urce,ta$,c"mm,status,ierr)
$%amples# "loc+ing Message Passing (outines
.as* 3 (ings tas* 1 and a;aits ret%rn (ing
C Lang%age - )'#c*ing Message Passing R#%tines !5a$('e
#include "mpi.h"
#include <stdio.h>
int main(argc,argv)
int argc;
char *argv[]; {
int numtass, ran, dest, source, rc, count, tag!B;
char inmsg, outmsg!>C>;
"#$%)tatus )tat;
"#$%$nit(&argc,&argv);
"#$%+omm%si7e("#$%+1""%21345, &numtass);
"#$%+omm%ran("#$%+1""%21345, &ran);
i' (ran !! D) {
dest ! B;
source ! B;
rc ! "#$%)end(&outmsg, B, "#$%+E/3, dest, tag, "#$%+1""%21345);
rc ! "#$%3ecv(&inmsg, B, "#$%+E/3, source, tag, "#$%+1""%21345,
&)tat);
6
else i' (ran !! B) {
dest ! D;
source ! D;
rc ! "#$%3ecv(&inmsg, B, "#$%+E/3, source, tag, "#$%+1""%21345,
&)tat);
rc ! "#$%)end(&outmsg, B, "#$%+E/3, dest, tag, "#$%+1""%21345);
6
rc ! "#$%Fet%count(&)tat, "#$%+E/3, &count);
print'("-as 9dG 3eceived 9d char(s) 'rom tas 9d <ith tag 9d .n",
ran, count, )tat."#$%)1*3+,, )tat."#$%-/F);
"#$%=inali7e();
6
9#rtran - )'#c*ing Message Passing R#%tines !5a$('e
program ping
include >mpi'.h>
integer numtass, ran, dest, source, count, tag, ierr
integer stat("#$%)-/-*)%)$A,)
character inmsg, outmsg
outmsg ! >C>
tag ! B
call "#$%$8$-(ierr)
call "#$%+1""%3/8@("#$%+1""%21345, ran, ierr)
call "#$%+1""%)$A,("#$%+1""%21345, numtass, ierr)
i' (ran .eH. D) then
dest ! B
source ! B
call "#$%),85(outmsg, B, "#$%+E/3/+-,3, dest, tag,
& "#$%+1""%21345, ierr)
call "#$%3,+I(inmsg, B, "#$%+E/3/+-,3, source, tag,
& "#$%+1""%21345, stat, ierr)
else i' (ran .eH. B) then
dest ! D
source ! D
call "#$%3,+I(inmsg, B, "#$%+E/3/+-,3, source, tag,
& "#$%+1""%21345, stat, err)
call "#$%),85(outmsg, B, "#$%+E/3/+-,3, dest, tag,
& "#$%+1""%21345, err)
endi'
call "#$%F,-%+1*8-(stat, "#$%+E/3/+-,3, count, ierr)
print *, >-as >,ran,>G 3eceived>, count, >char(s) 'rom tas>,
& stat("#$%)1*3+,), ><ith tag>,stat("#$%-/F)
call "#$%=$8/4$A,(ierr)
end
P#int t# P#int C#$$%nicati#n R#%tines
-on."loc+ing Message Passing (outines
.he $#re c#$$#n'/ %sed MPI n#n-b'#c*ing $essage (assing r#%tines are described be'#;.
MPI&Isend
Identi4ies an area in $e$#r/ t# ser"e as a send b%44er. Pr#cessing c#ntin%es i$$ediate'/
;ith#%t ;aiting 4#r the $essage t# be c#(ied #%t 4r#$ the a(('icati#n b%44er. A
c#$$%nicati#n reA%est hand'e is ret%rned 4#r hand'ing the (ending $essage stat%s. .he
(r#gra$ sh#%'d n#t $#di4/ the a(('icati#n b%44er %nti' s%bseA%ent ca''s t# MPIFWait #r
MPIF.est indicate that the n#n-b'#c*ing send has c#$('eted.
MPI_Isend (!uf,c"unt,datat#pe,dest,ta$,c"mm,re7uest)
MPI_I()*+ (!uf,c"unt,datat#pe,dest,ta$,c"mm,re7uest,ierr)
MPI&Irec!
Identi4ies an area in $e$#r/ t# ser"e as a recei"e b%44er. Pr#cessing c#ntin%es
i$$ediate'/ ;ith#%t act%a''/ ;aiting 4#r the $essage t# be recei"ed and c#(ied int# the
the a(('icati#n b%44er. A c#$$%nicati#n reA%est hand'e is ret%rned 4#r hand'ing the
(ending $essage stat%s. .he (r#gra$ $%st %se ca''s t# MPIFWait #r MPIF.est t#
deter$ine ;hen the n#n-b'#c*ing recei"e #(erati#n c#$('etes and the reA%ested $essage
is a"ai'ab'e in the a(('icati#n b%44er.
MPI_Irec, (!uf,c"unt,datat#pe,s"urce,ta$,c"mm,re7uest)
MPI_I2)%< (!uf,c"unt,datat#pe,s"urce,ta$,c"mm,re7uest,ierr)
MPI&Issend
+#n-b'#c*ing s/nchr#n#%s send. Si$i'ar t# MPIFIsend78: e5ce(t MPIFWait78 #r
MPIF.est78 indicates ;hen the destinati#n (r#cess has recei"ed the $essage.
MPI_Issend (!uf,c"unt,datat#pe,dest,ta$,c"mm,re7uest)
MPI_I(()*+ (!uf,c"unt,datat#pe,dest,ta$,c"mm,re7uest,ierr)
MPI&Ibsend
+#n-b'#c*ing b%44ered send. Si$i'ar t# MPIF)send78 e5ce(t MPIFWait78 #r MPIF.est78
indicates ;hen the destinati#n (r#cess has recei"ed the $essage. M%st be %sed ;ith the
MPIF)%44erFattach r#%tine.
MPI_I!send (!uf,c"unt,datat#pe,dest,ta$,c"mm,re7uest)
MPI_IB()*+ (!uf,c"unt,datat#pe,dest,ta$,c"mm,re7uest,ierr)
MPI&Irsend
+#n-b'#c*ing read/ send. Si$i'ar t# MPIFRsend78 e5ce(t MPIFWait78 #r MPIF.est78
indicates ;hen the destinati#n (r#cess has recei"ed the $essage. Sh#%'d #n'/ be %sed i4
the (r#gra$$er is certain that the $atching recei"e has a'read/ been (#sted.
MPI_Irsend (!uf,c"unt,datat#pe,dest,ta$,c"mm,re7uest)
MPI_I2()*+ (!uf,c"unt,datat#pe,dest,ta$,c"mm,re7uest,ierr)
MPI&Test
MPI&Testany
MPI&Testall
MPI&Testsome
MPIF.est chec*s the stat%s #4 a s(eci4ied n#n-b'#c*ing send #r recei"e #(erati#n. .he
>4'ag> (ara$eter is ret%rned '#gica' tr%e 718 i4 the #(erati#n has c#$('eted: and '#gica'
4a'se 738 i4 n#t. 9#r $%'ti('e n#n-b'#c*ing #(erati#ns: the (r#gra$$er can s(eci4/ an/: a''
#r s#$e c#$('eti#ns.
MPI_-est (re7uest,fla$,status)
MPI_-estan# (c"unt,arra#_"f_re7uests,index,fla$,status)
MPI_-estall (c"unt,arra#_"f_re7uests,fla$,arra#_"f_statuses)
MPI_-ests"me (inc"unt,arra#_"f_re7uests,"utc"unt,
...... arra#_"f_"ffsets, arra#_"f_statuses)
MPI_-)(- (re7uest,fla$,status,ierr)
MPI_-)(-&*; (c"unt,arra#_"f_re7uests,index,fla$,status,ierr)
MPI_-)(-&'' (c"unt,arra#_"f_re7uests,fla$,arra#_"f_statuses,ierr)
MPI_-)(-(/M) (inc"unt,arra#_"f_re7uests,"utc"unt,
...... arra#_"f_"ffsets, arra#_"f_statuses,ierr)
MPI&Iprobe
Per4#r$s a n#n-b'#c*ing test 4#r a $essage. .he >;i'dcards> MPIFA+GFS=URC! and
MPIFA+GF.AG $a/ be %sed t# test 4#r a $essage 4r#$ an/ s#%rce #r ;ith an/ tag. .he
integer >4'ag> (ara$eter is ret%rned '#gica' tr%e 718 i4 a $essage has arri"ed: and '#gica'
4a'se 738 i4 n#t. 9#r the C r#%tine: the act%a' s#%rce and tag ;i'' be ret%rned in the stat%s
str%ct%re as stat%s.MPIFS=URC! and stat%s.MPIF.AG. 9#r the 9#rtran r#%tine: the/ ;i''
be ret%rned in the integer arra/ stat%s7MPIFS=URC!8 and stat%s7MPIF.AG8.
MPI_Ipr"!e (s"urce,ta$,c"mm,fla$,status)
MPI_IP2/B) (s"urce,ta$,c"mm,fla$,status,ierr)
$%amples# -on."loc+ing Message Passing (outines
+earest neighb#r e5change in ring t#(#'#g/
C Lang%age - +#n-)'#c*ing Message Passing R#%tines !5a$('e
#include "mpi.h"
#include <stdio.h>
int main(argc,argv)
int argc;
char *argv[]; {
int numtass, ran, neCt, prev, 0u'[J], tagB!B, tagJ!J;
"#$%3eHuest reHs[K];
"#$%)tatus stats[K];
"#$%$nit(&argc,&argv);
"#$%+omm%si7e("#$%+1""%21345, &numtass);
"#$%+omm%ran("#$%+1""%21345, &ran);
prev ! ranLB;
neCt ! ranMB;
i' (ran !! D) prev ! numtass L B;
i' (ran !! (numtass L B)) neCt ! D;
"#$%$recv(&0u'[D], B, "#$%$8-, prev, tagB, "#$%+1""%21345, &reHs[D]);
"#$%$recv(&0u'[B], B, "#$%$8-, neCt, tagJ, "#$%+1""%21345, &reHs[B]);
"#$%$send(&ran, B, "#$%$8-, prev, tagJ, "#$%+1""%21345, &reHs[J]);
"#$%$send(&ran, B, "#$%$8-, neCt, tagB, "#$%+1""%21345, &reHs[N]);

{ do some <or 6
"#$%2aitall(K, reHs, stats);
"#$%=inali7e();
6
9#rtran - +#n-)'#c*ing Message Passing R#%tines !5a$('e
program ringtopo
include >mpi'.h>
integer numtass, ran, neCt, prev, 0u'(J), tagB, tagJ, ierr
integer stats("#$%)-/-*)%)$A,,K), reHs(K)
tagB ! B
tagJ ! J
call "#$%$8$-(ierr)
call "#$%+1""%3/8@("#$%+1""%21345, ran, ierr)
call "#$%+1""%)$A,("#$%+1""%21345, numtass, ierr)
prev ! ran L B
neCt ! ran M B
i' (ran .eH. D) then
prev ! numtass L B
endi'
i' (ran .eH. numtass L B) then
neCt ! D
endi'
call "#$%$3,+I(0u'(B), B, "#$%$8-,F,3, prev, tagB,
& "#$%+1""%21345, reHs(B), ierr)
call "#$%$3,+I(0u'(J), B, "#$%$8-,F,3, neCt, tagJ,
& "#$%+1""%21345, reHs(J), ierr)
call "#$%$),85(ran, B, "#$%$8-,F,3, prev, tagJ,
& "#$%+1""%21345, reHs(N), ierr)
call "#$%$),85(ran, B, "#$%$8-,F,3, neCt, tagB,
& "#$%+1""%21345, reHs(K), ierr)
+ do some <or
call "#$%2/$-/44(K, reHs, stats, ierr);
call "#$%=$8/4$A,(ierr)
end
C#''ecti"e C#$$%nicati#n R#%tines
A'' #r +#ne6
C#''ecti"e c#$$%nicati#n $%st in"#'"e all (r#cesses in the sc#(e #4 a c#$$%nicat#r. A''
(r#cesses are b/ de4a%'t: $e$bers in the c#$$%nicat#r MPIFC=MMFW=RL-.
It is the (r#gra$$erIs res(#nsibi'it/ t# ins%re that a'' (r#cesses ;ithin a c#$$%nicat#r
(artici(ate in an/ c#''ecti"e #(erati#ns.
./(es #4 C#''ecti"e =(erati#ns6
Sync0ronization - (r#cesses ;ait %nti' a'' $e$bers #4 the gr#%( ha"e reached the
s/nchr#ni<ati#n (#int.
*ata Mo!ement - br#adcast: scatter@gather: a'' t# a''.
Collecti!e Computation 7red%cti#ns8 - #ne $e$ber #4 the gr#%( c#''ects data 4r#$ the
#ther $e$bers and (er4#r$s an #(erati#n 7$in: $a5: add: $%'ti('/: etc.8 #n that data.
Pr#gra$$ing C#nsiderati#ns and Restricti#ns6
C#''ecti"e #(erati#ns are b'#c*ing.
C#''ecti"e c#$$%nicati#n r#%tines d# n#t ta*e $essage tag arg%$ents.
C#''ecti"e #(erati#ns ;ithin s%bsets #4 (r#cesses are acc#$('ished b/ 4irst (artiti#ning
the s%bsets int# ne; gr#%(s and then attaching the ne; gr#%(s t# ne; c#$$%nicat#rs
7disc%ssed in the Gr#%( and C#$$%nicat#r Manage$ent R#%tines secti#n8.
Can #n'/ be %sed ;ith MPI (rede4ined datat/(es - n#t ;ith MPI -eri"ed -ata ./(es.
Collecti!e Communication (outines
MPI&"arrier
Creates a barrier s/nchr#ni<ati#n in a gr#%(. !ach tas*: ;hen reaching the MPIF)arrier
ca'': b'#c*s %nti' a'' tas*s in the gr#%( reach the sa$e MPIF)arrier ca''.
MPI_Barrier (c"mm)
MPI_B&22I)2 (c"mm,ierr)
MPI&"cast
)r#adcasts 7sends8 a $essage 4r#$ the (r#cess ;ith ran* >r##t> t# a'' #ther (r#cesses in
the gr#%(.
MPI_Bcast (!uffer,c"unt,datat#pe,r""t,c"mm)
MPI_B%&(- (!uffer,c"unt,datat#pe,r""t,c"mm,ierr)
MPI&Scatter
-istrib%tes distinct $essages 4r#$ a sing'e s#%rce tas* t# each tas* in the gr#%(.
MPI_(catter (send!uf,sendcnt,sendt#pe,rec,!uf,
...... rec,cnt,rec,t#pe,r""t,c"mm)
MPI_(%&--)2 (send!uf,sendcnt,sendt#pe,rec,!uf,
...... rec,cnt,rec,t#pe,r""t,c"mm,ierr)
MPI&,at0er
Gathers distinct $essages 4r#$ each tas* in the gr#%( t# a sing'e destinati#n tas*. .his
r#%tine is the re"erse #(erati#n #4 MPIFScatter.
MPI_4ather (send!uf,sendcnt,sendt#pe,rec,!uf,
...... rec,c"unt,rec,t#pe,r""t,c"mm)
MPI_4&-9)2 (send!uf,sendcnt,sendt#pe,rec,!uf,
...... rec,c"unt,rec,t#pe,r""t,c"mm,ierr)
MPI&llgat0er
C#ncatenati#n #4 data t# a'' tas*s in a gr#%(. !ach tas* in the gr#%(: in e44ect: (er4#r$s a
#ne-t#-a'' br#adcasting #(erati#n ;ithin the gr#%(.
MPI_&ll$ather (send!uf,sendc"unt,sendt#pe,rec,!uf,
...... rec,c"unt,rec,t#pe,c"mm)
MPI_&''4&-9)2 (send!uf,sendc"unt,sendt#pe,rec,!uf,
...... rec,c"unt,rec,t#pe,c"mm,inf")
MPI&(educe
A(('ies a red%cti#n #(erati#n #n a'' tas*s in the gr#%( and ('aces the res%'t in #ne tas*.
MPI_2educe (send!uf,rec,!uf,c"unt,datat#pe,"p,r""t,c"mm)
MPI_2)+:%) (send!uf,rec,!uf,c"unt,datat#pe,"p,r""t,c"mm,ierr)
.he (rede4ined MPI red%cti#n #(erati#ns a((ear be'#;. Users can a's# de4ine their #;n
red%cti#n 4%ncti#ns b/ %sing the MPIF=(Fcreate r#%tine.
MPI (eduction Operation C *ata Types Fortran *ata Type
MPI_M&X
$a5i$%$ integer: 4'#at integer: rea': c#$('e5
MPI_MI*
$ini$%$ integer: 4'#at integer: rea': c#$('e5
MPI_(:M
s%$ integer: 4'#at integer: rea': c#$('e5
MPI_P2/+
(r#d%ct integer: 4'#at integer: rea': c#$('e5
MPI_'&*+
'#gica' A+- integer '#gica'
MPI_B&*+
bit-;ise A+- integer: MPIF)G.! integer: MPIF)G.!
MPI_'/2
'#gica' =R integer '#gica'
MPI_B/2
bit-;ise =R integer: MPIF)G.! integer: MPIF)G.!
MPI_'X/2
'#gica' H=R integer '#gica'
MPI_BX/2
bit-;ise H=R integer: MPIF)G.! integer: MPIF)G.!
MPI_M&X'/%
$a5 "a'%e and 4'#at: d#%b'e and '#ng rea': c#$('e5:d#%b'e
'#cati#n d#%b'e (recisi#n
MPI_MI*'/%
$in "a'%e and
'#cati#n
4'#at: d#%b'e and '#ng
d#%b'e
rea': c#$('e5: d#%b'e
(recisi#n
MPI&llreduce
A(('ies a red%cti#n #(erati#n and ('aces the res%'t in a'' tas*s in the gr#%(. .his is
eA%i"a'ent t# an MPIFRed%ce 4#''#;ed b/ an MPIF)cast.
MPI_&llreduce (send!uf,rec,!uf,c"unt,datat#pe,"p,c"mm)
MPI_&''2)+:%) (send!uf,rec,!uf,c"unt,datat#pe,"p,c"mm,ierr)
MPI&(educe&scatter
9irst d#es an e'e$ent-;ise red%cti#n #n a "ect#r acr#ss a'' tas*s in the gr#%(. +e5t: the
res%'t "ect#r is s('it int# disD#int seg$ents and distrib%ted acr#ss the tas*s. .his is
eA%i"a'ent t# an MPIFRed%ce 4#''#;ed b/ an MPIFScatter #(erati#n.
MPI_2educe_scatter (send!uf,rec,!uf,rec,c"unt,datat#pe,
...... "p,c"mm)
MPI_2)+:%)_(%&--)2 (send!uf,rec,!uf,rec,c"unt,datat#pe,
...... "p,c"mm,ierr)
MPI&lltoall
!ach tas* in a gr#%( (er4#r$s a scatter #(erati#n: sending a distinct $essage t# a'' the
tas*s in the gr#%( in #rder b/ inde5.
MPI_&llt"all (send!uf,sendc"unt,sendt#pe,rec,!uf,
...... rec,cnt,rec,t#pe,c"mm)
MPI_&''-/&'' (send!uf,sendc"unt,sendt#pe,rec,!uf,
...... rec,cnt,rec,t#pe,c"mm,ierr)
MPI&Scan
Per4#r$s a scan #(erati#n ;ith res(ect t# a red%cti#n #(erati#n acr#ss a tas* gr#%(.
MPI_(can (send!uf,rec,!uf,c"unt,datat#pe,"p,c"mm)
MPI_(%&* (send!uf,rec,!uf,c"unt,datat#pe,"p,c"mm,ierr)
$%amples# Collecti!e Communications
Per4#r$ a scatter #(erati#n #n the r#;s #4 an arra/
C Lang%age - C#''ecti"e C#$$%nicati#ns !5a$('e
#include "mpi.h"
#include <stdio.h>
#de'ine )$A, K
int main(argc,argv)
int argc;
char *argv[]; {
int numtass, ran, sendcount, recvcount, source;
'loat send0u'[)$A,][)$A,] ! {
{B.D, J.D, N.D, K.D6,
{O.D, P.D, Q.D, R.D6,
{S.D, BD.D, BB.D, BJ.D6,
{BN.D, BK.D, BO.D, BP.D6 6;
'loat recv0u'[)$A,];
"#$%$nit(&argc,&argv);
"#$%+omm%ran("#$%+1""%21345, &ran);
"#$%+omm%si7e("#$%+1""%21345, &numtass);
i' (numtass !! )$A,) {
source ! B;
sendcount ! )$A,;
recvcount ! )$A,;
"#$%)catter(send0u',sendcount,"#$%=41/-,recv0u',recvcount,
"#$%=41/-,source,"#$%+1""%21345);
print'("ran! 9d 3esultsG 9' 9' 9' 9'.n",ran,recv0u'[D],
recv0u'[B],recv0u'[J],recv0u'[N]);
6
else
print'(""ust speci': 9d processors. -erminating..n",)$A,);
"#$%=inali7e();
6
9#rtran - C#''ecti"e C#$$%nicati#ns !5a$('e
program scatter
include >mpi'.h>
integer )$A,
parameter()$A,!K)
integer numtass, ran, sendcount, recvcount, source, ierr
real*K send0u'()$A,,)$A,), recv0u'()$A,)
+ =ortran stores this arra: in column maTor order, so the
+ scatter <ill actuall: scatter columns, not ro<s.
data send0u' ;B.D, J.D, N.D, K.D,
& O.D, P.D, Q.D, R.D,
& S.D, BD.D, BB.D, BJ.D,
& BN.D, BK.D, BO.D, BP.D ;
call "#$%$8$-(ierr)
call "#$%+1""%3/8@("#$%+1""%21345, ran, ierr)
call "#$%+1""%)$A,("#$%+1""%21345, numtass, ierr)
i' (numtass .eH. )$A,) then
source ! B
sendcount ! )$A,
recvcount ! )$A,
call "#$%)+/--,3(send0u', sendcount, "#$%3,/4, recv0u',
& recvcount, "#$%3,/4, source, "#$%+1""%21345, ierr)
print *, >ran! >,ran,> 3esultsG >,recv0u'
else
print *, >"ust speci':>,)$A,,> processors. -erminating.>
endi'
call "#$%=$8/4$A,(ierr)
end
Sa$('e (r#gra$ #%t(%t6
ran! D 3esultsG B.DDDDDD J.DDDDDD N.DDDDDD K.DDDDDD
ran! B 3esultsG O.DDDDDD P.DDDDDD Q.DDDDDD R.DDDDDD
ran! J 3esultsG S.DDDDDD BD.DDDDDD BB.DDDDDD BJ.DDDDDD
ran! N 3esultsG BN.DDDDDD BK.DDDDDD BO.DDDDDD BP.DDDDDD
-eri"ed -ata ./(es
As (re"i#%s'/ $enti#ned: MPI (rede4ines its (ri$iti"e data t/(es6
C *ata Types Fortran *ata Types
"#$%+E/3
"#$%)E13-
"#$%$8-
"#$%418F
"#$%*8)$F8,5%+E/3
"#$%*8)$F8,5%)E13-
"#$%*8)$F8,5%418F
"#$%*8)$F8,5
"#$%=41/-
"#$%51*?4,
"#$%418F%51*?4,
"#$%?U-,
"#$%#/+@,5
"#$%+E/3/+-,3
"#$%$8-,F,3
"#$%3,/4
"#$%51*?4,%#3,+$)$18
"#$%+1"#4,V
"#$%51*?4,%+1"#4,V
"#$%41F$+/4
"#$%?U-,
"#$%#/+@,5
MPI a's# (r#"ides 4aci'ities 4#r /#% t# de4ine /#%r #;n data str%ct%res based %(#n
seA%ences #4 the MPI (ri$iti"e data t/(es. S%ch %ser de4ined str%ct%res are ca''ed deri"ed
data t/(es.
Pri$iti"e data t/(es are c#ntig%#%s. -eri"ed data t/(es a''#; /#% t# s(eci4/ n#n-
c#ntig%#%s data in a c#n"enient $anner and t# treat it as th#%gh it ;as c#ntig%#%s.
MPI (r#"ides se"era' $eth#ds 4#r c#nstr%cting deri"ed data t/(es6
o C#ntig%#%s
o 2ect#r
o Inde5ed
o Str%ct
*eri!ed *ata Type (outines
MPI&Type&contiguous
.he si$('est c#nstr%ct#r. Pr#d%ces a ne; data t/(e b/ $a*ing c#%nt c#(ies #4 an e5isting
data t/(e.
MPI_-#pe_c"nti$u"us (c"unt,"ldt#pe,ne=t#pe)
MPI_-;P)_%/*-I4:/:( (c"unt,"ldt#pe,ne=t#pe,ierr)
MPI&Type&!ector
MPI&Type&0!ector
Si$i'ar t# c#ntig%#%s: b%t a''#;s 4#r reg%'ar ga(s 7stride8 in the dis('ace$ents.
MPIF./(eFh"ect#r is identica' t# MPIF./(eF"ect#r e5ce(t that stride is s(eci4ied in
b/tes.
MPI_-#pe_,ect"r (c"unt,!l"c1len$th,stride,"ldt#pe,ne=t#pe)
MPI_-;P)_<)%-/2 (c"unt,!l"c1len$th,stride,"ldt#pe,ne=t#pe,ierr)
MPI&Type&inde%ed
MPI&Type&0inde%ed
An arra/ #4 dis('ace$ents #4 the in(%t data t/(e is (r#"ided as the $a( 4#r the ne; data
t/(e. MPIF./(eFhinde5ed is identica' t# MPIF./(eFinde5ed e5ce(t that #44sets are
s(eci4ied in b/tes.
MPI_-#pe_indexed (c"unt,!l"c1lens>?,"ffsets>?,"ld_t#pe,ne=t#pe)
MPI_-;P)_I*+)X)+ (c"unt,!l"c1lens(),"ffsets(),"ld_t#pe,ne=t#pe,ierr)
MPI&Type&struct
.he ne; data t/(e is 4#r$ed acc#rding t# c#$('ete'/ de4ined $a( #4 the c#$(#nent data
t/(es.
MPI_-#pe_struct (c"unt,!l"c1lens>?,"ffsets>?,"ld_t#pes,ne=t#pe)
MPI_-;P)_(-2:%- (c"unt,!l"c1lens(),"ffsets(),"ld_t#pes,ne=t#pe,ierr)
MPI&Type&e%tent
Ret%rns the si<e in b/tes #4 the s(eci4ied data t/(e. Use4%' 4#r the MPI s%br#%tines that
reA%ire s(eci4icati#n #4 #44sets in b/tes.
MPI_-#pe_extent (datat#pe,extent)
MPI_-;P)_)X-)*- (datat#pe,extent,ierr)
MPI&Type&commit
C#$$its ne; datat/(e t# the s/ste$. ReA%ired 4#r a'' %ser c#nstr%cted 7deri"ed8
datat/(es.
MPI_-#pe_c"mmit (datat#pe)
MPI_-;P)_%/MMI- (datat#pe,ierr)
MPI&Type&free
-ea''#cates the s(eci4ied datat/(e #bDect. Use #4 this r#%tine is es(ecia''/ i$(#rtant t#
(re"ent $e$#r/ e5ha%sti#n i4 $an/ datat/(e #bDects are created: as in a '##(.
MPI_-#pe_free (datat#pe)
MPI_-;P)_62)) (datat#pe,ierr)
$%amples# Contiguous *eri!ed *ata Type
Create a data t/(e re(resenting a r#; #4 an arra/ and distrib%te a di44erent r#; t# a''
(r#cesses.
C Lang%age - C#ntig%#%s -eri"ed -ata ./(e !5a$('e
#include "mpi.h"
#include <stdio.h>
#de'ine )$A, K
int main(argc,argv)
int argc;
char *argv[]; {
int numtass, ran, source!D, dest, tag!B, i;
'loat a[)$A,][)$A,] !
{B.D, J.D, N.D, K.D,
O.D, P.D, Q.D, R.D,
S.D, BD.D, BB.D, BJ.D,
BN.D, BK.D, BO.D, BP.D6;
'loat 0[)$A,];
"#$%)tatus stat;
"#$%5atat:pe ro<t:pe;
"#$%$nit(&argc,&argv);
"#$%+omm%ran("#$%+1""%21345, &ran);
"#$%+omm%si7e("#$%+1""%21345, &numtass);
"#$%-:pe%contiguous()$A,, "#$%=41/-, &ro<t:pe);
"#$%-:pe%commit(&ro<t:pe);
i' (numtass !! )$A,) {
i' (ran !! D) {
'or (i!D; i<numtass; iMM)
"#$%)end(&a[i][D], B, ro<t:pe, i, tag, "#$%+1""%21345);
6
"#$%3ecv(0, )$A,, "#$%=41/-, source, tag, "#$%+1""%21345, &stat);
print'("ran! 9d 0! 9N.B' 9N.B' 9N.B' 9N.B'.n",
ran,0[D],0[B],0[J],0[N]);
6
else
print'(""ust speci': 9d processors. -erminating..n",)$A,);
"#$%-:pe%'ree(&ro<t:pe);
"#$%=inali7e();
6
9#rtran - C#ntig%#%s -eri"ed -ata ./(e !5a$('e
program contiguous
include >mpi'.h>
integer )$A,
parameter()$A,!K)
integer numtass, ran, source, dest, tag, i, ierr
real*K a(DG)$A,LB,DG)$A,LB), 0(DG)$A,LB)
integer stat("#$%)-/-*)%)$A,), columnt:pe
+ =ortran stores this arra: in column maTor order
data a ;B.D, J.D, N.D, K.D,
& O.D, P.D, Q.D, R.D,
& S.D, BD.D, BB.D, BJ.D,
& BN.D, BK.D, BO.D, BP.D ;
call "#$%$8$-(ierr)
call "#$%+1""%3/8@("#$%+1""%21345, ran, ierr)
call "#$%+1""%)$A,("#$%+1""%21345, numtass, ierr)
call "#$%-U#,%+18-$F*1*)()$A,, "#$%3,/4, columnt:pe, ierr)
call "#$%-U#,%+1""$-(columnt:pe, ierr)

tag ! B
i' (numtass .eH. )$A,) then
i' (ran .eH. D) then
do BD i!D, numtassLB
call "#$%),85(a(D,i), B, columnt:pe, i, tag,
& "#$%+1""%21345,ierr)
BD continue
endi'
source ! D
call "#$%3,+I(0, )$A,, "#$%3,/4, source, tag,
& "#$%+1""%21345, stat, ierr)
print *, >ran! >,ran,> 0! >,0
else
print *, >"ust speci':>,)$A,,> processors. -erminating.>
endi'
call "#$%-U#,%=3,,(columnt:pe, ierr)
call "#$%=$8/4$A,(ierr)
end
Sa$('e (r#gra$ #%t(%t6
ran! D 0! B.D J.D N.D K.D
ran! B 0! O.D P.D Q.D R.D
ran! J 0! S.D BD.D BB.D BJ.D
ran! N 0! BN.D BK.D BO.D BP.D
$%amples# 1ector *eri!ed *ata Type
Create a data t/(e re(resenting a c#'%$n #4 an arra/ and distrib%te di44erent c#'%$ns t#
a'' (r#cesses.
C Lang%age - 2ect#r -eri"ed -ata ./(e !5a$('e
#include "mpi.h"
#include <stdio.h>
#de'ine )$A, K
int main(argc,argv)
int argc;
char *argv[]; {
int numtass, ran, source!D, dest, tag!B, i;
'loat a[)$A,][)$A,] !
{B.D, J.D, N.D, K.D,
O.D, P.D, Q.D, R.D,
S.D, BD.D, BB.D, BJ.D,
BN.D, BK.D, BO.D, BP.D6;
'loat 0[)$A,];
"#$%)tatus stat;
"#$%5atat:pe columnt:pe;
"#$%$nit(&argc,&argv);
"#$%+omm%ran("#$%+1""%21345, &ran);
"#$%+omm%si7e("#$%+1""%21345, &numtass);

"#$%-:pe%vector()$A,, B, )$A,, "#$%=41/-, &columnt:pe);
"#$%-:pe%commit(&columnt:pe);
i' (numtass !! )$A,) {
i' (ran !! D) {
'or (i!D; i<numtass; iMM)
"#$%)end(&a[D][i], B, columnt:pe, i, tag, "#$%+1""%21345);
6

"#$%3ecv(0, )$A,, "#$%=41/-, source, tag, "#$%+1""%21345, &stat);
print'("ran! 9d 0! 9N.B' 9N.B' 9N.B' 9N.B'.n",
ran,0[D],0[B],0[J],0[N]);
6
else
print'(""ust speci': 9d processors. -erminating..n",)$A,);

"#$%-:pe%'ree(&columnt:pe);
"#$%=inali7e();
6
9#rtran - 2ect#r -eri"ed -ata ./(e !5a$('e
program vector
include >mpi'.h>
integer )$A,
parameter()$A,!K)
integer numtass, ran, source, dest, tag, i, ierr
real*K a(DG)$A,LB,DG)$A,LB), 0(DG)$A,LB)
integer stat("#$%)-/-*)%)$A,), ro<t:pe
+ =ortran stores this arra: in column maTor order
data a ;B.D, J.D, N.D, K.D,
& O.D, P.D, Q.D, R.D,
& S.D, BD.D, BB.D, BJ.D,
& BN.D, BK.D, BO.D, BP.D ;
call "#$%$8$-(ierr)
call "#$%+1""%3/8@("#$%+1""%21345, ran, ierr)
call "#$%+1""%)$A,("#$%+1""%21345, numtass, ierr)
call "#$%-U#,%I,+-13()$A,, B, )$A,, "#$%3,/4, ro<t:pe, ierr)
call "#$%-U#,%+1""$-(ro<t:pe, ierr)

tag ! B
i' (numtass .eH. )$A,) then
i' (ran .eH. D) then
do BD i!D, numtassLB
call "#$%),85(a(i,D), B, ro<t:pe, i, tag,
& "#$%+1""%21345, ierr)
BD continue
endi'
source ! D
call "#$%3,+I(0, )$A,, "#$%3,/4, source, tag,
& "#$%+1""%21345, stat, ierr)
print *, >ran! >,ran,> 0! >,0
else
print *, >"ust speci':>,)$A,,> processors. -erminating.>
endi'
call "#$%-U#,%=3,,(ro<t:pe, ierr)
call "#$%=$8/4$A,(ierr)
end
Sa$('e (r#gra$ #%t(%t6
ran! D 0! B.D O.D S.D BN.D
ran! B 0! J.D P.D BD.D BK.D
ran! J 0! N.D Q.D BB.D BO.D
ran! N 0! K.D R.D BJ.D BP.D
$%amples# Inde%ed *eri!ed *ata Type
Create a datat/(e b/ e5tracting "ariab'e (#rti#ns #4 an arra/ and distrib%te t# a'' tas*s.
C Lang%age - Inde5ed -eri"ed -ata ./(e !5a$('e
#include "mpi.h"
#include <stdio.h>
#de'ine 8,4,",8-) P
int main(argc,argv)
int argc;
char *argv[]; {
int numtass, ran, source!D, dest, tag!B, i;
int 0loclengths[J], displacements[J];
'loat a[BP] !
{B.D, J.D, N.D, K.D, O.D, P.D, Q.D, R.D,
S.D, BD.D, BB.D, BJ.D, BN.D, BK.D, BO.D, BP.D6;
'loat 0[8,4,",8-)];
"#$%)tatus stat;
"#$%5atat:pe indeCt:pe;
"#$%$nit(&argc,&argv);
"#$%+omm%ran("#$%+1""%21345, &ran);
"#$%+omm%si7e("#$%+1""%21345, &numtass);
0loclengths[D] ! K;
0loclengths[B] ! J;
displacements[D] ! O;
displacements[B] ! BJ;

"#$%-:pe%indeCed(J, 0loclengths, displacements, "#$%=41/-,
&indeCt:pe);
"#$%-:pe%commit(&indeCt:pe);
i' (ran !! D) {
'or (i!D; i<numtass; iMM)
"#$%)end(a, B, indeCt:pe, i, tag, "#$%+1""%21345);
6

"#$%3ecv(0, 8,4,",8-), "#$%=41/-, source, tag, "#$%+1""%21345, &stat);
print'("ran! 9d 0! 9N.B' 9N.B' 9N.B' 9N.B' 9N.B' 9N.B'.n",
ran,0[D],0[B],0[J],0[N],0[K],0[O]);

"#$%-:pe%'ree(&indeCt:pe);
"#$%=inali7e();
6
9#rtran - Inde5ed -eri"ed -ata ./(e !5a$('e
program indeCed
include >mpi'.h>
integer 8,4,",8-)
parameter(8,4,",8-)!P)
integer numtass, ran, source, dest, tag, i, ierr
integer 0loclengths(DGB), displacements(DGB)
real*K a(DGBO), 0(DG8,4,",8-)LB)
integer stat("#$%)-/-*)%)$A,), indeCt:pe
data a ;B.D, J.D, N.D, K.D, O.D, P.D, Q.D, R.D,
& S.D, BD.D, BB.D, BJ.D, BN.D, BK.D, BO.D, BP.D ;
call "#$%$8$-(ierr)
call "#$%+1""%3/8@("#$%+1""%21345, ran, ierr)
call "#$%+1""%)$A,("#$%+1""%21345, numtass, ierr)
0loclengths(D) ! K
0loclengths(B) ! J
displacements(D) ! O
displacements(B) ! BJ
call "#$%-U#,%$85,V,5(J, 0loclengths, displacements, "#$%3,/4,
& indeCt:pe, ierr)
call "#$%-U#,%+1""$-(indeCt:pe, ierr)

tag ! B
i' (ran .eH. D) then
do BD i!D, numtassLB
call "#$%),85(a, B, indeCt:pe, i, tag, "#$%+1""%21345, ierr)
BD continue
endi'
source ! D
call "#$%3,+I(0, 8,4,",8-), "#$%3,/4, source, tag, "#$%+1""%21345,
& stat, ierr)
print *, >ran! >,ran,> 0! >,0
call "#$%-U#,%=3,,(indeCt:pe, ierr)
call "#$%=$8/4$A,(ierr)
end
Sa$('e (r#gra$ #%t(%t6
ran! D 0! P.D Q.D R.D S.D BN.D BK.D
ran! B 0! P.D Q.D R.D S.D BN.D BK.D
ran! J 0! P.D Q.D R.D S.D BN.D BK.D
ran! N 0! P.D Q.D R.D S.D BN.D BK.D
$%amples# Struct *eri!ed *ata Type
Create a data t/(e that re(resents a (artic'e and distrib%te an arra/ #4 s%ch (artic'es t# a''
(r#cesses.
C Lang%age - Str%ct -eri"ed -ata ./(e !5a$('e
#include "mpi.h"
#include <stdio.h>
#de'ine 8,4," JO
int main(argc,argv)
int argc;
char *argv[]; {
int numtass, ran, source!D, dest, tag!B, i;
t:pede' struct {
'loat C, :, 7;
'loat velocit:;
int n, t:pe;
6 #article;
#article p[8,4,"], particles[8,4,"];
"#$%5atat:pe particlet:pe, oldt:pes[J];
int 0loccounts[J];
;* "#$%/int t:pe used to 0e consistent <ith s:ntaC o' *;
;* "#$%-:pe%eCtent routine *;
"#$%/int o''sets[J], eCtent;
"#$%)tatus stat;
"#$%$nit(&argc,&argv);
"#$%+omm%ran("#$%+1""%21345, &ran);
"#$%+omm%si7e("#$%+1""%21345, &numtass);

;* )etup description o' the K "#$%=41/- 'ields C, :, 7, velocit: *;
o''sets[D] ! D;
oldt:pes[D] ! "#$%=41/-;
0loccounts[D] ! K;
;* )etup description o' the J "#$%$8- 'ields n, t:pe *;
;* 8eed to 'irst 'igure o''set 0: getting si7e o' "#$%=41/- *;
"#$%-:pe%eCtent("#$%=41/-, &eCtent);
o''sets[B] ! K * eCtent;
oldt:pes[B] ! "#$%$8-;
0loccounts[B] ! J;
;* 8o< de'ine structured t:pe and commit it *;
"#$%-:pe%struct(J, 0loccounts, o''sets, oldt:pes, &particlet:pe);
"#$%-:pe%commit(&particlet:pe);
;* $nitiali7e the particle arra: and then send it to each tas *;
i' (ran !! D) {
'or (i!D; i<8,4,"; iMM) {
particles[i].C ! i * B.D;
particles[i].: ! i * LB.D;
particles[i].7 ! i * B.D;
particles[i].velocit: ! D.JO;
particles[i].n ! i;
particles[i].t:pe ! i 9 J;
6
'or (i!D; i<numtass; iMM)
"#$%)end(particles, 8,4,", particlet:pe, i, tag, "#$%+1""%21345);
6

"#$%3ecv(p, 8,4,", particlet:pe, source, tag, "#$%+1""%21345, &stat);
;* #rint a sample o' <hat <as received *;
print'("ran! 9d 9N.J' 9N.J' 9N.J' 9N.J' 9d 9d.n", ran,p[N].C,
p[N].:,p[N].7,p[N].velocit:,p[N].n,p[N].t:pe);

"#$%-:pe%'ree(&particlet:pe);
"#$%=inali7e();
6
9#rtran - Str%ct -eri"ed -ata ./(e !5a$('e
program struct
include >mpi'.h>
integer 8,4,"
parameter(8,4,"!JO)
integer numtass, ran, source, dest, tag, i, ierr
integer stat("#$%)-/-*)%)$A,)
t:pe #article
seHuence
real*K C, :, 7, velocit:
integer n, t:pe
end t:pe #article
t:pe (#article) p(8,4,"), particles(8,4,")
integer particlet:pe, oldt:pes(DGB), 0loccounts(DGB),
& o''sets(DGB), eCtent
call "#$%$8$-(ierr)
call "#$%+1""%3/8@("#$%+1""%21345, ran, ierr)
call "#$%+1""%)$A,("#$%+1""%21345, numtass, ierr)
+ )etup description o' the K "#$%3,/4 'ields C, :, 7, velocit:
o''sets(D) ! D
oldt:pes(D) ! "#$%3,/4
0loccounts(D) ! K
+ )etup description o' the J "#$%$8-,F,3 'ields n, t:pe
+ 8eed to 'irst 'igure o''set 0: getting si7e o' "#$%3,/4
call "#$%-U#,%,V-,8-("#$%3,/4, eCtent, ierr)
o''sets(B) ! K * eCtent
oldt:pes(B) ! "#$%$8-,F,3
0loccounts(B) ! J
+ 8o< de'ine structured t:pe and commit it
call "#$%-U#,%)-3*+-(J, 0loccounts, o''sets, oldt:pes,
& particlet:pe, ierr)
call "#$%-U#,%+1""$-(particlet:pe, ierr)

+ $nitiali7e the particle arra: and then send it to each tas
tag ! B
i' (ran .eH. D) then
do BD i!D, 8,4,"LB
particles(i) ! #article ( B.D*i, LB.D*i, B.D*i,
& D.JO, i, mod(i,J) )
BD continue
do JD i!D, numtassLB
call "#$%),85(particles, 8,4,", particlet:pe, i, tag,
& "#$%+1""%21345, ierr)
JD continue
endi'
source ! D
call "#$%3,+I(p, 8,4,", particlet:pe, source, tag,
& "#$%+1""%21345, stat, ierr)
print *, >ran! >,ran,> p(N)! >,p(N)
call "#$%-U#,%=3,,(particlet:pe, ierr)
call "#$%=$8/4$A,(ierr)
end
Sa$('e (r#gra$ #%t(%t6
ran! D N.DD LN.DD N.DD D.JO N B
ran! J N.DD LN.DD N.DD D.JO N B
ran! B N.DD LN.DD N.DD D.JO N B
ran! N N.DD LN.DD N.DD D.JO N B
Gr#%( and C#$$%nicat#r Manage$ent R#%tines
Gr#%(s "s. C#$$%nicat#rs6
A gr#%( is an #rdered set #4 (r#cesses. !ach (r#cess in a gr#%( is ass#ciated ;ith a
%niA%e integer ran*. Ran* "a'%es start at <er# and g# t# +-1: ;here + is the n%$ber #4
(r#cesses in the gr#%(. In MPI: a gr#%( is re(resented ;ithin s/ste$ $e$#r/ as an
#bDect. It is accessib'e t# the (r#gra$$er #n'/ b/ a >hand'e>. A gr#%( is a';a/s
ass#ciated ;ith a c#$$%nicat#r #bDect.
A c#$$%nicat#r enc#$(asses a gr#%( #4 (r#cesses that $a/ c#$$%nicate ;ith each
#ther. A'' MPI $essages $%st s(eci4/ a c#$$%nicat#r. In the si$('est sense: the
c#$$%nicat#r is an e5tra >tag> that $%st be inc'%ded ;ith MPI ca''s. Li*e gr#%(s:
c#$$%nicat#rs are re(resented ;ithin s/ste$ $e$#r/ as #bDects and are accessib'e t#
the (r#gra$$er #n'/ b/ >hand'es>. 9#r e5a$('e: the hand'e 4#r the c#$$%nicat#r that
c#$(rises a'' tas*s is MPIFC=MMFW=RL-.
9r#$ the (r#gra$$erIs (ers(ecti"e: a gr#%( and a c#$$%nicat#r are #ne. .he gr#%(
r#%tines are (ri$ari'/ %sed t# s(eci4/ ;hich (r#cesses sh#%'d be %sed t# c#nstr%ct a
c#$$%nicat#r.
Pri$ar/ P%r(#ses #4 Gr#%( and C#$$%nicat#r =bDects6
1. A''#; /#% t# #rgani<e tas*s: based %(#n 4%ncti#n: int# tas* gr#%(s.
2. !nab'e C#''ecti"e C#$$%nicati#ns #(erati#ns acr#ss a s%bset #4 re'ated tas*s.
3. Pr#"ide basis 4#r i$('e$enting %ser de4ined "irt%a' t#(#'#gies
. Pr#"ide 4#r sa4e c#$$%nicati#ns
Pr#gra$$ing C#nsiderati#ns and Restricti#ns6
Gr#%(s@c#$$%nicat#rs are d/na$ic - the/ can be created and destr#/ed d%ring (r#gra$
e5ec%ti#n.
Pr#cesses $a/ be in $#re than #ne gr#%(@c#$$%nicat#r. .he/ ;i'' ha"e a %niA%e ran*
;ithin each gr#%(@c#$$%nicat#r.
MPI (r#"ides #"er 3 r#%tines re'ated t# gr#%(s: c#$$%nicat#rs: and "irt%a' t#(#'#gies.
./(ica' %sage6
1. !5tract hand'e #4 g'#ba' gr#%( 4r#$ MPIFC=MMFW=RL- %sing
MPIFC#$$Fgr#%(
2. 9#r$ ne; gr#%( as a s%bset #4 g'#ba' gr#%( %sing MPIFGr#%(Finc'
3. Create ne; c#$$%nicat#r 4#r ne; gr#%( %sing MPIFC#$$Fcreate
. -eter$ine ne; ran* in ne; c#$$%nicat#r %sing MPIFC#$$Fran*
&. C#nd%ct c#$$%nicati#ns %sing an/ MPI $essage (assing r#%tine
6. When 4inished: 4ree %( ne; c#$$%nicat#r and gr#%( 7#(ti#na'8 %sing
MPIFC#$$F4ree and MPIFGr#%(F4ree
,roup and Communicator Management (outines
Create t;# di44erent (r#cess gr#%(s 4#r se(arate c#''ecti"e c#$$%nicati#ns e5change.
ReA%ires creating ne; c#$$%nicat#rs a's#.
C Lang%age - Gr#%( and C#$$%nicat#r R#%tines !5a$('e
#include "mpi.h"
#include <stdio.h>
#de'ine 8#31+) R
int main(argc,argv)
int argc;
char *argv[]; {
int ran, ne<%ran, send0u', recv0u', numtass,
ransB[K]!{D,B,J,N6, ransJ[K]!{K,O,P,Q6;
"#$%Froup orig%group, ne<%group;
"#$%+omm ne<%comm;
"#$%$nit(&argc,&argv);
"#$%+omm%ran("#$%+1""%21345, &ran);
"#$%+omm%si7e("#$%+1""%21345, &numtass);
i' (numtass (! 8#31+)) {
print'(""ust speci': "#%#31+)! 9d. -erminating..n",8#31+));
"#$%=inali7e();
eCit(D);
6
send0u' ! ran;
;* ,Ctract the original group handle *;
"#$%+omm%group("#$%+1""%21345, &orig%group);
;* 5ivide tass into t<o distinct groups 0ased upon ran *;
i' (ran < 8#31+);J) {
"#$%Froup%incl(orig%group, 8#31+);J, ransB, &ne<%group);
6
else {
"#$%Froup%incl(orig%group, 8#31+);J, ransJ, &ne<%group);
6
;* +reate ne< ne< communicator and then per'orm collective
communications *;
"#$%+omm%create("#$%+1""%21345, ne<%group, &ne<%comm);
"#$%/llreduce(&send0u', &recv0u', B, "#$%$8-, "#$%)*", ne<%comm);
"#$%Froup%ran (ne<%group, &ne<%ran);
print'("ran! 9d ne<ran! 9d recv0u'! 9d.n",ran,ne<%ran,recv0u');
"#$%=inali7e();
6
9#rtran - Gr#%( and C#$$%nicat#r R#%tines !5a$('e
program group
include >mpi'.h>
integer 8#31+)
parameter(8#31+)!R)
integer ran, ne<%ran, send0u', recv0u', numtass
integer ransB(K), ransJ(K), ierr
integer orig%group, ne<%group, ne<%comm
data ransB ;D, B, J, N;, ransJ ;K, O, P, Q;
call "#$%$8$-(ierr)
call "#$%+1""%3/8@("#$%+1""%21345, ran, ierr)
call "#$%+1""%)$A,("#$%+1""%21345, numtass, ierr)
i' (numtass .ne. 8#31+)) then
print *, >"ust speci': "#31+)! >,8#31+),> -erminating.>
call "#$%=$8/4$A,(ierr)
stop
endi'
send0u' ! ran
+ ,Ctract the original group handle
call "#$%+1""%F31*#("#$%+1""%21345, orig%group, ierr)
+ 5ivide tass into t<o distinct groups 0ased upon ran
i' (ran .lt. 8#31+);J) then
call "#$%F31*#%$8+4(orig%group, 8#31+);J, ransB,
& ne<%group, ierr)
else
call "#$%F31*#%$8+4(orig%group, 8#31+);J, ransJ,
& ne<%group, ierr)
endi'
call "#$%+1""%+3,/-,("#$%+1""%21345, ne<%group,
& ne<%comm, ierr)
call "#$%/443,5*+,(send0u', recv0u', B, "#$%$8-,F,3,
& "#$%)*", ne<%comm, ierr)
call "#$%F31*#%3/8@(ne<%group, ne<%ran, ierr)
print *, >ran! >,ran,> ne<ran! >,ne<%ran,> recv0u'! >,
& recv0u'
call "#$%=$8/4$A,(ierr)
end
Sa$('e (r#gra$ #%t(%t6
ran! Q ne<ran! N recv0u'! JJ
ran! D ne<ran! D recv0u'! P
ran! B ne<ran! B recv0u'! P
ran! J ne<ran! J recv0u'! P
ran! P ne<ran! J recv0u'! JJ
ran! N ne<ran! N recv0u'! P
ran! K ne<ran! D recv0u'! JJ
ran! O ne<ran! B recv0u'! JJ
2irt%a' .#(#'#gies
What Are .he/?
In ter$s #4 MPI: a "irt%a' t#(#'#g/ describes a $a((ing@#rdering #4 MPI (r#cesses int# a
ge#$etric >sha(e>.
.he t;# $ain t/(es #4 t#(#'#gies s%((#rted b/ MPI are Cartesian 7grid8 and Gra(h.
MPI t#(#'#gies are "irt%a' - there $a/ be n# re'ati#n bet;een the (h/sica' str%ct%re #4
the (ara''e' $achine and the (r#cess t#(#'#g/.
2irt%a' t#(#'#gies are b%i't %(#n MPI c#$$%nicat#rs and gr#%(s.
M%st be >(r#gra$$ed> b/ the a(('icati#n de"e'#(er.
Wh/ Use .he$?
C#n"enience
o 2irt%a' t#(#'#gies $a/ be %se4%' 4#r a(('icati#ns ;ith s(eci4ic c#$$%nicati#n
(atterns - (atterns that $atch an MPI t#(#'#g/ str%ct%re.
o 9#r e5a$('e: a Cartesian t#(#'#g/ $ight (r#"e c#n"enient 4#r an a(('icati#n that
reA%ires -;a/ nearest neighb#r c#$$%nicati#ns 4#r grid based data.
C#$$%nicati#n !44icienc/
o S#$e hard;are architect%res $a/ i$(#se (ena'ties 4#r c#$$%nicati#ns bet;een
s%ccessi"e'/ distant >n#des>.
o A (artic%'ar i$('e$entati#n $a/ #(ti$i<e (r#cess $a((ing based %(#n the
(h/sica' characteristics #4 a gi"en (ara''e' $achine.
o .he $a((ing #4 (r#cesses int# an MPI "irt%a' t#(#'#g/ is de(endent %(#n the
MPI i$('e$entati#n: and $a/ be t#ta''/ ign#red.
!5a$('e6
A si$('i4ied $a((ing #4 (r#cesses int# a Cartesian "irt%a' t#(#'#g/ a((ears be'#;6
1irtual Topology (outines
Create a 5 Cartesian t#(#'#g/ 4r#$ 16 (r#cess#rs and ha"e each (r#cess e5change its
ran* ;ith 4#%r neighb#rs.
C Lang%age - Cartesian 2irt%a' .#(#'#g/ !5a$('e
#include "mpi.h"
#include <stdio.h>
#de'ine )$A, BP
#de'ine *# D
#de'ine 5128 B
#de'ine 4,=- J
#de'ine 3$FE- N
int main(argc,argv)
int argc;
char *argv[]; {
int numtass, ran, source, dest, out0u', i, tag!B,

in0u'[K]!{"#$%#31+%8*44,"#$%#31+%8*44,"#$%#31+%8*44,"#$%#31+%8*44,6,
n0rs[K], dims[J]!{K,K6,
periods[J]!{D,D6, reorder!D, coords[J];
"#$%3eHuest reHs[R];
"#$%)tatus stats[R];
"#$%+omm cartcomm;
"#$%$nit(&argc,&argv);
"#$%+omm%si7e("#$%+1""%21345, &numtass);
i' (numtass !! )$A,) {
"#$%+art%create("#$%+1""%21345, J, dims, periods, reorder,
&cartcomm);
"#$%+omm%ran(cartcomm, &ran);
"#$%+art%coords(cartcomm, ran, J, coords);
"#$%+art%shi't(cartcomm, D, B, &n0rs[*#], &n0rs[5128]);
"#$%+art%shi't(cartcomm, B, B, &n0rs[4,=-], &n0rs[3$FE-]);
out0u' ! ran;
'or (i!D; i<K; iMM) {
dest ! n0rs[i];
source ! n0rs[i];
"#$%$send(&out0u', B, "#$%$8-, dest, tag,
"#$%+1""%21345, &reHs[i]);
"#$%$recv(&in0u'[i], B, "#$%$8-, source, tag,
"#$%+1""%21345, &reHs[iMK]);
6
"#$%2aitall(R, reHs, stats);

print'("ran! 9d coords! 9d 9d neigh0ors(u,d,l,r)! 9d 9d 9d 9d.n",
ran,coords[D],coords[B],n0rs[*#],n0rs[5128],n0rs[4,=-],
n0rs[3$FE-]);
print'("ran! 9d in0u'(u,d,l,r)! 9d 9d 9d 9d.n",
ran,in0u'[*#],in0u'[5128],in0u'[4,=-],in0u'[3$FE-]);
6
else
print'(""ust speci': 9d processors. -erminating..n",)$A,);

"#$%=inali7e();
6
9#rtran - Cartesian 2irt%a' .#(#'#g/ !5a$('e
program cartesian
include >mpi'.h>
integer )$A,, *#, 5128, 4,=-, 3$FE-
parameter()$A,!BP)
parameter(*#!B)
parameter(5128!J)
parameter(4,=-!N)
parameter(3$FE-!K)
integer numtass, ran, source, dest, out0u', i, tag, ierr,
& in0u'(K), n0rs(K), dims(J), coords(J),
& stats("#$%)-/-*)%)$A,, R), reHs(R), cartcomm,
& periods(J), reorder
data in0u' ;"#$%#31+%8*44,"#$%#31+%8*44,"#$%#31+%8*44,
& "#$%#31+%8*44;, dims ;K,K;, tag ;B;,
& periods ;D,D;, reorder ;D;
call "#$%$8$-(ierr)
call "#$%+1""%)$A,("#$%+1""%21345, numtass, ierr)

i' (numtass .eH. )$A,) then
call "#$%+/3-%+3,/-,("#$%+1""%21345, J, dims, periods, reorder,
& cartcomm, ierr)
call "#$%+1""%3/8@(cartcomm, ran, ierr)
call "#$%+/3-%+1135)(cartcomm, ran, J, coords, ierr)
print *,>ran! >,ran,>coords! >,coords
call "#$%+/3-%)E$=-(cartcomm, D, B, n0rs(*#), n0rs(5128), ierr)
call "#$%+/3-%)E$=-(cartcomm, B, B, n0rs(4,=-), n0rs(3$FE-),
& ierr)
out0u' ! ran
do i!B,K
dest ! n0rs(i)
source ! n0rs(i)
call "#$%$),85(out0u', B, "#$%$8-,F,3, dest, tag,
& "#$%+1""%21345, reHs(i), ierr)
call "#$%$3,+I(in0u'(i), B, "#$%$8-,F,3, source, tag,
& "#$%+1""%21345, reHs(iMK), ierr)
enddo
call "#$%2/$-/44(R, reHs, stats, ierr)
print *,>ran! >,ran,> coords! >,coords,
& > neigh0ors(u,d,l,r)! >,n0rs
print *,>ran! >,ran,> >,
& > in0u'(u,d,l,r)! >,in0u'
else
print *, >"ust speci':>,)$A,,> processors. -erminating.>
endi'
call "#$%=$8/4$A,(ierr)
end
Sa$('e (r#gra$ #%t(%t6 7(artia'8
ran! D coords! D D neigh0ors(u,d,l,r)! LN K LN B
ran! D in0u'(u,d,l,r)! LN K LN B
ran! B coords! D B neigh0ors(u,d,l,r)! LN O D J
ran! B in0u'(u,d,l,r)! LN O D J
ran! J coords! D J neigh0ors(u,d,l,r)! LN P B N
ran! J in0u'(u,d,l,r)! LN P B N
. . . . .
ran! BK coords! N J neigh0ors(u,d,l,r)! BD LN BN BO
ran! BK in0u'(u,d,l,r)! BD LN BN BO
ran! BO coords! N N neigh0ors(u,d,l,r)! BB LN BK LN
ran! BO in0u'(u,d,l,r)! BB LN BK LN
A )rie4 W#rd #n MPI-2
?ist#r/6
Intenti#na''/: the MPI s(eci4icati#n did n#t address se"era' >di44ic%'t> iss%es. 9#r reas#ns
#4 e5(edienc/: these iss%es ;ere de4erred t# a sec#nd s(eci4icati#n: ca''ed MPI-2.
In March 111&: 4#''#;ing the re'ease #4 the initia' MPI s(eci4icati#n: the MPI 9#r%$
began disc%ssing enhance$ents t# the MPI standard. 9#''#;ing this6
o -ece$ber 111&6 S%(erc#$(%ting I1& c#n4erence - )irds #4 a 9eather $eeting t#
disc%ss (r#(#sed e5tensi#ns t# MPI.
o +#"e$ber 11166 S%(erc#$(%ting I16 c#n4erence - MPI-2 dra4t $ade a"ai'ab'e.
P%b'ic c#$$ents s#'icited. Meeting t# disc%ss MPI-2 e5tensi#ns.
o .he dra4t (resented at S%(erc#$(%ting I16 sh#rt'/ therea4ter beca$e the MPI-2
standard.
+#t a'' MPI 'ibraries (r#"ide a 4%'' i$('e$entati#n #4 MPI-2.
Je/ Areas #4 +e; 9%ncti#na'it/6
*ynamic Processes - e5tensi#ns that re$#"e the static (r#cess $#de' #4 MPI. Pr#"ides
r#%tines t# create ne; (r#cesses.
One.Sided Communications - (r#"ides r#%tines 4#r #ne directi#na' c#$$%nicati#ns.
Inc'%de shared $e$#r/ #(erati#ns 7(%t@get8 and re$#te acc%$%'ate #(erati#ns.
$%tended Collecti!e Operations - a''#;s 4#r n#n-b'#c*ing c#''ecti"e #(erati#ns and
a(('icati#n #4 c#''ecti"e #(erati#ns t# inter-c#$$%nicat#rs
$%ternal Interfaces - de4ines r#%tines that a''#; de"e'#(ers t# 'a/er #n t#( #4 MPI: s%ch
as 4#r deb%ggers and (r#4i'ers.
dditional )anguage "indings - describes CCC bindings and disc%sses 9#rtran-13
iss%es.
Parallel I2O - describes MPI s%((#rt 4#r (ara''e' I@=.
M#re In4#r$ati#n #n MPI-26
.he Arg#nne +ati#na' Lab MPI ;eb (ages ha"e MPI-2 in4#r$ati#n. See the Re4erences
secti#n 4#r 'in*s.
LL+L S(eci4ic In4#r$ati#n and Rec#$$endati#ns
A'th#%gh the MPI (r#gra$$ing inter4ace has been standardi<ed: i$('e$entati#ns ;i'' di44er: as
;i'' the ;a/ MPI (r#gra$s are c#$(i'ed and r%n #n di44erent ('at4#r$s. A s%$$ar/ #4 LCIs
MPI en"ir#n$ent is (r#"ided here: h#;e"er %sers ;i'' de4inite'/ ;ant t# c#ns%'t the t%t#ria's
$enti#ned be'#; 4#r a'' #4 the detai's.
Lin%5 C'%sters6
.he M2APIC? 3.1 MPI 'ibrar/ is the de4a%'t MPI 'ibrar/ #n these ('at4#r$s.
.his is an MPI-1 i$('e$entati#n: n#t MPI-2: b%t d#es inc'%de MPI-I@= s%((#rt
+#t thread-sa4e
C: CCC: 9#rtran,,@13@1& are s%((#rted
M2APIC?2 and =(enMPI are a's# a"ai'ab'e #n $#st c'%sters
o Use the use @l c#$$and t# see ;hatIs a"ai'ab'e
o L#ad desired (ac*age ;ith use package_name
C#$(i'ing and r%nning MPI (r#gra$s: see6 Lin%5 C'%sters ="er"ie;
I)M )G@L and )G@P C'%sters6
.he I)M MPI 'ibrar/ is the #n'/ s%((#rted 'ibrar/ #n these ('at4#r$s.
.his is an I)M i$('e$entati#n based #n MPIC?2. Inc'%des MPI-2 4%ncti#na'it/ $in%s
-/na$ic Pr#cesses.
.hread-sa4e
C: CCC: 9#rtran,,@13@1& are s%((#rted
C#$(i'ing and r%nning MPI (r#gra$s: see6
o )G@L )asics6 asc.''n'.g#"@c#$(%tingFres#%rces@b'%egene'@basics@
o )G@P .%t#ria'6 c#$(%ting.''n'.g#"@t%t#ria's@bg(@
T0is completes t0e tutorial3

P'ease c#$('ete the #n'ine e"a'%ati#n 4#r$ - %n'ess /#% are d#ing the
e5ercise: in ;hich case ('ease c#$('ete it at the end #4 the e5ercise.
'0ere 4ould you li+e to go no45
!5ercise
Agenda
)ac* t# the t#(
Re4erences and M#re In4#r$ati#n
A%th#r6 )'aise )arne/: Li"er$#re C#$(%ting.
MPI ;eb (ages at Arg#nne +ati#na' Lab#rat#r/
htt(6@@;;;-%ni5.$cs.an'.g#"@$(i
>Using MPI>: Gr#((: L%s* and S*De''%$. MI. Press: 111.
Li"er$#re C#$(%ting s(eci4ic in4#r$ati#n6
o MPI at LL+L
c#$(%ting.''n'.g#"@$(i
o I)M P=W!R S/ste$s ="er"ie; t%t#ria'
c#$(%ting.''n'.g#"@t%t#ria's@ib$Fs(
o Lin%5 C'%sters ="er"ie; t%t#ria'
c#$(%ting.''n'.g#"@t%t#ria's@'in%5Fc'%sters
I)M Para''e' !n"ir#n$ent Man%a's
htt(6@@;;;-1.ib$.c#$@ser"ers@eser"er@(series@'ibrar/@s(Fb##*s
I)M C#$(i'er -#c%$entati#n6
9#rtran6 ;;;-.ib$.c#$@s#4t;are@ad@4#rtran
C@CCC6 ;;;-.ib$.c#$@s#4t;are@ad@cai5
>RS@6333 SP6 Practica' MPI Pr#gra$$ing>: G%*i/a A#/a$a and L%n +a*an#: RS@6333
.echnica' S%((#rt Center: I)M La(an. A"ai'ab'e 4r#$ I)MIs Redb##*s ser"er at
htt(6@@;;;.redb##*s.ib$.c#$.
>A UserIs G%ide t# MPI>: Peter S. Pachec#. -e(art$ent #4 Mathe$atics: Uni"ersit/ #4
San 9rancisc#.
A((endi5 A6 MPI-1 R#%tine Inde5
.hese $an (ages ;ere deri"ed 4r#$ the M2APIC? 3.1 i$('e$entati#n #4 MPI and $a/
di44er 4r#$ the $an (ages #4 #ther i$('e$entati#ns.
+#t a'' MPI r#%tines are sh#;n
$n!ironment Management (outines
MPIFAb#rt MPIF!rrhand'erFcreat
e
MPIF!rrhand'erF4ree MPIF!rrhand'erFge
t
MPIF!rrhand'erFset MPIF!rr#rFc'ass MPIF!rr#rFstring MPIF9ina'i<e
MPIFGetF(r#cess#r
Fna$e
MPIFInit MPIFInitia'i<ed MPIFWtic*
MPIFWti$e
Point.to.Point Communication (outines
MPIF)send MPIF)sendFinit MPIF)%44erFattach MPIF)%44erFdetach
MPIFCance' MPIFGetFc#%nt MPIFGetFe'e$ents MPIFIbsend
MPIFI(r#be MPIFIrec" MPIFIrsend MPIFIsend
MPIFIssend MPIFPr#be MPIFRec" MPIFRec"Finit
MPIFReA%estF4ree MPIFRsend MPIFRsendFinit MPIFSend
MPIFSendFinit MPIFSendrec" MPIFSendrec"Fre('ac
e
MPIFSsend
MPIFSsendFinit MPIFStart MPIFStarta'' MPIF.est
MPIF.estFcance''ed MPIF.esta'' MPIF.estan/ MPIF.ests#$e
MPIFWait MPIFWaita'' MPIFWaitan/ MPIFWaits#$e
Collecti!e Communication (outines
MPIFA''gather MPIFA''gather" MPIFA''red%ce MPIFA''t#a''
MPIFA''t#a''" MPIF)arrier MPIF)cast MPIFGather
MPIFGather" MPIF=(Fcreate MPIF=(F4ree MPIFRed%ce
MPIFRed%ceFscatter MPIFScan MPIFScatter MPIFScatter"
Process ,roup (outines
MPIFGr#%(Fc#$(ar MPIFGr#%(Fdi44erence MPIFGr#%(Fe5c' MPIFGr#%(F4ree
e
MPIFGr#%(Finc' MPIFGr#%(Fintersecti
#n
MPIFGr#%(FrangeFe
5c'
MPIFGr#%(FrangeF
inc'
MPIFGr#%(Fran* MPIFGr#%(Fsi<e MPIFGr#%(Ftrans'ate
Fran*s
MPIFGr#%(F%ni#n
Communicators (outines
MPIFC#$$Fc#$(a
re
MPIFC#$$Fcreate MPIFC#$$Fd%( MPIFC#$$F4ree
MPIFC#$$Fgr#%( MPIFC#$$Fran* MPIFC#$$Fre$#teF
gr#%(
MPIFC#$$Fre$#t
eFsi<e
MPIFC#$$Fsi<e MPIFC#$$Fs('it MPIFC#$$FtestFinte
r
MPIFInterc#$$Fcr
eate
MPIFInterc#$$F$e
rge

*eri!ed Types (outines
MPIF./(eFc#$$it MPIF./(eFc#ntig%#%s MPIF./(eFe5tent MPIF./(eF4ree
MPIF./(eFhinde5ed MPIF./(eFh"ect#r MPIF./(eFinde5ed MPIF./(eF'b
MPIF./(eFsi<e MPIF./(eFstr%ct MPIF./(eF%b MPIF./(eF"ect#r
1irtual Topology (outines
MPIFCartFc##rds MPIFCartFcreate MPIFCartFget MPIFCartF$a(
MPIFCartFran* MPIFCartFshi4t MPIFCartFs%b MPIFCartdi$Fget
MPIF-i$sFcreate MPIFGra(hFcreate MPIFGra(hFget MPIFGra(hF$a(
MPIFGra(hFneighb
#rs
MPIFGra(hFneighb#rs
Fc#%nt
MPIFGra(hdi$sFget MPIF.#(#Ftest
Miscellaneous (outines
MPIFAddress MPIFAttrFde'ete MPIFAttrFget MPIFAttrF(%t
MPIFJe/"a'Fcreate MPIFJe/"a'F4ree MPIFPac* MPIFPac*Fsi<e
MPIFPc#ntr#' MPIFUn(ac*
4i'e6@@@!6@9ai/a<M23(r#Dects@-CPCM23IMP=R.A+.@MessageM23PassingM23Inter4ace
M237MPI8.ht$
Last M#di4ied6 3&@21@2311 16&1613 b'aisebN''n'.g#"
UCRL-MI-133316