PARALLEL COMPUTING WITH MATLAB

Step by step guide (beginners edition)

by Jose M. Mier, jmierlo2@illinois.edu
under the instruction and guidance of Nils Oberg, noberg@illinois.edu
Ven Te Chow Hydrosystems Laboratory
Civil & Environmental Engineering Department
University of Illinois at Urbana-Champaign

This guide is a compilation of the steps and clues necessary to get your already-running
MATLAB code and make it run in parallel in several processors, so that you can speed up
your executions and results

Original:
Last Revision:

April 2010
-

TABLE OF CONTENTS

OBJECTIVE:

STEP 1: Have your code ready and running in ordinary MATLAB

STEP 2: Prepare your ordinary-running code to become a parallel-running code

STEP 3: Things to consider in your code when going parallel

EXAMPLE CODE

PERFORMANCE CONSIDERATIONS

OBJECTIVE:
This guide is a compilation of the steps and clues necessary to get your already-running
MATLAB code and make it run in parallel in several processors, so that you can speed up
your executions and results.

STEP 1: Have your code ready and running in ordinary MATLAB

First thing, make sure your code is running and doing what you want it to do in your ordinary
MATLAB execution in your PC (= personal computer, I dont care if Windows or Macintosh or
Linux or Xxx).
Also, to run in parallel, you must use a version of MATLAB that is 2007b or later.

STEP 2: Prepare your ordinary-running code to become a parallel-running code

In order to tell MATLAB that you want to run using multiple processors, you need two main
instructions in your code:
matlabpool: tells MATLAB that you want to use multiple processors in your run
at the beginning of the code, type:
matlabpool('open',#);

#: number of cores you want to use (max is number of cores in your PC)
- this instruction opens the parallel computing toolbox
- at this moment, MATLAB checks for a distributed-computing license (there
are only 19 of these available campus-wide, so be aware that if there is no one
available, you may get an error message, and will have to wait).
at the end of the code, type:
matlabpool('close');

- this will close the parallel computing, however, the license will still be in use
until you close your MATLAB session.
parfor:

is the parallel computing version of the traditional for command for a loop.
use it in place of for, just as simple as that:
parfor i = 1 : 100
(your code)
end

- the loop variable (i) can only be a vector of consecutive integers

- the code inside this loop is the portion of your code that will be executed in
parallel
%% check MATLAB help for particular conditions for both matlabpool and parfor %%

STEP 3: Things to consider in your code when going parallel

On the right side of your MATLAB Editor window you can see the warnings and errors that
MATLAB is detecting in your code before you even run.
In the case of having a parfor loop, MATLAB may become a little bit more picky than usual.
Here are some tips:
- when using parfor, all the executions of the loop need to be independent from
each other. This is such that the parfor loop doesnt even executes consecutively
(i = 1, 2, 3, 4, ), but quite randomly (i = 3, 4, 59, 37, ) decided by MATLAB.
This way, you cannot have any variable that can be modified by all the different
processors at the same time, unless it has the proper dimensional index (x(i,j,k)).
-

the parfor loop can only be done for one level, i.e., you cannot have a parfor
loop inside another parfor loop. Sorry! However, the parfor loop doesnt need to
be the outermost, it can be inside a regular for loop if needed.

You can have for loops inside the parfor loop, but you will need to pre-allocate
all the variables that grow inside the for loops. Just pre-allocating immediately
before the corresponding for loop will do. (This is usually a recommendation for
speed in regular MATLAB runs, but it is a must in parallel mode.)

You cannot have any instruction that calls to the workspace inside the parfor loop,
such as delete variables (clear) or list variables (who).

Be careful if you have figures inside the parfor loop. They should work, and you
can even save them. But if you are displaying on screen, make sure you close the
figures within the loop, so that they dont flood your desktop!

The best way to get your results out is to have a one-dimensional variable inside
the parfor loop, with the dimension being the parfor loop itself (see example).

You can also print to file and save data inside the parfor loop, this is also a good
way to take results out of the parfor loop.

(include here any more tips that you may want to share after working in parallel
mode yourself)

After all this, we can say that parallel computing done this way in MATLAB is great when you
need to run a code for (a lot of) different cases, when the cases are independent from each
other (also known as embarrassingly parallel applications ).

EXAMPLE CODE
This code will run in parallel mode in MATLAB:
matlabpool ('open',2);
x = zeros(100,10);
parfor i = 1:100
y = zeros(1,10);
for j = 1:10
y(j) = i;
end
y
x(i,:) = y;
end
x
matlabpool close;

%
%
%
%
%

Call to open the distributed processing

Initialize the main variable
Parallel loop
Initialize the secondary variable
Inner loop

% Display the inner variable (note the random

execution about "i" in the command window)
% Get values from loop into the main variable
% Display main variable
% Close the distributed computing

This code will NOT run in parallel mode:

matlabpool ('open',2);
parfor i = 1:100
a = 3;
(your code)
end
a
matlabpool close;

% Call to open the distributed processing

% Parallel loop

% Display main variable

% Close the distributed computing

It wont work because we are using outside the parfor loop a variable (a) that gets
modified inside the parfor loop (even though it is a constant in this case).
Solutions:
a) Instead, we may need to define a outside the parfor loop (if it is going to remain
constant):
matlabpool ('open',2);
a = 3;
parfor i = 1:100
(your code)
end
a
matlabpool close;

% Call to open the distributed processing

% Parallel loop
% Display main variable
% Close the distributed computing

b) Or assign it a dimension according to the parfor loop (i):

matlabpool ('open',2);
a = zeros(1,100);
parfor i = 1:100
a(i) = 3;
(your code)
end
a
matlabpool close;

% Call to open the distributed processing

% Initialize the main variable
% Parallel loop

% Display main variable

% Close the distributed computing

This case may run even if we dont initialize the variable, but it is good to do it, for
more broad cases.

PERFORMANCE CONSIDERATIONS
For an overview of performance comparisons when running a MATLAB code in different
hardware with various numbers of processors, check Nils Oberg et al.s paper entitled
MATLAB Parallel Computing Toolbox Benchmark for an Embarrassingly Parallel Application
(see http://vtchl.illinois.edu/publications/matlab/parallel_benchmark_2008).