Insert Quality Here

Introduction:

The Message Passing interface (MPI) provides a standard way to distribute functions/objects/data across a cluster of systems -- whether embarrassingly parallel, or something more robust.  It’s an amazingly simple api to get started with, though working out how to send and receive the messages in the right order can get a bit tricky.  It only takes 6 commands to get going:

 

MPI::Init(argc , argv)

MPI::COMM_WORLD.Get_size()

MPI::COMM_WORLLD.Get_rank()

MPI::COMM_WORLD.Send()

MPI::COM_WORLD.Recv()

MPI::Finalize()

 

The way it works is every bit of code that is in between the MPI::Init(), and MPI::Finalize() gets executed across your cluster of systems whether they be a bunch of PCs,  a cluster of Tesla™ C1060s , or your BlueGene®/L  installation.  MPI just uses TCP to pass messages via MPI::COMM_WORLD.Send(), and MPI::COMM_WORLD.Recv() around to nodes in the cluster.  From the MPI.Net C# tutorial,

 “Unlike multi-threading, where different threads share the same program state, each of the MPI processes has its own, local program state that cannot be observed or modified by any other process except in response to a message. Therefore, the MPI processes themselves can be as distributed as the network permits, with different processes running on different machines or even different architectures.”  (pg3, MPI.Net C# Tutorial)

A Quick Example:

Provided you’ve got an MPI cluster working (try here, or here for a .Net version for setup instructions) you can run the “Hello World” of MPI:

#include “mpi.h”

#include <stdio.h>

#include <cstdlib>

 

 

int main(int argc, char *argv[]){

 

int nProcs, iMyProc;

MPI::Init( argc, argv );

nProcs = MPI::COMM_WORLD.Get_size();

iMyProc = MPI::COMM_WORLD.Get_rank();

cout << “Hello from process ”;

cout << iMyProc << “ of ”;

cout << nProcs << endl;

MPI::Finalize();

}

 

There’s no instructions being passed explicitly by the little chunk of code above, so there’s no need for calling either of MPI::COMM_WORLD.Send(), or MPI::COMM_WORLD.Recv(), but sending and receiving messages to nodes, or groups of nodes in your cluster is… well, really the whole point of it all, and the Send(), and Recv() calls are how you do it. 

Why would you care at all?

The computing world is getting parallel in a big way (thank you, Google) .  You don’t have to be particularly observant to have noticed this, but it is absolutely true.  MPI has been around for quite awhile, was created by the national labs to run their clusters, and is still used for that purpose specifically to deal with the problem of running multiple program functions across multiple systems on the same data – or different data.  That’s all well and good you say, but I’m not a national lab, so what should I use it for.  My plan is to use it to distribute tests to run across the cluster of Atalasoft engineer’s desktop systems to speed up UI tests, and to create some real load (stay tuned here to see how that adventure goes). 

 

Other Good Links:

Richard Edgar’s introduction to MPI:

http://www.cs264.org/files/IntroMPI.pdf

MPI (the spec):

http://www.mcs.anl.gov/research/projects/mpi/

MPICH2 (good open source implementation):

http://www.mcs.anl.gov/research/projects/mpich2/index.php

MPI.Net:

http://www.osl.iu.edu/research/mpi.net/