[mpich-discuss] MPI Benchmark Suite

[Jeff Hammond]

What is a macrobenchmark?  The more complexity you add to a benchmark,
the harder it is to see the details.  This is why most MPI benchmarks
are microbenchmarks.  As soon as you add complexity in the usage of
MPI, you run into issues with algorithmic variation.  For example, all
of the "macrobenchmarks" I have seen (e.g. NAS PB) are written to
MPI-1.  They are also quite simple in their use of MPI, relying upon
barrier synchronization when it may not be necessary or appropriate.
You may see papers that show UPC beating MPI for NAS PB, but these
comparisons are apples-to-oranges; if one rewrote the code to use
nonblocking collectives, RMA or other less synchronous approach, the
performance would likely be as good if not better than UPC.  In this
respect, one is not so much evaluating MPI performance (which is the
point of an MPI benchmark, no?) but rather evaluating the use of a
simplistic and perhaps overly synchronous programming model that
happens to use MPI-1.

What is it you really want to do?  Do you have some new-fangled
programming model (NFPM) and you want to beat up on poor old
bulk-synchronous MPI-1?  If so, just don't do that.  Write your own
comparison between NFMP and MPI-3 using all the same algorithmic
tricks that NFMP does.  If you want to understand the performance of
MPI in detail when employed by nontrivial applications, then you
should try to find a nontrivial application that uses MPI very
effectively.  Unfortunately, many apps use MPI in a simplistic way
because they are not communication limited in the way that
strong-scaled macrobenchmarks are, hence no exotic use of MPI is
required.  I find that better algorithms are usually a more effective
way to improve application performance than going OCD on my use of
MPI...

It is entirely possible that there does not exist anything out there
that you can reuse.  However, this is clearly no problem for you since
you are now well-rested and can complete the project by yourself :-)
I am, of course, referencing
http://www.cs.cmu.edu/~jsimsa/images/dilbert.jpg on your home page :-)

Best,

Heff

On Fri, Jul 5, 2013 at 11:22 AM, Pavan Balaji <balaji at mcs.anl.gov> wrote:
>
> On 07/05/2013 10:38 AM, Jiri Simsa wrote:
>>
>> Could anyone point me in the direction of programs that are
>> representative of MPI programs? I am looking for something like PARSEC
>> (http://parsec.cs.princeton.edu/) for MPI. In other words, I am
>> interested in macro-bechnmarks not micro-benchmarks. Thank you.
>
>
> You could try the Sequoia suite.  There's also the NAS parallel benchmarks
> and Graph500 benchmarks.
>
>  -- Pavan
>
> --
> Pavan Balaji
> http://www.mcs.anl.gov/~balaji
> _______________________________________________
> discuss mailing list     discuss at mpich.org
> To manage subscription options or unsubscribe:
> https://lists.mpich.org/mailman/listinfo/discuss



-- 
Jeff Hammond
jeff.science at gmail.com