<div dir="ltr">Jeff,<div><br></div><div style>I am not trying to benchmark performance of MPI or <span style="font-family:arial,sans-serif;font-size:12.571428298950195px">beat up on poor old </span><span style="font-family:arial,sans-serif;font-size:12.571428298950195px">bulk-synchronous MPI-1</span>. My goal is to understand which API functions are most commonly used in programs that are representative of MPI programs. </div>
<div style><br></div><div style>I am doing a Ph.D. on systematic testing on concurrent software. In my experience, software developers use concurrency for the sake of performance, but are unaware of the quantitative implications their designs have on the ability to thoroughly test the code they write. Recently, I have been involved in a project that explores the performance-testability trade-off in the context of multi-threaded programs and I am interested in doing the same in the context of MPI programs. As a start I would like to get a better sense of how people use MPI.</div>
<div style><br></div><div style>I hope this puts any worries you might have about unfair performance comparisons to rest and sheds some light on what I mean by a "macro-benchmark".</div><div style><br></div><div style>
Best,</div><div style><br></div><div style>--Jiri</div><div style><br></div><div style><br></div><div style><br></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Fri, Jul 5, 2013 at 1:01 PM, Jeff Hammond <span dir="ltr"><<a href="mailto:jeff.science@gmail.com" target="_blank">jeff.science@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">What is a macrobenchmark? The more complexity you add to a benchmark,<br>
the harder it is to see the details. This is why most MPI benchmarks<br>
are microbenchmarks. As soon as you add complexity in the usage of<br>
MPI, you run into issues with algorithmic variation. For example, all<br>
of the "macrobenchmarks" I have seen (e.g. NAS PB) are written to<br>
MPI-1. They are also quite simple in their use of MPI, relying upon<br>
barrier synchronization when it may not be necessary or appropriate.<br>
You may see papers that show UPC beating MPI for NAS PB, but these<br>
comparisons are apples-to-oranges; if one rewrote the code to use<br>
nonblocking collectives, RMA or other less synchronous approach, the<br>
performance would likely be as good if not better than UPC. In this<br>
respect, one is not so much evaluating MPI performance (which is the<br>
point of an MPI benchmark, no?) but rather evaluating the use of a<br>
simplistic and perhaps overly synchronous programming model that<br>
happens to use MPI-1.<br>
<br>
What is it you really want to do? Do you have some new-fangled<br>
programming model (NFPM) and you want to beat up on poor old<br>
bulk-synchronous MPI-1? If so, just don't do that. Write your own<br>
comparison between NFMP and MPI-3 using all the same algorithmic<br>
tricks that NFMP does. If you want to understand the performance of<br>
MPI in detail when employed by nontrivial applications, then you<br>
should try to find a nontrivial application that uses MPI very<br>
effectively. Unfortunately, many apps use MPI in a simplistic way<br>
because they are not communication limited in the way that<br>
strong-scaled macrobenchmarks are, hence no exotic use of MPI is<br>
required. I find that better algorithms are usually a more effective<br>
way to improve application performance than going OCD on my use of<br>
MPI...<br>
<br>
It is entirely possible that there does not exist anything out there<br>
that you can reuse. However, this is clearly no problem for you since<br>
you are now well-rested and can complete the project by yourself :-)<br>
I am, of course, referencing<br>
<a href="http://www.cs.cmu.edu/~jsimsa/images/dilbert.jpg" target="_blank">http://www.cs.cmu.edu/~jsimsa/images/dilbert.jpg</a> on your home page :-)<br>
<br>
Best,<br>
<br>
Heff<br>
<div><div class="h5"><br>
On Fri, Jul 5, 2013 at 11:22 AM, Pavan Balaji <<a href="mailto:balaji@mcs.anl.gov">balaji@mcs.anl.gov</a>> wrote:<br>
><br>
> On 07/05/2013 10:38 AM, Jiri Simsa wrote:<br>
>><br>
>> Could anyone point me in the direction of programs that are<br>
>> representative of MPI programs? I am looking for something like PARSEC<br>
>> (<a href="http://parsec.cs.princeton.edu/" target="_blank">http://parsec.cs.princeton.edu/</a>) for MPI. In other words, I am<br>
>> interested in macro-bechnmarks not micro-benchmarks. Thank you.<br>
><br>
><br>
> You could try the Sequoia suite. There's also the NAS parallel benchmarks<br>
> and Graph500 benchmarks.<br>
><br>
> -- Pavan<br>
><br>
> --<br>
> Pavan Balaji<br>
> <a href="http://www.mcs.anl.gov/~balaji" target="_blank">http://www.mcs.anl.gov/~balaji</a><br>
</div></div>> _______________________________________________<br>
> discuss mailing list <a href="mailto:discuss@mpich.org">discuss@mpich.org</a><br>
> To manage subscription options or unsubscribe:<br>
> <a href="https://lists.mpich.org/mailman/listinfo/discuss" target="_blank">https://lists.mpich.org/mailman/listinfo/discuss</a><br>
<span class="HOEnZb"><font color="#888888"><br>
<br>
<br>
--<br>
Jeff Hammond<br>
<a href="mailto:jeff.science@gmail.com">jeff.science@gmail.com</a><br>
</font></span></blockquote></div><br></div>