[mpich-discuss] Affinity with MPICH_ASYNC_PROGRESS

[Jed Brown]

On Sat, Feb 23, 2013 at 8:21 PM, Jeff Hammond <jhammond at alcf.anl.gov> wrote:

> I personally think that MKL, meaning the BLAS, is an amazing
> programming model.
>

Are you trolling me?


>  However, my point here was merely to say that
> Intel has an integrated software stack that manages thread resources
> intelligently relative to N independent implementations of MPI,
> BLAS/LAPACK, and OpenMP/TBB, for example.  I was under the impression
> that Intel presents stupid oversubscription, which a good thing for
> users but essentially impossible to implement portably.
>

It's relatively easy to be integrated when you're not interoperable.


> I think the only way for async progress to work well is to have
> fine-grain locking since of MPI, as is done in PAMID.  Any
> implementation that resorts to fat-locking is probably better off
> without async progress unless the application is doing something
> really silly (like never calling MPI on a rank that is the target of
> MPI RMA).
>

There's more than 100k cycles between times that I enter the MPI stack.
There are only two threads ever that contend for locks (my funneled thread
and MPICH's async-progress thread). I'm not convinced that you need super
fine-grained locks to make progress during that time period. FWIW, my
experience has been that standard Nemesis does a _much_ better job of
making asynchronous progress than Cray's implementation. If we could look
at the code for Cray's implementation, we might be able to get a better
idea of why.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mpich.org/pipermail/discuss/attachments/20130223/f7512fa4/attachment.html>