[mpich-discuss] determinism in collectives

Dave Goodell goodell at mcs.anl.gov
Tue Nov 13 12:17:36 CST 2012

On Nov 12, 2012, at 9:00 PM CST, Nichols A. Romero wrote:

> Are collective operations, such as sums, using MPI_Reduce deterministic? In other words, if I run MPI_Reduce N times on P tasks, will the sums always occur in the same order. Thus it would be reproducible to the expected precision.

In practice, this usually holds, and it's even recommended by the MPI standard (see MPI-3, sec. 5.9, p.175, l.9-13).  However, you may not be able to capture all inputs to a collective routine, and it's entirely reasonable for future versions of MPICH (or any MPI library) to attempt to use adaptive methods for selecting collective algorithms.  Such adaptive behavior could easily disturb the commutivity/associativity of your reduction collective.


More information about the discuss mailing list