[mpich-discuss] In-order messages

Thakur, Rajeev thakur at anl.gov
Thu Mar 12 13:18:36 CDT 2020

Matching specifies which buffer the incoming message goes into. Two sends issued in order with the same tag and destination will match two receives issued in order with the same tag or MPI_ANY_TAG at the destination. The two sends could take different paths through the network, one more congested than the other, and hence reach the destination out of order. But they will get placed in the intended buffers.


From: "Larson, Jeffrey M." <jmlarson at anl.gov>
Date: Thursday, March 12, 2020 at 1:12 PM
To: "Thakur, Rajeev" <thakur at anl.gov>, "discuss at mpich.org" <discuss at mpich.org>
Cc: "Navarro, John-Luke Nicolas" <jnavarro at anl.gov>, "Hudson, Stephen Tobias P" <shudson at anl.gov>
Subject: Re: [mpich-discuss] In-order messages

On 3/12/20 1:04 PM, Thakur, Rajeev wrote:
The standard specifies the order in which messages match, not complete. They can complete in any order.


From: "Larson, Jeffrey M. via discuss" <discuss at mpich.org><mailto:discuss at mpich.org>
Reply-To: "discuss at mpich.org"<mailto:discuss at mpich.org> <discuss at mpich.org><mailto:discuss at mpich.org>
Date: Thursday, March 12, 2020 at 12:46 PM
To: "discuss at mpich.org"<mailto:discuss at mpich.org> <discuss at mpich.org><mailto:discuss at mpich.org>
Cc: "Larson, Jeffrey M." <jmlarson at anl.gov><mailto:jmlarson at anl.gov>, "Navarro, John-Luke Nicolas" <jnavarro at anl.gov><mailto:jnavarro at anl.gov>, "Hudson, Stephen Tobias P" <shudson at anl.gov><mailto:shudson at anl.gov>
Subject: [mpich-discuss] In-order messages

Hello MPICH friends,

Consider the simple two-rank MPI scenario:

  *   Rank 1 is doing calculations and giving chunks of data to rank 0 using nonblocking sends and tag=0.
  *   When rank 1 is finished, it will send it's last data (or no data) with tag=1.
  *   Rank 0 is using probes to see when data is ready to be received. It receives with any tag, knowing when to stop receiving, or give new data when a tag=0 is received.
Is it possible that rank 0 receives a tag=1 message when there are outstanding tag=0 messages?

Looking at section 3.5 of the MPI standard lets me know that

"Messages are non-overtaking: If a sender sends two messages in succession to the same destination, and both match the same receive, then this operation cannot receive the second message if the first one is still pending."

But I'm not sure if this applies to the above case. Is anytag "the same receive"?
If rank 1 puts data in its buffer, doesn't the network have to be used to communicate that to the buffer of rank 0?
While rank 1 is putting data into its buffer in order, is it possible that a tiny tag=1 message is registered in the rank 0 buffer before a massive tag=0 message?

Thank you for your help,

Hi Rajeev,

What is the difference between matching and completing?

Does this mean that the quote from the standard that I gave means the messages between two fixed ranks could overtake each other? That is, they "match" but don't "complete" in the same order?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mpich.org/pipermail/discuss/attachments/20200312/abdd1cc4/attachment-0001.html>

More information about the discuss mailing list