[mpich-discuss] MPI_Send and recv

Thu Dec 5 14:59:01 CST 2013

Hi, 

I come back to this old discussion, regarding MPI_Send potentially blocking until a matching MPI_Recv is posted. 

I happen to write a master-worker type program where workers periodically send progress status to the master through two MPI_Send. 
The master has an MPI_Irecv posted for the first one and when this one completes, it posts an MPI_Recv for the second one. Those messages being small and the worker not expecting any answer before continuing to work, you can imagine that a systematically blocking MPI_Send could degrade the performance if the master is busy doing something else. 

So my question is: to your knowledge, do implementations of MPI (in particular Mpich, and those running on Cray Linux and BlueGene/P and Q) systematically block on MPI_Send? (I would understand if it blocked because too many MPI_Send have not been matched on the receiver) 

Otherwise I might have to use MPI_Isend everywhere... 

Thanks! 

Matthieu 

----- Mail original -----

De: "Wesley Bland" <wbland at mcs.anl.gov> 
À: discuss at mpich.org 
Envoyé: Mercredi 3 Juillet 2013 15:40:11 
Objet: Re: [mpich-discuss] MPI_Send and recv 

On Jul 2, 2013, at 9:03 PM, Matthieu Dorier < matthieu.dorier at irisa.fr > wrote: 

<blockquote>

Hi, 

These messages piqued my curiosity; 

If I have two processes, process 0 does an MPI_Send to process 1 and eventually, process 1 posts a matching MPI_Recv or MPI_Irecv; does process 0 waits for the MPI_Recv/Irecv to be posted before returning from the MPI_Send? (in other words if process 0 does 2 MPI_Send and process 1 does 2 MPI_Recv, is the linearization "Send; Send; Recv; Recv" possible?) 

It is possible that process 0 could return from the MPI_Send before process 1 posts the corresponding MPI_Recv, but it's not guaranteed. The only thing that is guaranteed from the standard is that when MPI_Send returns, the buffer is available to use again. It is valid to wait for the MPI_Recv to post as well. So to answer your example, S - S - R - R is valid, as is S - R - S - R. 

<blockquote>

Second question: in the situation described by Sufeng, where a process does a MPI_Send to itself, MPI has enough information to see that the process is sending to itself while it has not posted any matching receive, why would MPI block instead of printing an error and aborting? Is it because in a multithreaded program, the matching receive could be posted by another thread? (Yet in that case I always thought it was unsafe to use the same communicator from multiple threads…) 
</blockquote>

There are probably many reasons why MPI doesn't try to do internal error checking. The first is performance. If a program is correct, then checking for all of the cases that might cause a deadlock would be not only very difficult, but also expensive. Therefore, we just say that the user needs to write a correct program and MPI makes no guarantees that anything smart will happen in the background. It's possible that someone could write Nice-MPI that does lots of error checking and verification, but I would think that most users wouldn't want it as it would add overhead. 

Second, it's not always unsafe to use the same communicator in multiple threads. If you use different tags for each thread, then it shouldn't be a problem to communicate between them. 

Wesley 

<blockquote>

Thanks, 

Matthieu 

----- Mail original -----

<blockquote>
De: "Sufeng Niu" < sniu at hawk.iit.edu > 
À: "Pavan Balaji" < balaji at mcs.anl.gov > 
Cc: discuss at mpich.org 
Envoyé: Mardi 2 Juillet 2013 17:20:25 
Objet: Re: [mpich-discuss] MPI_Send and recv 

Thank you so much! Now I make sense 

Sufeng 

On Tue, Jul 2, 2013 at 5:19 PM, Pavan Balaji < balaji at mcs.anl.gov > wrote: 

<blockquote>

On 07/02/2013 05:18 PM, Sufeng Niu wrote: 

<blockquote>
Thanks a lot, that is most confusing part for me in MPI. I would like to 
clear this type of problems. 

So the blocking begins to send message to itself, but there is no 
receive posted. Thus, the MPI block it. Am I right here? 

</blockquote>

Correct. 

<blockquote>
Thus, I need a nonblocking receive to wait the message here. Can I use 
blocking receive? 

</blockquote>

If you use blocking receive, you'll block in the receive; how will you reach the send? 

-- Pavan 

-- 
Pavan Balaji 
http://www.mcs.anl.gov/~balaji 

</blockquote>

-- 
Best Regards, 
Sufeng Niu 
ECASP lab, ECE department, Illinois Institute of Technology 
Tel: 312-731-7219 
_______________________________________________ 
discuss mailing list discuss at mpich.org 
To manage subscription options or unsubscribe: 
https://lists.mpich.org/mailman/listinfo/discuss 
</blockquote>

_______________________________________________ 
discuss mailing list discuss at mpich.org 
To manage subscription options or unsubscribe: 
https://lists.mpich.org/mailman/listinfo/discuss 
</blockquote>

_______________________________________________ 
discuss mailing list discuss at mpich.org 
To manage subscription options or unsubscribe: 
https://lists.mpich.org/mailman/listinfo/discuss 
</blockquote>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mpich.org/pipermail/discuss/attachments/20131205/9bd4d1f3/attachment.html>