[mpich-discuss] MPI_Send and recv

Fri Dec 6 08:22:47 CST 2013

That's another idea. The messages I'm talking about are less than 256 bytes.

Matthieu

----- Mail original -----
> De: "Jeff Hammond" <jeff.science at gmail.com>
> À: discuss at mpich.org
> Envoyé: Vendredi 6 Décembre 2013 14:10:15
> Objet: Re: [mpich-discuss] MPI_Send and recv
> 
> For large messages, MPI_Send may behave like MPI_Ssend on Blue Gene
> and probably other platforms.  You have to specify the message sizes
> your interested in for me to provide more accurate information.
> 
> If you don't want data transfer to block, just use very small
> Send-Recv for control and move data with RMA.
> 
> Jeff
> 
> On Fri, Dec 6, 2013 at 12:46 AM, Matthieu Dorier
> <matthieu.dorier at irisa.fr> wrote:
> > Thanks Jeff for your answer.
> > By "systematically" I mean "whether a remote buffer is full or not"
> > (i.e.
> > "every time"), as opposed to an MPI_Send that "may block under some
> > conditions, but not always".
> >
> > Matthieu
> >
> > ________________________________
> >
> > De: "Jeff Hammond" <jeff.science at gmail.com>
> > À: discuss at mpich.org
> > Envoyé: Vendredi 6 Décembre 2013 03:34:00
> >
> > Objet: Re: [mpich-discuss] MPI_Send and recv
> >
> > Blue Gene does not always block on mpi_send. I don't know what you
> > mean by
> > "systematically"...
> >
> > If you are sending small messages (less than ~1KB), they will use
> > eager
> > protocol and not block until you blow up the remote buffer.
> >
> > Rendezvous very likely blocks until the recv is posted.
> >
> > There are slides online about Cray MPI protocols that may show
> > similar
> > behavior.
> >
> > If you want to be sure you don't block on send, you need to use
> > isend. That
> > much should be obvious.
> >
> > Jeff
> >
> > Sent from my iPhone
> >
> > On Dec 5, 2013, at 2:59 PM, Matthieu Dorier
> > <matthieu.dorier at irisa.fr>
> > wrote:
> >
> > Hi,
> >
> > I come back to this old discussion, regarding MPI_Send potentially
> > blocking
> > until a matching MPI_Recv is posted.
> >
> > I happen to write a master-worker type program where workers
> > periodically
> > send progress status to the master through two MPI_Send.
> > The master has an MPI_Irecv posted for the first one and when this
> > one
> > completes, it posts an MPI_Recv for the second one. Those messages
> > being
> > small and the worker not expecting any answer before continuing to
> > work, you
> > can imagine that a systematically blocking MPI_Send could degrade
> > the
> > performance if the master is busy doing something else.
> >
> > So my question is: to your knowledge, do implementations of MPI (in
> > particular Mpich, and those running on Cray Linux and BlueGene/P
> > and Q)
> > systematically block on MPI_Send? (I would understand if it blocked
> > because
> > too many MPI_Send have not been matched on the receiver)
> >
> > Otherwise I might have to use MPI_Isend everywhere...
> >
> > Thanks!
> >
> > Matthieu
> >
> > ________________________________
> >
> > De: "Wesley Bland" <wbland at mcs.anl.gov>
> > À: discuss at mpich.org
> > Envoyé: Mercredi 3 Juillet 2013 15:40:11
> > Objet: Re: [mpich-discuss] MPI_Send and recv
> >
> > On Jul 2, 2013, at 9:03 PM, Matthieu Dorier
> > <matthieu.dorier at irisa.fr>
> > wrote:
> >
> > Hi,
> >
> > These messages piqued my curiosity;
> >
> > If I have two processes, process 0 does an MPI_Send to process 1
> > and
> > eventually, process 1 posts a matching MPI_Recv or MPI_Irecv; does
> > process 0
> > waits for the MPI_Recv/Irecv to be posted before returning from the
> > MPI_Send? (in other words if process 0 does 2 MPI_Send and process
> > 1 does 2
> > MPI_Recv, is the linearization "Send; Send; Recv; Recv" possible?)
> >
> >
> > It is possible that process 0 could return from the MPI_Send before
> > process
> > 1 posts the corresponding MPI_Recv, but it's not guaranteed. The
> > only thing
> > that is guaranteed from the standard is that when MPI_Send returns,
> > the
> > buffer is available to use again.  It is valid to wait for the
> > MPI_Recv to
> > post as well. So to answer your example, S - S - R - R is valid, as
> > is S - R
> > - S - R.
> >
> >
> > Second question: in the situation described by Sufeng, where a
> > process does
> > a MPI_Send to itself, MPI has enough information to see that the
> > process is
> > sending to itself while it has not posted any matching receive, why
> > would
> > MPI block instead of printing an error and aborting? Is it because
> > in a
> > multithreaded program, the matching receive could be posted by
> > another
> > thread? (Yet in that case I always thought it was unsafe to use the
> > same
> > communicator from multiple threads…)
> >
> >
> > There are probably many reasons why MPI doesn't try to do internal
> > error
> > checking. The first is performance. If a program is correct, then
> > checking
> > for all of the cases that might cause a deadlock would be not only
> > very
> > difficult, but also expensive. Therefore, we just say that the user
> > needs to
> > write a correct program and MPI makes no guarantees that anything
> > smart will
> > happen in the background. It's possible that someone could write
> > Nice-MPI
> > that does lots of error checking and verification, but I would
> > think that
> > most users wouldn't want it as it would add overhead.
> >
> > Second, it's not always unsafe to use the same communicator in
> > multiple
> > threads. If you use different tags for each thread, then it
> > shouldn't be a
> > problem to communicate between them.
> >
> > Wesley
> >
> >
> > Thanks,
> >
> > Matthieu
> >
> > ________________________________
> >
> > De: "Sufeng Niu" <sniu at hawk.iit.edu>
> > À: "Pavan Balaji" <balaji at mcs.anl.gov>
> > Cc: discuss at mpich.org
> > Envoyé: Mardi 2 Juillet 2013 17:20:25
> > Objet: Re: [mpich-discuss] MPI_Send and recv
> >
> > Thank you so much! Now I make sense
> >
> > Sufeng
> >
> >
> > On Tue, Jul 2, 2013 at 5:19 PM, Pavan Balaji <balaji at mcs.anl.gov>
> > wrote:
> >>
> >>
> >> On 07/02/2013 05:18 PM, Sufeng Niu wrote:
> >>>
> >>> Thanks a lot, that is most confusing part for me in MPI. I would
> >>> like to
> >>> clear this type of problems.
> >>>
> >>> So the blocking begins to send message to itself, but there is no
> >>> receive posted. Thus, the MPI block it. Am I right here?
> >>
> >>
> >> Correct.
> >>
> >>
> >>> Thus, I need a nonblocking receive to wait the message here. Can
> >>> I use
> >>> blocking receive?
> >>
> >>
> >> If you use blocking receive, you'll block in the receive; how will
> >> you
> >> reach the send?
> >>
> >>  -- Pavan
> >>
> >>
> >> --
> >> Pavan Balaji
> >> http://www.mcs.anl.gov/~balaji
> >
> >
> >
> >
> > --
> > Best Regards,
> > Sufeng Niu
> > ECASP lab, ECE department, Illinois Institute of Technology
> > Tel: 312-731-7219
> >
> > _______________________________________________
> > discuss mailing list     discuss at mpich.org
> > To manage subscription options or unsubscribe:
> > https://lists.mpich.org/mailman/listinfo/discuss
> >
> >
> > _______________________________________________
> > discuss mailing list     discuss at mpich.org
> > To manage subscription options or unsubscribe:
> > https://lists.mpich.org/mailman/listinfo/discuss
> >
> >
> >
> > _______________________________________________
> > discuss mailing list     discuss at mpich.org
> > To manage subscription options or unsubscribe:
> > https://lists.mpich.org/mailman/listinfo/discuss
> >
> >
> > _______________________________________________
> > discuss mailing list     discuss at mpich.org
> > To manage subscription options or unsubscribe:
> > https://lists.mpich.org/mailman/listinfo/discuss
> >
> >
> > _______________________________________________
> > discuss mailing list     discuss at mpich.org
> > To manage subscription options or unsubscribe:
> > https://lists.mpich.org/mailman/listinfo/discuss
> >
> >
> >
> > _______________________________________________
> > discuss mailing list     discuss at mpich.org
> > To manage subscription options or unsubscribe:
> > https://lists.mpich.org/mailman/listinfo/discuss
> 
> 
> 
> --
> Jeff Hammond
> jeff.science at gmail.com
> _______________________________________________
> discuss mailing list     discuss at mpich.org
> To manage subscription options or unsubscribe:
> https://lists.mpich.org/mailman/listinfo/discuss