<div dir="ltr">I wrote an example of an interposition library that inserts usleep or nanosleep calls at microsecond granularity to blocking MPI operations. I have done minimal testing and not in the relevant circumstances (load-imbalance) so take it with a huge grain of salt. The intent is to be educational.<div><br></div><div><a href="https://github.com/jeffhammond/NiceWait">https://github.com/jeffhammond/NiceWait</a><br></div><div><br></div><div>Jeff</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Jun 4, 2020 at 8:38 AM Jeff Hammond <<a href="mailto:jeff.science@gmail.com">jeff.science@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">You might be able to implement this across the board using <a href="https://github.com/LLNL/wrap" target="_blank">https://github.com/LLNL/wrap</a> and interposing all calls to MPI_Wait transparently rather than changing the application code.<div><br></div><div>Jeff</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Jun 4, 2020 at 8:30 AM Mccall, Kurt E. (MSFC-EV41) via discuss <<a href="mailto:discuss@mpich.org" target="_blank">discuss@mpich.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div lang="EN-US">
<div>
<p class="MsoNormal"><span style="color:rgb(31,73,125)">Thanks Hui. I will use MPI_Iprobe + MPI_Test.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125)"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125)">Kurt<u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125)"><u></u> <u></u></span></p>
<div>
<div style="border-right:none;border-bottom:none;border-left:none;border-top:1pt solid rgb(225,225,225);padding:3pt 0in 0in">
<p class="MsoNormal"><b>From:</b> Zhou, Hui <<a href="mailto:zhouh@anl.gov" target="_blank">zhouh@anl.gov</a>> <br>
<b>Sent:</b> Thursday, June 4, 2020 10:27 AM<br>
<b>To:</b> <a href="mailto:discuss@mpich.org" target="_blank">discuss@mpich.org</a><br>
<b>Cc:</b> Mccall, Kurt E. (MSFC-EV41) <<a href="mailto:kurt.e.mccall@nasa.gov" target="_blank">kurt.e.mccall@nasa.gov</a>><br>
<b>Subject:</b> [EXTERNAL] Re: [mpich-discuss] Controlling MPICH busy-wait behavior<u></u><u></u></p>
</div>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">There is no such option for MPICH. If you configure with ch3:sock, which waits for the progress with blocking `poll`, I think it will not hog the CPU by default, for other configurations, it always busy waits.
<br>
<br>
Is using MPI_Iprobe + MPI_Test an option for you? If your process can afford latency, then you can control your own sleep cycles between MPI_Test. If MPICH will implement such non-busy-wait option, internally it will just do the same test-sleep cycles.
<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<div>
<div>
<p class="MsoNormal">-- <br>
Hui Zhou<u></u><u></u></p>
</div>
</div>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<div style="border-right:none;border-bottom:none;border-left:none;border-top:1pt solid rgb(181,196,223);padding:3pt 0in 0in">
<p class="MsoNormal"><b><span style="font-size:12pt;color:black">From: </span></b><span style="font-size:12pt;color:black">"Mccall, Kurt E. (MSFC-EV41) via discuss" <<a href="mailto:discuss@mpich.org" target="_blank">discuss@mpich.org</a>><br>
<b>Reply-To: </b>"<a href="mailto:discuss@mpich.org" target="_blank">discuss@mpich.org</a>" <<a href="mailto:discuss@mpich.org" target="_blank">discuss@mpich.org</a>><br>
<b>Date: </b>Wednesday, June 3, 2020 at 3:57 PM<br>
<b>To: </b>"<a href="mailto:discuss@mpich.org" target="_blank">discuss@mpich.org</a>" <<a href="mailto:discuss@mpich.org" target="_blank">discuss@mpich.org</a>><br>
<b>Cc: </b>"Mccall, Kurt E. (MSFC-EV41)" <<a href="mailto:kurt.e.mccall@nasa.gov" target="_blank">kurt.e.mccall@nasa.gov</a>><br>
<b>Subject: </b>[mpich-discuss] Controlling MPICH busy-wait behavior<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<p class="MsoNormal">Is there a means to tell MPICH to yield to other processes when performing a busy-wait for incoming messages? The page below distinguishes between “aggressive busy-wait” and “degraded busy-wait”, where the latter is what I’d like to try.<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal"><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__stackoverflow.com_questions_14560714_probe-2Dseems-2Dto-2Dconsume-2Dthe-2Dcpu_14562840-2314562840&d=DwMGaQ&c=ApwzowJNAKKw3xye91w7BE1XMRKi2LN9kiMk5Csz9Zk&r=6cP1IfXu3IZOHSDh_vBqciYiIh4uuVgs1MSi5K7l5fQ&m=stZqd9Fs3P0zmm4350zML37qvC5jT4e9jhayHaxrcbw&s=auPT9hTHN3AOIqD0_f-gKxki1W12bFbMF-6s6AGIL_w&e=" target="_blank">https://stackoverflow.com/questions/14560714/probe-seems-to-consume-the-cpu/14562840#14562840</a><u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">Thanks,<u></u><u></u></p>
<p class="MsoNormal">Kurt<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
</div>
_______________________________________________<br>
discuss mailing list <a href="mailto:discuss@mpich.org" target="_blank">discuss@mpich.org</a><br>
To manage subscription options or unsubscribe:<br>
<a href="https://lists.mpich.org/mailman/listinfo/discuss" rel="noreferrer" target="_blank">https://lists.mpich.org/mailman/listinfo/discuss</a><br>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr">Jeff Hammond<br><a href="mailto:jeff.science@gmail.com" target="_blank">jeff.science@gmail.com</a><br><a href="http://jeffhammond.github.io/" target="_blank">http://jeffhammond.github.io/</a></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature">Jeff Hammond<br><a href="mailto:jeff.science@gmail.com" target="_blank">jeff.science@gmail.com</a><br><a href="http://jeffhammond.github.io/" target="_blank">http://jeffhammond.github.io/</a></div>