<div dir="ltr">Hi Hui,<div><br></div><div>Thank you for the clarification.<br><br>Regards,</div><div>Rupsa</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sun, Jan 30, 2022 at 11:08 AM Zhou, Hui <<a href="mailto:zhouh@anl.gov">zhouh@anl.gov</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
Hi Rupsa,</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
The error message could be better. By "Invalid port", it actually meant the connection was timed out. So your 2-second time out actually worked. You should use
<code>MPI_Comm_set_errhandler</code> to set error handler to <code>MPI_ERRORS_RETURN</code>, so that your application can check error and respond accordingly. The default error handler is to abort.</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
As I checked, the current code actually will leak resources when timeout happens. I just created this issue --
<a href="https://github.com/pmodels/mpich/issues/5815" id="gmail-m_8920232771774177740LPlnkOWALinkPreview" target="_blank">https://github.com/pmodels/mpich/issues/5815</a> -- to track it.</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
-- <br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
Hui Zhou<br>
</div>
<div>
<div id="gmail-m_8920232771774177740LPBorder_GTaHR0cHM6Ly9naXRodWIuY29tL3Btb2RlbHMvbXBpY2gvaXNzdWVzLzU4MTU." style="width:100%;margin-top:16px;margin-bottom:16px;max-width:800px;min-width:424px">
<table id="gmail-m_8920232771774177740LPContainer197281" role="presentation" style="padding:12px 36px 12px 12px;width:100%;border-width:1px;border-style:solid;border-color:rgb(200,200,200);border-radius:2px">
<tbody>
<tr style="border-spacing:0px" valign="top">
<td>
<div id="gmail-m_8920232771774177740LPImageContainer197281" style="margin-right:12px;height:120px;overflow:hidden;width:240px">
<a id="gmail-m_8920232771774177740LPImageAnchor197281" href="https://github.com/pmodels/mpich/issues/5815" target="_blank"><img id="gmail-m_8920232771774177740LPThumbnailImageId197281" alt="" style="display: block;" width="240" height="120" src="https://opengraph.githubassets.com/f9215523be8f812ac3716771553a3d691ec7f493ddeb590e541727c7eb0340eb/pmodels/mpich/issues/5815"></a></div>
</td>
<td style="width:100%">
<div id="gmail-m_8920232771774177740LPTitle197281" style="font-size:21px;font-weight:300;margin-right:8px;font-family:wf_segoe-ui_light,"Segoe UI Light","Segoe WP Light","Segoe UI","Segoe WP",Tahoma,Arial,sans-serif;margin-bottom:12px">
<a id="gmail-m_8920232771774177740LPUrlAnchor197281" href="https://github.com/pmodels/mpich/issues/5815" style="text-decoration:none" target="_blank">ch4/ofi: better error handling in MPI_Comm_{accept,connect} · Issue #5815 · pmodels/mpich</a></div>
<div id="gmail-m_8920232771774177740LPDescription197281" style="font-size:14px;max-height:100px;color:rgb(102,102,102);font-family:wf_segoe-ui_normal,"Segoe UI","Segoe WP",Tahoma,Arial,sans-serif;margin-bottom:12px;margin-right:8px;overflow:hidden">
In MPIDI_OFI_dynamic_send and MPIDI_OFI_dynamic_recv, when error happens, we should try cancel the send or recv. Otherwise, we may leave or accumulated unclosed handle in libfabric. Separately, whe...</div>
<div id="gmail-m_8920232771774177740LPMetadata197281" style="font-size:14px;font-weight:400;color:rgb(166,166,166);font-family:wf_segoe-ui_normal,"Segoe UI","Segoe WP",Tahoma,Arial,sans-serif">
<a href="http://github.com" target="_blank">github.com</a></div>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<br>
<div id="gmail-m_8920232771774177740appendonsend"></div>
<hr style="display:inline-block;width:98%">
<div id="gmail-m_8920232771774177740divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> Rupsa Chakraborty <<a href="mailto:c.rupsa@gmail.com" target="_blank">c.rupsa@gmail.com</a>><br>
<b>Sent:</b> Saturday, January 29, 2022 1:29 AM<br>
<b>To:</b> Zhou, Hui <<a href="mailto:zhouh@anl.gov" target="_blank">zhouh@anl.gov</a>>; <a href="mailto:protze@itc.rwth-aachen.de" target="_blank">protze@itc.rwth-aachen.de</a> <<a href="mailto:protze@itc.rwth-aachen.de" target="_blank">protze@itc.rwth-aachen.de</a>><br>
<b>Cc:</b> <a href="mailto:discuss@mpich.org" target="_blank">discuss@mpich.org</a> <<a href="mailto:discuss@mpich.org" target="_blank">discuss@mpich.org</a>><br>
<b>Subject:</b> Re: [mpich-discuss] How to gracefully exit an MPI server process when it is waiting on MPI_Comm_accept</font>
<div> </div>
</div>
<div>
<div dir="ltr">Hi Hui, Joachim,
<div><br>
</div>
<div>Thanks for the information. </div>
<div>I am not able to find in the internet a clear example as to how to populate MPI_Info and use it with MPI_Comm_accept. I tried to do the following, but this is giving errors as shown below.<br>
Could you provide a small example to set up a timeout with MPI_Comm_accept? Do I need to populate port info too with MPI_Info?<br>
<br>
MPI_Init(NULL, NULL); <br>
char portName[MPI_MAX_PORT_NAME]; <br>
MPI_Open_port(MPI_INFO_NULL, portName); <br>
<br>
MPI_Info info;<br>
MPI_Info_create( &info );<br>
MPI_Info_set(info, "timeout", "2");<br>
<br>
MPI_Comm intercomm;<br>
MPI_Comm_accept(portName, info, 0, MPI_COMM_SELF, &intercomm); </div>
<div> <br>
</div>
<div>Error:<br>
$ ./server <br>
Waiting to accept.... <br>
Abort(671693606) on node 0 (rank 0 in comm 0): Fatal error in internal_Comm_accept: Invalid port, error stack:<br>
internal_Comm_accept(102)....: MPI_Comm_accept(port_name=tag#0$connentry#020098ADAC1785B70000000000000000$, info=0x9c000002, 0, MPI_COMM_SELF, newcomm=0x7f79b17dbcb8) failed<br>
MPID_Comm_accept(442)........: <br>
dynamic_intercomm_create(400): <br>
peer_intercomm_create(334)...: <br>
(unknown)(): Invalid port<br>
</div>
<div><br>
</div>
<div>Regards,</div>
<div>Rupsa</div>
</div>
<br>
<div>
<div dir="ltr">On Fri, Jan 28, 2022 at 8:47 PM Joachim Protze <<a href="mailto:protze@itc.rwth-aachen.de" target="_blank">protze@itc.rwth-aachen.de</a>> wrote:<br>
</div>
<blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
Hi,<br>
<br>
since you have a separate thread for the accept, you could just set a <br>
flag at the main thread, open a connection from the main thread to match <br>
the accept. The accept thread sees the flag, immediately closes the <br>
connection and leaves the while loop.<br>
<br>
- Joachim<br>
<br>
Am 28.01.22 um 05:54 schrieb Rupsa Chakraborty via discuss:<br>
> <br>
> Hello,<br>
> <br>
> I am asking this question in this forum as I had already asked this <br>
> question in stack overflow and did not receive any answer/comment:<br>
> <a href="https://stackoverflow.com/questions/70745740/how-to-gracefully-exit-an-mpi-server-process-when-it-is-waiting-on-mpi-comm-acce" rel="noreferrer" target="_blank">
https://stackoverflow.com/questions/70745740/how-to-gracefully-exit-an-mpi-server-process-when-it-is-waiting-on-mpi-comm-acce</a>
<br>
> <<a href="https://stackoverflow.com/questions/70745740/how-to-gracefully-exit-an-mpi-server-process-when-it-is-waiting-on-mpi-comm-acce" rel="noreferrer" target="_blank">https://stackoverflow.com/questions/70745740/how-to-gracefully-exit-an-mpi-server-process-when-it-is-waiting-on-mpi-comm-acce</a>><br>
> <br>
> My concern is as follows:<br>
> I have an MPI server program that calls MPI_Comm_accept, in an infinite <br>
> loop, on a separate thread. The main thread spawns this thread and in <br>
> parallel does some other work. At some point of time the main thread <br>
> decides to exit. It closes the port using MPI_Close_port(portName), <br>
> however the MPI_Comm_accept() is still waiting for connection requests.<br>
> <br>
> <br>
> My question is:<br>
> - How does the main thread exit gracefully? Is there an equivalent of <br>
> socket close in MPI? I am using mpich which I can use from main thread?<br>
> <br>
> - Is there anyway I can set timeout to the MPI_Comm_accept call?<br>
> <br>
> Here is a dummy code that looks similar to my server code:<br>
> <br>
> |// MPI Server Program void accept(std::string portName) { while (true) <br>
> { MPI_Comm intercomm; MPI_Comm_accept(portName.c_str(), MPI_INFO_NULL, <br>
> 0, MPI_COMM_SELF, &intercomm); // handle connection <br>
> MPI_Comm_disconnect(&intercomm); } } int main() { char ch; <br>
> MPI_Init(NULL, NULL); char portName[MPI_MAX_PORT_NAME]; <br>
> MPI_Open_port(MPI_INFO_NULL, portName); <br>
> publishName("nameServerFile.txt", "ocean", std::string(portName)); <br>
> std::thread th(&accept, std::string(portName)); // do something <br>
> MPI_Close_port(portName); std::cout <<"\nClosed port" << std::endl; <br>
> th.join(); // main thread waits infinitely at this join // as child <br>
> thread is still waiting on MPI_Comm_accept MPI_Finalize(); }|<br>
> <br>
> <br>
> <br>
> _______________________________________________<br>
> discuss mailing list <a href="mailto:discuss@mpich.org" target="_blank">discuss@mpich.org</a><br>
> To manage subscription options or unsubscribe:<br>
> <a href="https://lists.mpich.org/mailman/listinfo/discuss" rel="noreferrer" target="_blank">
https://lists.mpich.org/mailman/listinfo/discuss</a><br>
<br>
<br>
-- <br>
Dr. rer. nat. Joachim Protze<br>
<br>
IT Center<br>
Group: High Performance Computing<br>
Division: Computational Science and Engineering<br>
RWTH Aachen University<br>
Seffenter Weg 23<br>
D 52074 Aachen (Germany)<br>
Tel: +49 241 80- 24765<br>
Fax: +49 241 80-624765<br>
<a href="mailto:protze@itc.rwth-aachen.de" target="_blank">protze@itc.rwth-aachen.de</a><br>
<a href="http://www.itc.rwth-aachen.de" rel="noreferrer" target="_blank">www.itc.rwth-aachen.de</a><br>
</blockquote>
</div>
</div>
</div>
</blockquote></div>