<html><head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
I would say yes,.
<div class=""><br class="">
</div>
<div class="">One question about your first example: did you call MPI_Comm_disconnect for both, the inter-com and the intra-com? </div>
<div class=""><br class="">
</div>
<div class=""><br class="">
<div>
<blockquote type="cite" class="">
<div class="">On Feb 29, 2016, at 5:07 PM, K. N. Ramachandran <<a href="mailto:knram06@gmail.com" class="">knram06@gmail.com</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div dir="ltr" class="">Does rank 0 stop here since ranks 0 and 1 are still connected through Comm_World? If so, then the code attached before does not explicitly demonstrate the problem, since I can't disconnect from Comm_World.</div>
<div class="gmail_extra"><br class="">
<div class="gmail_quote">On Mon, Feb 29, 2016 at 6:04 PM, K. N. Ramachandran <span dir="ltr" class="">
<<a href="mailto:knram06@gmail.com" target="_blank" class="">knram06@gmail.com</a>></span> wrote:<br class="">
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr" class="">Hi Halim,
<div class=""><br class="">
</div>
<div class="">Please find attached a minimal example that seems to reproduce this. This is a more simplified version that does not do nameserver lookup.</div>
<div class=""><br class="">
</div>
<div class="">I compiled this as</div>
<div class="">mpicxx -g test_two_procs.c -L /opt/mpich-3.2_install/lib/ -o two_procs</div>
<div class=""><br class="">
</div>
<div class="">Run this with</div>
<div class="">mpiexec -np 2 ./two_procs</div>
<div class=""><br class="">
</div>
<div class="">We can see that rank 0 waits at Finalize, until rank 1 reaches it, even though both rank 0 and rank 1 call MPI_Comm_disconnect. It is my understanding that rank 0 should have finished without having to wait for rank 1.</div>
<div class=""><br class="">
</div>
<div class="">Also MPI_Comm_disconnect cannot be called on MPI_Comm_world.</div>
<div class=""><br class="">
</div>
<div class="">I have another minimal example, that involves nameserver lookup but I think this example should demonstrate it for now. Hope this helps.</div>
<div class=""><br class="">
</div>
<div class="gmail_extra">
<div class="">
<div class="h5"><br class="">
<div class="gmail_quote">On Mon, Feb 29, 2016 at 5:39 PM, Halim Amer <span dir="ltr" class="">
<<a href="mailto:aamer@anl.gov" target="_blank" class="">aamer@anl.gov</a>></span> wrote:<br class="">
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
MPI_Finalize is collective over a set of connected processes. If the server hangs in MPI_Finalize, it means that it is still connected to a subset of the clients. It is difficult to know the reason without a concrete piece of code. If you send us a minimal
example that reproduces the problem, we might be able to identify the issue.<br class="">
<br class="">
--Halim<br class="">
<br class="">
<a href="http://www.mcs.anl.gov/~aamer" rel="noreferrer" target="_blank" class="">www.mcs.anl.gov/~aamer</a><span class=""><br class="">
<br class="">
On 2/29/16 9:00 AM, K. N. Ramachandran wrote:<br class="">
</span>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<span class="">Hello all,<br class="">
<br class="">
I had tried just calling MPI_Comm_disconnect instead of MPI_Comm_split.<br class="">
<br class="">
I had tried this on just the server side, as well as both on the server<br class="">
and client, but I still see the issue of busy-wait at MPI_Finalize on<br class="">
the server side. Can anyone give any further inputs on this?<br class="">
<br class="">
It looks like the server process should be able to terminate early, but<br class="">
is held up by the client, even though they should be disconnected from<br class="">
each other.<br class="">
<br class="">
<br class="">
<br class="">
On Sat, Feb 27, 2016 at 7:00 PM, K. N. Ramachandran <<a href="mailto:knram06@gmail.com" target="_blank" class="">knram06@gmail.com</a><br class="">
</span><span class=""><mailto:<a href="mailto:knram06@gmail.com" target="_blank" class="">knram06@gmail.com</a>>> wrote:<br class="">
<br class="">
Hi Pavan,<br class="">
<br class="">
Thank you for the reply. I have presented only a very simplified<br class="">
case of one server and one client and that is why the problem looks<br class="">
strange.<br class="">
<br class="">
The general case is one server acting as a meeting point and N<br class="">
clients join the server and one intra comm is formed among them all.<br class="">
Then the server splits off and terminates, leaving the intracomm and<br class="">
then letting the clients work amongst themselves now.<br class="">
<br class="">
I had also tried MPI_Comm_disconnect on the server, after calling<br class="">
MPI_Comm_split, but even in that case, the server busy-waits for the<br class="">
client at Finalize. The single server and single client was only to<br class="">
demonstrate the problem I am facing.<br class="">
<br class="">
Please let me know if you might need more information. Thanks.<br class="">
<br class="">
On Sat, Feb 27, 2016 at 11:59 AM, Balaji, Pavan <<a href="mailto:balaji@anl.gov" target="_blank" class="">balaji@anl.gov</a><br class="">
</span><span class=""> <mailto:<a href="mailto:balaji@anl.gov" target="_blank" class="">balaji@anl.gov</a>>> wrote:<br class="">
<br class="">
<br class="">
It's unclear what exactly you are trying to do here. Why are<br class="">
the clients connecting to the server and then immediately<br class="">
"splitting off"?<br class="">
<br class="">
Your "split-off" functionality needs to be implemented using<br class="">
MPI_Comm_disconnect, not using MPI_Comm_split. Comm_split<br class="">
divides a communicator into smaller communicators, but all<br class="">
processes are still very much connected. So as long as the<br class="">
server process is connected to the client processes, it might<br class="">
still receive messages from the client process and thus cannot<br class="">
simply exit. Comm_disconnect, on the other hand, disconnects<br class="">
the client processes from the server processes.<br class="">
<br class="">
But then again, I have no idea why you are connecting to the<br class="">
server and disconnecting immediately.<br class="">
<br class="">
-- Pavan<br class="">
<br class="">
> On Feb 26, 2016, at 5:31 PM, K. N. Ramachandran<br class="">
</span>
<div class="">
<div class=""> <<a href="mailto:knram06@gmail.com" target="_blank" class="">knram06@gmail.com</a> <mailto:<a href="mailto:knram06@gmail.com" target="_blank" class="">knram06@gmail.com</a>>> wrote:<br class="">
><br class="">
> Hello all,<br class="">
><br class="">
> I have recently begun working on a project that uses<br class="">
MPICH-3.2 and I am trying to resolve an issue where a server<br class="">
process busy waits at MPI_Finalize.<br class="">
><br class="">
> We are trying to create a server process that accepts<br class="">
incoming connections from a known number of clients (say, N<br class="">
clients), forms a new communicator amongst everyone (server and<br class="">
clients) and then splits itself from the group and terminates,<br class="">
so that the clients now only work with each other.<br class="">
><br class="">
> For very problem specific reasons, we cannot do<br class="">
> 'mpiexec -np N (other args)'<br class="">
><br class="">
> So we have a server that publishes a service name to a<br class="">
nameserver and clients lookup the name to join the server. The<br class="">
server and client processes are started with separate calls to<br class="">
mpiexec, one to start the server and the rest N calls to start<br class="">
the clients.<br class="">
><br class="">
> The server process busy-waits at the MPI_Finalize call, after<br class="">
it splits from the communicator and only finishes when all other<br class="">
clients reach their MPI_Finalize too.<br class="">
><br class="">
> Consider a simplified case of only one server and one client.<br class="">
The simplified pseudocode is:<br class="">
><br class="">
> Server process:<br class="">
> MPI_Init();<br class="">
> MPI_Open_port(...);<br class="">
> MPI_Publish_name(...); //publish service name to nameserver<br class="">
><br class="">
> MPI_accept(...); // accept incoming connections and store<br class="">
into intercomm<br class="">
> MPI_Intercomm_merge(...); // merge new client into intra-comm<br class="">
><br class="">
> // now split the server from the client<br class="">
> MPI_Comm_rank(intra comm, rank); // rank=0<br class="">
> MPI_Comm_split(intra comm, (rank==0), rank, lone comm);<br class="">
><br class="">
> MPI_Finalize(); // busy-waits here till client's sleep duration<br class="">
><br class="">
> Client process: (simplified - assuming only one client is<br class="">
trying to connect)<br class="">
> MPI_Init();<br class="">
> MPI_Lookup_name(..);<br class="">
> MPI_Connect(...)<br class="">
><br class="">
> // merge<br class="">
> MPI_Intercomm_merge(...); // merge with server<br class="">
><br class="">
> // get rank and split<br class="">
> MPI_Comm_rank(intra comm, rank); // rank=1<br class="">
> MPI_Comm_split(intra comm, rank==0, rank, lone comm);<br class="">
><br class="">
> sleep(10); // sleep for 10 seconds - causes server to busy<br class="">
wait at MPI_Finalize for sleep duration<br class="">
><br class="">
> MPI_Finalize(); // server and client finish here<br class="">
><br class="">
> So my questions are:<br class="">
><br class="">
> 1) Is busy-wait at MPI_Finalize the expected behaviour?<br class="">
><br class="">
> 2) How to truly "disconnect" the server, so that it can end<br class="">
immediately at MPI_Finalize()? I had tried MPI_Comm_disconnect<br class="">
(also MPI_Comm_free) on both the server and client, but that<br class="">
didn't help.<br class="">
><br class="">
> 3) We don't want to see the server process consuming one<br class="">
core at 100% while it waits at MPI_Finalize. Are other<br class="">
alternatives apart from making the server process sleep, wakeup<br class="">
and keep polling a client, and then finally call MPI_Finalize?<br class="">
><br class="">
> Thank you for any inputs that you can give here.<br class="">
><br class="">
><br class="">
> Regards,<br class="">
> K.N.Ramachandran<br class="">
> _______________________________________________<br class="">
</div>
</div>
> discuss mailing list <a href="mailto:discuss@mpich.org" target="_blank" class="">
discuss@mpich.org</a> <mailto:<a href="mailto:discuss@mpich.org" target="_blank" class="">discuss@mpich.org</a>><span class=""><br class="">
> To manage subscription options or unsubscribe:<br class="">
> <a href="https://lists.mpich.org/mailman/listinfo/discuss" rel="noreferrer" target="_blank" class="">
https://lists.mpich.org/mailman/listinfo/discuss</a><br class="">
<br class="">
_______________________________________________<br class="">
</span> discuss mailing list <a href="mailto:discuss@mpich.org" target="_blank" class="">
discuss@mpich.org</a> <mailto:<a href="mailto:discuss@mpich.org" target="_blank" class="">discuss@mpich.org</a>><span class=""><br class="">
To manage subscription options or unsubscribe:<br class="">
<a href="https://lists.mpich.org/mailman/listinfo/discuss" rel="noreferrer" target="_blank" class="">
https://lists.mpich.org/mailman/listinfo/discuss</a><br class="">
<br class="">
<br class="">
<br class="">
<br class="">
Regards,<br class="">
K.N.Ramachandran<br class="">
<br class="">
<br class="">
<br class="">
<br class="">
Regards,<br class="">
K.N.Ramachandran<br class="">
<br class="">
<br class="">
_______________________________________________<br class="">
discuss mailing list <a href="mailto:discuss@mpich.org" target="_blank" class="">discuss@mpich.org</a><br class="">
To manage subscription options or unsubscribe:<br class="">
<a href="https://lists.mpich.org/mailman/listinfo/discuss" rel="noreferrer" target="_blank" class="">https://lists.mpich.org/mailman/listinfo/discuss</a><br class="">
<br class="">
</span></blockquote>
<div class="">
<div class="">_______________________________________________<br class="">
discuss mailing list <a href="mailto:discuss@mpich.org" target="_blank" class="">discuss@mpich.org</a><br class="">
To manage subscription options or unsubscribe:<br class="">
<a href="https://lists.mpich.org/mailman/listinfo/discuss" rel="noreferrer" target="_blank" class="">https://lists.mpich.org/mailman/listinfo/discuss</a><br class="">
</div>
</div>
</blockquote>
</div>
<br class="">
<br clear="all" class="">
<div class=""><br class="">
</div>
</div>
</div>
Thanks,<br class="">
<div class="">
<div dir="ltr" class="">K.N.Ramachandran</div>
</div>
</div>
</div>
</blockquote>
</div>
<br class="">
<br clear="all" class="">
<div class=""><br class="">
</div>
-- <br class="">
<div class="gmail_signature">
<div dir="ltr" class="">K.N.Ramachandran<br class="">
<div class="">Ph: 814-441-4279</div>
</div>
</div>
</div>
_______________________________________________<br class="">
discuss mailing list <a href="mailto:discuss@mpich.org" class="">discuss@mpich.org</a><br class="">
To manage subscription options or unsubscribe:<br class="">
<a href="https://lists.mpich.org/mailman/listinfo/discuss" class="">https://lists.mpich.org/mailman/listinfo/discuss</a></div>
</blockquote>
</div>
<br class="">
</div>
</body>
</html>