<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><br class=""><div><br class=""><blockquote type="cite" class=""><div class="">On May 10, 2020, at 3:16 AM, Martin Ivanov <<a href="mailto:marto1980@gmail.com" class="">marto1980@gmail.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class="">Hello Tony, <div class="">Thank you very much for your reply. I am posting you the output of 'ps x', when '<span style="font-family: monospace;" class="">mpirun -n 5 mpich-3.0.4/examples/hellow' freezes:</span></div><div class=""><span style="font-family: monospace;" class=""><br class=""></span></div><div class=""><span style="font-family: monospace;" class="">After that freeze, I killed mpirun with Ctrl + C:</span></div><div class=""><font face="monospace" class="">"</font></div><div class=""><span style="font-family:monospace" class=""><span style="" class="">marto@dragonfly% mpirun -n 5 mpich-3.0.4/examples/hellow </span><br class="">^C[mpiexec@dragonfly] Sending Ctrl-C to processes as requested
<br class="">[mpiexec@dragonfly] Press Ctrl-C again to force abort
<br class="">[proxy:0:0@dragonfly] HYDT_dmxu_poll_wait_for_event (./tools/demux/demux_poll.c:71): assert (!(pollfds[i].revents & ~POLLIN & ~POLLOUT & ~POLLHUP & ~POLLERR)) failed
<br class="">[proxy:0:0@dragonfly] main (./pm/pmiserv/pmip.c:206): demux engine error waiting for event
<br class="">[mpiexec@dragonfly] control_cb (./pm/pmiserv/pmiserv_cb.c:202): assert (!closed) failed
<br class="">[mpiexec@dragonfly] HYDT_dmxu_poll_wait_for_event (./tools/demux/demux_poll.c:77): callback returned error status
<br class="">[mpiexec@dragonfly] HYD_pmci_wait_for_completion (./pm/pmiserv/pmiserv_pmci.c:197): error waiting for event
<br class="">[mpiexec@dragonfly] main (./ui/mpich/mpiexec.c:331): process manager error waiting for completion<br class=""></span></div><div class=""><font face="monospace" class="">"</font></div><div class=""><font face="monospace" class=""><br class=""></font></div><div class=""><font face="monospace" class="">The next relaunch of mpirun with 5 cores was successful. For completeness, I am attaching the output of 'ps x' after mpirun with 2 cores freezes, which it actually with 2 cores always does.</font></div><div class=""><font face="monospace" class=""><br class=""></font></div><div class=""><font face="monospace" class="">I hope this was helpful. I am looking forward to your reply.</font></div><div class=""><font face="monospace" class=""><br class=""></font></div></div></div></blockquote><br class=""></div><div>Hi,</div><div><br class=""></div><div>Well, I took this as an opportunity to play with dfly again, so I replicated mpich 3.0.4 and see much the same behavior. Mpirun -n 2 hangs consistently, -n 4 works reliably (= # system cores), -n 5 sometimes.</div><div><br class=""></div><div>I’m not an mpich developer so I will leave further prognostication to their capable hands, but your suspicions about hwloc from the warning messages seem to be well-founded.</div><div><br class=""></div><div>Tony</div><div><br class=""></div></body></html>