<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div class="elementToProof" style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<span style="font-family: Calibri, Helvetica, sans-serif;">I don't think there's any uncompleted communication. The example I show below is one of the very simple examples shipping with MPICH itself (sorry if that wasn't completely clear). </span></div>
<div class="elementToProof" style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<span style="font-family: Calibri, Helvetica, sans-serif;"><br>
</span></div>
<div class="elementToProof" style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<span style="font-family: Calibri, Helvetica, sans-serif;">Cheers,</span></div>
<div class="elementToProof" style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<span style="font-family: Calibri, Helvetica, sans-serif;">Edric.</span></div>
<div id="appendonsend"></div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> Joachim Jenke via discuss <discuss@mpich.org><br>
<b>Sent:</b> 13 December 2023 17:16<br>
<b>To:</b> discuss@mpich.org <discuss@mpich.org>; Zhou, Hui <zhouh@anl.gov><br>
<b>Cc:</b> Joachim Jenke <jenke@itc.rwth-aachen.de><br>
<b>Subject:</b> Re: [mpich-discuss] Hang during MPI_Finalize using ch4:ofi:shm in mpich-4.1.2</font>
<div> </div>
</div>
<div>If your code hangs in MPI_Finalize for certain communication <br>
implementation, this sounds like an uncompleted communication. Are you <br>
sure that you have no MPI communication ongoing when calling MPI_Finalize?<br>
<br>
- Joachim<br>
<br>
Am 13.12.23 um 17:45 schrieb Edric Ellis via discuss:<br>
> Ok, that's good to know, I'll stick with simply "ofi:tcp" for now.<br>
> <br>
> Thanks,<br>
> Edric.<br>
> ------------------------------------------------------------------------<br>
> *From:* Zhou, Hui <zhouh@anl.gov><br>
> *Sent:* 13 December 2023 15:39<br>
> *To:* discuss@mpich.org <discuss@mpich.org><br>
> *Cc:* Edric Ellis <eellis@mathworks.com><br>
> *Subject:* Re: Hang during MPI_Finalize using ch4:ofi:shm in mpich-4.1.2<br>
> Hi Edric,<br>
> <br>
> I am not sure which part is hanging, but you don't need to enable <br>
> |ofi:shm| (libfabric shm provider). The ch4 device comes with its own <br>
> shared memory functionality.<br>
> <br>
> -- <br>
> Hui<br>
> ------------------------------------------------------------------------<br>
> *From:* Edric Ellis via discuss <discuss@mpich.org><br>
> *Sent:* Wednesday, December 13, 2023 7:05 AM<br>
> *To:* discuss@mpich.org <discuss@mpich.org><br>
> *Cc:* Edric Ellis <eellis@mathworks.com><br>
> *Subject:* [mpich-discuss] Hang during MPI_Finalize using ch4:ofi:shm in <br>
> mpich-4.1.2<br>
> I'm working on getting a build of mpich-4.1.2 ready to replace our old <br>
> build of mpich-3.3.2. With older MPICH releases, we used the "nemesis" <br>
> channel via ch3 to provide support for shared-memory configurations as <br>
> well as TCP/IP. In ch4, I thought the nearest equivalent would be:<br>
> <br>
> --with-device=ch4:ofi:tcp,shm<br>
> <br>
> The "tcp" portion of this seems to work just fine, but "shm" hangs <br>
> during (I think) MPI_Finalize, requiring a CTRL-C to kill it. For <br>
> example, in the build area,<br>
> <br>
> $ ./src/pm/hydra/mpiexec.hydra -n 2 ./examples/cpi<br>
> Process 0 of 2 is on uk-eellis-l<br>
> Process 1 of 2 is on uk-eellis-l<br>
> pi is approximately 3.1415926544231318, Error is 0.0000000008333387<br>
> wall clock time = 0.000019<br>
> ^C[mpiexec@uk-eellis-l] Sending Ctrl-C to processes as requested<br>
> [mpiexec@uk-eellis-l] Press Ctrl-C again to force abort<br>
> <br>
> ===================================================================================<br>
> = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES<br>
> = PID 829015 RUNNING AT uk-eellis-l<br>
> = EXIT CODE: 2<br>
> = CLEANING UP REMAINING PROCESSES<br>
> = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES<br>
> ===================================================================================<br>
> YOUR APPLICATION TERMINATED WITH THE EXIT STRING: Interrupt (signal 2)<br>
> This typically refers to a problem with your application.<br>
> Please see the FAQ page for debugging suggestions<br>
> <br>
> Things work fine if I force FI_PROVIDER=tcp. Am I missing something?<br>
> <br>
> Here's the configure line I'm using:<br>
> <br>
> $ ./configure --prefix <prefix> --with-device=ch4:ofi:tcp,shm <br>
> --enable-shared --with-libfabric=embedded --enable-fortran --enable-efa=no<br>
> <br>
> This is running on a Debian 11 system, gcc 10.3.0.<br>
> <br>
> Cheers,<br>
> Edric.<br>
> <br>
> <br>
> _______________________________________________<br>
> discuss mailing list discuss@mpich.org<br>
> To manage subscription options or unsubscribe:<br>
> <a href="https://lists.mpich.org/mailman/listinfo/discuss">
https://lists.mpich.org/mailman/listinfo/discuss</a><br>
<br>
-- <br>
Dr. rer. nat. Joachim Jenke<br>
<br>
IT Center<br>
Group: High Performance Computing<br>
Division: Computational Science and Engineering<br>
RWTH Aachen University<br>
Seffenter Weg 23<br>
D 52074 Aachen (Germany)<br>
Tel: +49 241 80- 24765<br>
Fax: +49 241 80-624765<br>
jenke@itc.rwth-aachen.de<br>
<a href="http://www.itc.rwth-aachen.de">www.itc.rwth-aachen.de</a><br>
<br>
</div>
</body>
</html>