<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Hi Erik,</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
It is not just a barrier in MPI_Finalize. It also needs to make sure pending communications are completed. Sometimes this is the user's fault, such as mismatched communication or partially freed communicators. But sometimes this is due to lower-level network
library. For example, some libfabric provider does not always flush send. We tried to detect and prevent all cases that result in hanging, but there always seem to be cases that escape our solution. If you can drill down some simple reproducible (even with
5% chance) cases, please create a github issue and we'll track it down.</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
By the way, if you have pointers on setting up Julia CI testing, we can try to setup nightly testing on our end as well. Catching the errors when it appear always makes troubleshooting easier.<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
-- <br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Hui<br>
</div>
<div id="appendonsend"></div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> Erik Schnetter via discuss <discuss@mpich.org><br>
<b>Sent:</b> Monday, November 29, 2021 12:30 PM<br>
<b>To:</b> discuss@mpich.org <discuss@mpich.org><br>
<b>Cc:</b> Erik Schnetter <schnetter@gmail.com><br>
<b>Subject:</b> [mpich-discuss] Why stuck in MPI_Finalize?</font>
<div> </div>
</div>
<div class="BodyFragment"><font size="2"><span style="font-size:11pt;">
<div class="PlainText">I have a Julia test case on macOS where MPICH randomly gets stuck in<br>
MPI_Finalize (with about a 5% chance). See e.g.<br>
<a href="https://github.com/JuliaParallel/MPI.jl/runs/4357341818">https://github.com/JuliaParallel/MPI.jl/runs/4357341818</a><br>
<br>
Can you advise under what circumstances MPICH could get stuck there?<br>
The respective run uses 3 processes, and all 3 processes call into<br>
MPI_Finalize, but no process returns.<br>
<br>
I assume that MPI_Finalize contains internally the equivalent to an<br>
MPI_Barrier, but that should succeed here. Are there other actions<br>
taken in MPI_Finalize that would require some kind of consistent state<br>
across the application? For example, if a communicator was created on<br>
all processes, but freed only on some processes, could this cause such<br>
a deadlock?<br>
<br>
-erik<br>
<br>
-- <br>
Erik Schnetter <schnetter@gmail.com><br>
<a href="http://www.perimeterinstitute.ca/personal/eschnetter/">http://www.perimeterinstitute.ca/personal/eschnetter/</a><br>
_______________________________________________<br>
discuss mailing list discuss@mpich.org<br>
To manage subscription options or unsubscribe:<br>
<a href="https://lists.mpich.org/mailman/listinfo/discuss">https://lists.mpich.org/mailman/listinfo/discuss</a><br>
</div>
</span></font></div>
</body>
</html>