<meta http-equiv="Content-Type" content="text/html; charset=utf-8"><div class="socmaildefaultfont" dir="ltr" style="font-family:Arial, Helvetica, sans-serif;font-size:10.5pt"><div dir="ltr"><div class="votecell post-layout--left" style="margin: 0px; padding: 0px 15px 0px 0px; border: 0px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-variant-numeric: inherit; font-variant-east-asian: inherit; font-weight: 400; font-stretch: inherit; font-size: 13px; line-height: inherit; font-family: Arial, "Helvetica Neue", Helvetica, sans-serif; vertical-align: top; box-sizing: inherit; flex-grow: 0; width: auto; grid-column: 1 / auto; color: rgb(36, 39, 41); letter-spacing: normal; orphans: 2; text-align: left; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-style: initial; text-decoration-color: initial;"><strong style="margin: 0px; padding: 0px; border: 0px; font-style: inherit; font-variant: inherit; font-weight: bold; font-stretch: inherit; font-size: inherit; line-height: inherit; font-family: inherit; vertical-align: baseline; box-sizing: inherit;">Storytime!</strong><span> </span>Consider the following scenario:</div>
<div class="postcell post-layout--right" style="margin: 0px; padding: 0px; border: 0px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-variant-numeric: inherit; font-variant-east-asian: inherit; font-weight: 400; font-stretch: inherit; font-size: 13px; line-height: inherit; font-family: Arial, "Helvetica Neue", Helvetica, sans-serif; vertical-align: top; box-sizing: inherit; flex-shrink: 1; width: auto; min-width: 0px; grid-column: 2 / auto; color: rgb(36, 39, 41); letter-spacing: normal; orphans: 2; text-align: left; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-style: initial; text-decoration-color: initial;"><div class="post-text" itemprop="text" style="margin: 0px 0px 5px; padding: 0px; border: 0px; font-style: inherit; font-variant: inherit; font-weight: inherit; font-stretch: inherit; font-size: 15px; line-height: 1.3; font-family: inherit; vertical-align: baseline; box-sizing: inherit; width: 667.003px; word-wrap: break-word;"><ol style="margin: 0px 0px 1em 30px; padding: 0px; border: 0px; font-style: inherit; font-variant: inherit; font-weight: inherit; font-stretch: inherit; font-size: inherit; line-height: inherit; font-family: inherit; vertical-align: baseline; list-style: decimal; box-sizing: inherit;"> <li style="margin: 0px 0px 0.5em; padding: 0px; border: 0px; font-style: inherit; font-variant: inherit; font-weight: inherit; font-stretch: inherit; font-size: inherit; line-height: inherit; font-family: inherit; vertical-align: baseline; box-sizing: inherit; word-wrap: break-word;">Using Hydra, MPICH spawns 2 different processes (Simulators). Call them Apple and Orange!</li> <li style="margin: 0px 0px 0.5em; padding: 0px; border: 0px; font-style: inherit; font-variant: inherit; font-weight: inherit; font-stretch: inherit; font-size: inherit; line-height: inherit; font-family: inherit; vertical-align: baseline; box-sizing: inherit; word-wrap: break-word;">Apple and Orange start, they load a dynamically linked library, and they use that library to call MPI_Init and do all MPI communications. (That DLM is what I make)</li> <li style="margin: 0px 0px 0.5em; padding: 0px; border: 0px; font-style: inherit; font-variant: inherit; font-weight: inherit; font-stretch: inherit; font-size: inherit; line-height: inherit; font-family: inherit; vertical-align: baseline; box-sizing: inherit; word-wrap: break-word;">Sometime later we initiate a checkpoint</li> <li style="margin: 0px 0px 0.5em; padding: 0px; border: 0px; font-style: inherit; font-variant: inherit; font-weight: inherit; font-stretch: inherit; font-size: inherit; line-height: inherit; font-family: inherit; vertical-align: baseline; box-sizing: inherit; word-wrap: break-word;">Apple saves everything it needs into a file which it loads on the restart</li> <li style="margin: 0px 0px 0.5em; padding: 0px; border: 0px; font-style: inherit; font-variant: inherit; font-weight: inherit; font-stretch: inherit; font-size: inherit; line-height: inherit; font-family: inherit; vertical-align: baseline; box-sizing: inherit; word-wrap: break-word;">Orange, however, saves the whole Linux process state and loads it back up at the restart.</li> <li style="margin: 0px 0px 0.5em; padding: 0px; border: 0px; font-style: inherit; font-variant: inherit; font-weight: inherit; font-stretch: inherit; font-size: inherit; line-height: inherit; font-family: inherit; vertical-align: baseline; box-sizing: inherit; word-wrap: break-word;">Now later, we want to load the simulation from the checkpoint to run it with different parameters.</li> <li style="margin: 0px 0px 0.5em; padding: 0px; border: 0px; font-style: inherit; font-variant: inherit; font-weight: inherit; font-stretch: inherit; font-size: inherit; line-height: inherit; font-family: inherit; vertical-align: baseline; box-sizing: inherit; word-wrap: break-word;">We call Hydra again with the new parameters to start Apple and Orange</li> <li style="margin: 0px 0px 0.5em; padding: 0px; border: 0px; font-style: inherit; font-variant: inherit; font-weight: inherit; font-stretch: inherit; font-size: inherit; line-height: inherit; font-family: inherit; vertical-align: baseline; box-sizing: inherit; word-wrap: break-word;">Apple comes up, loads its stuff, loads my library, and calls MPI_Init (Allgood)</li> <li style="margin: 0px; padding: 0px; border: 0px; font-style: inherit; font-variant: inherit; font-weight: inherit; font-stretch: inherit; font-size: inherit; line-height: inherit; font-family: inherit; vertical-align: baseline; box-sizing: inherit; word-wrap: break-word;"> <p style="margin: 0px 0px 1em; padding: 0px; border: 0px; font-style: inherit; font-variant: inherit; font-weight: inherit; font-stretch: inherit; font-size: inherit; line-height: inherit; font-family: inherit; vertical-align: baseline; box-sizing: inherit; clear: both;">Pear loads the whole old state back up! Nice, except that the MPI state variables get loaded too as a part of Process state, so the MPI in orange thinks that it has already been initialized, but it doesn't know about this new network and its variables are for an old MPI network from the first simulation.</p>
<p style="margin: 0px; padding: 0px; border: 0px; font-style: inherit; font-variant: inherit; font-weight: inherit; font-stretch: inherit; font-size: inherit; line-height: inherit; font-family: inherit; vertical-align: baseline; box-sizing: inherit; clear: both;"><a href="https://i.stack.imgur.com/yAXq0.png" rel="nofollow noreferrer" style="margin: 0px; padding: 0px; border: 0px; font-style: inherit; font-variant: inherit; font-weight: inherit; font-stretch: inherit; font-size: inherit; line-height: inherit; font-family: inherit; vertical-align: baseline; box-sizing: inherit; color: rgb(0, 89, 153); text-decoration: none; cursor: pointer;">This is a diagram for visualization</a></p> </li></ol>
<p style="margin: 0px 0px 1em; padding: 0px; border: 0px; font-style: inherit; font-variant: inherit; font-weight: inherit; font-stretch: inherit; font-size: inherit; line-height: inherit; font-family: inherit; vertical-align: baseline; box-sizing: inherit; clear: both;"><strong style="margin: 0px; padding: 0px; border: 0px; font-style: inherit; font-variant: inherit; font-weight: bold; font-stretch: inherit; font-size: inherit; line-height: inherit; font-family: inherit; vertical-align: baseline; box-sizing: inherit;">So here's my problem now:</strong></p>
<p style="margin: 0px 0px 1em; padding: 0px; border: 0px; font-style: inherit; font-variant: inherit; font-weight: inherit; font-stretch: inherit; font-size: inherit; line-height: inherit; font-family: inherit; vertical-align: baseline; box-sizing: inherit; clear: both;">How can I tell Orange to call MPI_Init again (or any other way really since I know duplicate calls to mpi_init is disallowed) so it can get to know of the new network?</p>
<p style="margin: 0px 0px 1em; padding: 0px; border: 0px; font-style: inherit; font-variant: inherit; font-weight: inherit; font-stretch: inherit; font-size: inherit; line-height: inherit; font-family: inherit; vertical-align: baseline; box-sizing: inherit; clear: both;"><strong style="margin: 0px; padding: 0px; border: 0px; font-style: inherit; font-variant: inherit; font-weight: bold; font-stretch: inherit; font-size: inherit; line-height: inherit; font-family: inherit; vertical-align: baseline; box-sizing: inherit;">Facts</strong>:</p>
<ol style="margin: 0px 0px 1em 30px; padding: 0px; border: 0px; font-style: inherit; font-variant: inherit; font-weight: inherit; font-stretch: inherit; font-size: inherit; line-height: inherit; font-family: inherit; vertical-align: baseline; list-style: decimal; box-sizing: inherit;"> <li style="margin: 0px 0px 0.5em; padding: 0px; border: 0px; font-style: inherit; font-variant: inherit; font-weight: inherit; font-stretch: inherit; font-size: inherit; line-height: inherit; font-family: inherit; vertical-align: baseline; box-sizing: inherit; word-wrap: break-word;">I can edit my library all I want. They are c++ libraries.</li> <li style="margin: 0px 0px 0.5em; padding: 0px; border: 0px; font-style: inherit; font-variant: inherit; font-weight: inherit; font-stretch: inherit; font-size: inherit; line-height: inherit; font-family: inherit; vertical-align: baseline; box-sizing: inherit; word-wrap: break-word;">I cannot edit how the Apple and Orange simulators save the checkpoints or restart. They are proprietary simulators with no source code available.</li> <li style="margin: 0px; padding: 0px; border: 0px; font-style: inherit; font-variant: inherit; font-weight: inherit; font-stretch: inherit; font-size: inherit; line-height: inherit; font-family: inherit; vertical-align: baseline; box-sizing: inherit; word-wrap: break-word;">Simulation run over days, so it is essential to run the common part first, and then save a checkpoint as a starting point for other simulations.</li></ol>
<p style="margin: 0px 0px 1em; padding: 0px; border: 0px; font-style: inherit; font-variant: inherit; font-weight: inherit; font-stretch: inherit; font-size: inherit; line-height: inherit; font-family: inherit; vertical-align: baseline; box-sizing: inherit; clear: both;"><strong style="margin: 0px; padding: 0px; border: 0px; font-style: inherit; font-variant: inherit; font-weight: bold; font-stretch: inherit; font-size: inherit; line-height: inherit; font-family: inherit; vertical-align: baseline; box-sizing: inherit;">My Research:</strong></p>
<ol style="margin: 0px 0px 1em 30px; padding: 0px; border: 0px; font-style: inherit; font-variant: inherit; font-weight: inherit; font-stretch: inherit; font-size: inherit; line-height: inherit; font-family: inherit; vertical-align: baseline; list-style: decimal; box-sizing: inherit;"> <li style="margin: 0px 0px 0.5em; padding: 0px; border: 0px; font-style: inherit; font-variant: inherit; font-weight: inherit; font-stretch: inherit; font-size: inherit; line-height: inherit; font-family: inherit; vertical-align: baseline; box-sizing: inherit; word-wrap: break-word;">I know calling duplicate calls to MPI_Init is not an option</li> <li style="margin: 0px 0px 0.5em; padding: 0px; border: 0px; font-style: inherit; font-variant: inherit; font-weight: inherit; font-stretch: inherit; font-size: inherit; line-height: inherit; font-family: inherit; vertical-align: baseline; box-sizing: inherit; word-wrap: break-word;">I have looked into the source code for MPICH where it checks if it's initialized, but I would rather not modify MPICH and have my own new silly version!</li> <li style="margin: 0px; padding: 0px; border: 0px; font-style: inherit; font-variant: inherit; font-weight: inherit; font-stretch: inherit; font-size: inherit; line-height: inherit; font-family: inherit; vertical-align: baseline; box-sizing: inherit; word-wrap: break-word;">I have looked at the mailing list, stack overflow, and Google with no luck on how to connect to the new MPI network when a process state is restored.</li></ol>
<p style="margin: 0px 0px 1em; padding: 0px; border: 0px; font-style: inherit; font-variant: inherit; font-weight: inherit; font-stretch: inherit; font-size: inherit; line-height: inherit; font-family: inherit; vertical-align: baseline; box-sizing: inherit; clear: both;"><strong style="margin: 0px; padding: 0px; border: 0px; font-style: inherit; font-variant: inherit; font-weight: bold; font-stretch: inherit; font-size: inherit; line-height: inherit; font-family: inherit; vertical-align: baseline; box-sizing: inherit;">Potential answer</strong>:</p>
<p style="margin: 0px 0px 1em; padding: 0px; border: 0px; font-style: inherit; font-variant: inherit; font-weight: inherit; font-stretch: inherit; font-size: inherit; line-height: inherit; font-family: inherit; vertical-align: baseline; box-sizing: inherit; clear: both;">Of course, I could have my library start a new process and do everything there, but there will be a big performance overhead in communication from a simulator to this new process. Besides, doing it correctly is another can of worms which I prefer not opening if I can.</p>
<p style="margin: 0px 0px 1em; padding: 0px; border: 0px; font-style: inherit; font-variant: inherit; font-weight: inherit; font-stretch: inherit; font-size: inherit; line-height: inherit; font-family: inherit; vertical-align: baseline; box-sizing: inherit; clear: both;"><strong style="margin: 0px; padding: 0px; border: 0px; font-style: inherit; font-variant: inherit; font-weight: bold; font-stretch: inherit; font-size: inherit; line-height: inherit; font-family: inherit; vertical-align: baseline; box-sizing: inherit;">So...</strong><span> </span>any ideas on how to tell this restored Orange process state that this is a new MPI network and you need to talk to this one now instead?</p>
<p style="margin: 0px 0px 1em; padding: 0px; border: 0px; font-style: inherit; font-variant: inherit; font-weight: inherit; font-stretch: inherit; font-size: inherit; line-height: inherit; font-family: inherit; vertical-align: baseline; box-sizing: inherit; clear: both;">Thanks in advance :)</p></div></div></div>
<div dir="ltr"><div class="socmaildefaultfont" dir="ltr" style="font-family:Arial, Helvetica, sans-serif;font-size:10.5pt">-o</div></div></div><br>