<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
I’m not sure I understand your question here, but I believe the way the checkpointing works (or worked, I don’t think it’s functional in recent versions) is that it would periodically freeze the execution to write a checkpoint of all of the data. That checkpointing
would take place periodically based on a parameter given by the user. If that interval is shorter than the interval it takes to make a checkpoint, then it would behave poorly (possible freezing the checkpoint to write another checkpoint, then freeze that checkpoint,
etc.). Honestly, the person who wrote this code in MPICH is gone and it’s not working in the current version so I can’t give you a lot more information than that. It’s on the roadmap to fix this for future versions, but for now, all I can tell you is to extend
the time between checkpoints because the checkpointing system isn’t robust enough to do much more than the simple thing.
<div><br>
</div>
<div>Thanks,</div>
<div>Wesley<br>
<div><br>
<div>
<div>On Jul 17, 2014, at 9:52 AM, Marcela Castro León <<a href="mailto:mcastrol@gmail.com">mcastrol@gmail.com</a>> wrote:</div>
<br class="Apple-interchange-newline">
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_default" style="font-family:georgia,serif;font-size:large">But Do you know why after the first checkpoint the execution is frezzed during a time similar to the interval and, after that, a new checkpoint is triggered? </div>
<div class="gmail_default" style="font-family:georgia,serif;font-size:large">Thank you.</div>
</div>
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">2014-07-17 16:43 GMT+02:00 Bland, Wesley B. <span dir="ltr">
<<a href="mailto:wbland@anl.gov" target="_blank">wbland@anl.gov</a>></span>:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word">You’ll probably need to somehow get your application to run longer then. Unfortunately, MPICH doesn’t support manually starting a checkpoint at this time.
<div>
<div class="h5">
<div><br>
</div>
<div><br>
<div>
<div>On Jul 17, 2014, at 9:29 AM, Marcela Castro León <<a href="mailto:mcastrol@gmail.com" target="_blank">mcastrol@gmail.com</a>> wrote:</div>
<br>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_default" style="font-family:georgia,serif;font-size:large">Hi</div>
<div class="gmail_default" style="font-family:georgia,serif;font-size:large"><br>
</div>
<div class="gmail_default" style="font-family:georgia,serif;font-size:large">I just want to make a one checkpoint during the execution, at the middle. </div>
<div class="gmail_default" style="font-family:georgia,serif;font-size:large">We'are trying to observe the I/O to reduce the time using a parallel file system.</div>
<div class="gmail_default" style="font-family:georgia,serif;font-size:large"><br>
</div>
<div class="gmail_default" style="font-family:georgia,serif;font-size:large">Thanks.<br>
</div>
</div>
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">2014-07-17 13:30 GMT+02:00 Bland, Wesley B. <span dir="ltr">
<<a href="mailto:wbland@anl.gov" target="_blank">wbland@anl.gov</a>></span>:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="auto">
<div>Is there a reason you can't just take the checkpoints less frequently?</div>
<div>
<div>
<div><br>
On Jul 17, 2014, at 4:41 AM, "Marcela Castro León" <<a href="mailto:mcastrol@gmail.com" target="_blank">mcastrol@gmail.com</a>> wrote:<br>
<br>
</div>
<blockquote type="cite">
<div>
<div dir="ltr">
<div class="gmail_default" style="font-family:georgia,serif;font-size:large">Hi,</div>
<div class="gmail_default" style="font-family:georgia,serif;font-size:large">I'm using mpich 2.1.5 compiled to use blcr checkpoint.</div>
<div class="gmail_default" style="font-family:georgia,serif;font-size:large">I'm having problems with the checkpoint interval.</div>
<div class="gmail_default" style="font-family:georgia,serif;font-size:large">When I execute:</div>
<div class="gmail_default"><font face="courier new, monospace">mpiexec -ckpointlib blcr -ckpoint-prefix /partnfs/mpichchk -ckpoint-interval 120 -f maquinas -n 16 ./bt.C.16<br>
</font></div>
<div class="gmail_default"><span style="font-family:georgia,serif;font-size:large"><br>
</span></div>
<div class="gmail_default"><font face="georgia, serif" size="4">In fact, at the second 120, the execution is interrupted for checkpointing, but, as the checkpoint last more than 120 seconds, another checkpoint is immediately triggered instead of resuming the
application.</font></div>
<div class="gmail_default"><font face="georgia, serif" size="4">I only achieve to get a checkpoint by setting a checkpoint interval almost at the end of the execution but it is not useful. </font></div>
<div class="gmail_default"><font face="georgia, serif" size="4"><br>
</font></div>
<div class="gmail_default"><font face="georgia, serif" size="4">How can I solve it?</font></div>
<div class="gmail_default"><span style="font-family:georgia,serif;font-size:large">Besides, Is there a way to know how long is the checkpoint in time?</span><br>
</div>
<div class="gmail_default"><font face="georgia, serif" size="4"><br>
</font></div>
<div class="gmail_default"><font face="georgia, serif" size="4">Thank you very much.</font></div>
<div class="gmail_default"><font face="georgia, serif" size="4"><br>
</font></div>
<div class="gmail_default"><font face="georgia, serif" size="4">Marcela</font></div>
<div class="gmail_default"><br>
</div>
<div class="gmail_default"><br>
</div>
<div class="gmail_default" style="font-family:georgia,serif;font-size:large"><br>
</div>
<div class="gmail_default">
<div class="gmail_default"><font face="comic sans ms, sans-serif">mpiexec -info</font></div>
<div class="gmail_default"><font face="comic sans ms, sans-serif">HYDRA build details:</font></div>
<div class="gmail_default"><font face="comic sans ms, sans-serif"> Version: 1.5</font></div>
<div class="gmail_default"><font face="comic sans ms, sans-serif"> Release Date: Mon Oct 8 14:00:48 CDT 2012</font></div>
<div class="gmail_default"><font face="comic sans ms, sans-serif"> CC: gcc </font></div>
<div class="gmail_default"><font face="comic sans ms, sans-serif"> CXX: c++ </font></div>
<div class="gmail_default"><font face="comic sans ms, sans-serif"> F77: gfortran </font></div>
<div class="gmail_default"><font face="comic sans ms, sans-serif"> F90: gfortran </font></div>
<div class="gmail_default"><font face="comic sans ms, sans-serif"> Configure options: '--disable-option-checking' '--prefix=/soft/mpich2/mpich2-1.5-blcr8.3' '--with-hydra-ckpointlib=blcr' '--with-blcr=/soft/blcr' '--enable-checkpointing'
'--cache-file=/dev/null' '--srcdir=.' 'CC=gcc' 'CFLAGS= -O2' 'LDFLAGS= -L/soft/blcr/lib64 -L/soft/blcr/lib' 'LIBS=-lrt -lcr -lpthread ' 'CPPFLAGS= -I/SRC/mpi/mpich2-1.5/src/mpl/include -I/SRC/mpi/mpich2-1.5/src/mpl/include -I/SRC/mpi/mpich2-1.5/src/openpa/src
-I/SRC/mpi/mpich2-1.5/src/openpa/src -I/SRC/mpi/mpich2-1.5/src/mpi/romio/include -I/soft/blcr/include'</font></div>
<div class="gmail_default"><font face="comic sans ms, sans-serif"> Process Manager: pmi</font></div>
<div class="gmail_default"><font face="comic sans ms, sans-serif"> Launchers available: ssh rsh fork slurm ll lsf sge manual persist</font></div>
<div class="gmail_default"><font face="comic sans ms, sans-serif"> Topology libraries available: hwloc</font></div>
<div class="gmail_default"><font face="comic sans ms, sans-serif"> Resource management kernels available: user slurm ll lsf sge pbs</font></div>
<div class="gmail_default"><font face="comic sans ms, sans-serif"> Checkpointing libraries available: blcr</font></div>
<div class="gmail_default"><font face="comic sans ms, sans-serif"> Demux engines available: poll select</font></div>
<div style="font-family:georgia,serif;font-size:large"><br>
</div>
</div>
<div class="gmail_default" style="font-family:georgia,serif;font-size:large"><br>
</div>
</div>
</div>
</blockquote>
</div>
</div>
<blockquote type="cite">
<div><span>_______________________________________________</span><br>
<span>discuss mailing list <a href="mailto:discuss@mpich.org" target="_blank">discuss@mpich.org</a></span><br>
<span>To manage subscription options or unsubscribe:</span><br>
<span><a href="https://lists.mpich.org/mailman/listinfo/discuss" target="_blank">https://lists.mpich.org/mailman/listinfo/discuss</a></span></div>
</blockquote>
</div>
<br>
_______________________________________________<br>
discuss mailing list <a href="mailto:discuss@mpich.org" target="_blank">discuss@mpich.org</a><br>
To manage subscription options or unsubscribe:<br>
<a href="https://lists.mpich.org/mailman/listinfo/discuss" target="_blank">https://lists.mpich.org/mailman/listinfo/discuss</a><br>
</blockquote>
</div>
<br>
</div>
_______________________________________________<br>
discuss mailing list <a href="mailto:discuss@mpich.org" target="_blank">discuss@mpich.org</a><br>
To manage subscription options or unsubscribe:<br>
<a href="https://lists.mpich.org/mailman/listinfo/discuss" target="_blank">https://lists.mpich.org/mailman/listinfo/discuss</a></blockquote>
</div>
<br>
</div>
</div>
</div>
</div>
<br>
_______________________________________________<br>
discuss mailing list <a href="mailto:discuss@mpich.org">discuss@mpich.org</a><br>
To manage subscription options or unsubscribe:<br>
<a href="https://lists.mpich.org/mailman/listinfo/discuss" target="_blank">https://lists.mpich.org/mailman/listinfo/discuss</a><br>
</blockquote>
</div>
<br>
</div>
_______________________________________________<br>
discuss mailing list <a href="mailto:discuss@mpich.org">discuss@mpich.org</a><br>
To manage subscription options or unsubscribe:<br>
<a href="https://lists.mpich.org/mailman/listinfo/discuss">https://lists.mpich.org/mailman/listinfo/discuss</a></blockquote>
</div>
<br>
</div>
</div>
</body>
</html>