<div dir="ltr">I see. Thanks a lot! and that make sense. I tried fflush(stdout), it seems that still not work properly. but I understand that printf is not a proper way to check the timing.<div><br></div><div style>Thank you!</div>
<div style>Sufeng</div><div><br><div class="gmail_extra"><br><br><div class="gmail_quote">On Tue, Jul 9, 2013 at 12:24 PM, Jeff Hammond <span dir="ltr"><<a href="mailto:jeff.science@gmail.com" target="_blank">jeff.science@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">...and depending on your filesystem (assuming you pipe stdout to a<br>
file), even that isn't sufficient.<br>
<br>
printf is a terrible way to evaluate temporal relationships and you<br>
shouldn't try to reason about parallel programs with it unless you've<br>
taken great care to ensure a global total ordering of output streams.<br>
<br>
Jeff<br>
<div class="HOEnZb"><div class="h5"><br>
On Tue, Jul 9, 2013 at 12:05 PM, Pavan Balaji <<a href="mailto:balaji@mcs.anl.gov">balaji@mcs.anl.gov</a>> wrote:<br>
><br>
> printf buffers output and doesn't write it out immediately (or in this case,<br>
> send it to mpiexec immediately). Try doing "fflush(stdout);" immediately<br>
> after printf.<br>
><br>
> -- Pavan<br>
><br>
><br>
> On 07/09/2013 12:01 PM, Sufeng Niu wrote:<br>
>><br>
>> Hello,<br>
>><br>
>> Sorry to post this long, stupid and simple question.<br>
>> I found that some time MPI_Barrier cannot stop all the process. I try to<br>
>> write a simple test program to create data_struct shown below:<br>
>><br>
>> #include <stdlib.h><br>
>> #include <stdio.h><br>
>> #include "mpi.h"<br>
>><br>
>> typedef struct<br>
>> {<br>
>> int a;<br>
>> char b;<br>
>> int c;<br>
>> int d;<br>
>> } foo;<br>
>><br>
>> int main(int argc, char *argv[])<br>
>> {<br>
>><br>
>> int rank, size;<br>
>> int i;<br>
>><br>
>> foo x;<br>
>><br>
>> MPI_Init(&argc, &argv);<br>
>> MPI_Comm_size(MPI_COMM_WORLD, &size);<br>
>> MPI_Comm_rank(MPI_COMM_WORLD, &rank);<br>
>><br>
>> char processor_name[MPI_MAX_PROCESSOR_NAME];<br>
>> int name_len;<br>
>> MPI_Get_processor_name(processor_name, &name_len);<br>
>> printf("-- processor %s, rank %d out of %d processors\n",<br>
>> processor_name, rank, size);<br>
>><br>
>> *MPI_Barrier(MPI_COMM_WORLD);*<br>
>><br>
>><br>
>> int count=4;<br>
>><br>
>> MPI_Datatype testtype;<br>
>> MPI_Datatype types[4] = {MPI_INT, MPI_CHAR, MPI_INT, MPI_DOUBLE};<br>
>> int len[4] = {1, 1, 1, 1};<br>
>> MPI_Aint disp[4];<br>
>> long int base;<br>
>><br>
>> MPI_Address(&x, disp);<br>
>> MPI_Address(&(x.a), disp+1);<br>
>> MPI_Address(&(x.b), disp+2);<br>
>> MPI_Address(&(x.c), disp+3);<br>
>> base = disp[0];<br>
>> for(i=0; i<4; i++) disp[i] -= base;<br>
>><br>
>> MPI_Type_struct(count, len, disp, types, &testtype);<br>
>> MPI_Type_commit(&testtype);<br>
>><br>
>> if(rank == 0){<br>
>> x.a = 2;<br>
>> x.b = 0;<br>
>> x.c = 10;<br>
>> x.d = 3;<br>
>> }<br>
>><br>
>> printf("rank %d(before): x value is %d, %d, %d, %d\n", rank, x.a,<br>
>> x.b, x.c, x.d);<br>
>> *MPI_Barrier(MPI_COMM_WORLD);*<br>
>><br>
>> MPI_Bcast(&x, 1, testtype, 0, MPI_COMM_WORLD);<br>
>><br>
>> printf("rank %d(after): x value is %d, %d, %d, %d\n", rank, x.a,<br>
>> x.b, x.c, x.d);<br>
>><br>
>> MPI_Finalize();<br>
>><br>
>> return 0;<br>
>> }<br>
>><br>
>> the output should be looks like:<br>
>> -- processor <a href="http://iocfccd3.aps.anl.gov" target="_blank">iocfccd3.aps.anl.gov</a> <<a href="http://iocfccd3.aps.anl.gov" target="_blank">http://iocfccd3.aps.anl.gov</a>>, rank 0<br>
>> out of 4 processors<br>
>> -- processor <a href="http://iocfccd3.aps.anl.gov" target="_blank">iocfccd3.aps.anl.gov</a> <<a href="http://iocfccd3.aps.anl.gov" target="_blank">http://iocfccd3.aps.anl.gov</a>>, rank 3<br>
>> out of 4 processors<br>
>> -- processor <a href="http://iocfccd3.aps.anl.gov" target="_blank">iocfccd3.aps.anl.gov</a> <<a href="http://iocfccd3.aps.anl.gov" target="_blank">http://iocfccd3.aps.anl.gov</a>>, rank 1<br>
>> out of 4 processors<br>
>> -- processor <a href="http://iocfccd3.aps.anl.gov" target="_blank">iocfccd3.aps.anl.gov</a> <<a href="http://iocfccd3.aps.anl.gov" target="_blank">http://iocfccd3.aps.anl.gov</a>>, rank 2<br>
>><br>
>> out of 4 processors<br>
>> rank 0(before): x value is 2, 0, 10, 3<br>
>> rank 1(before): x value is 1197535864, -1, 4994901, 0<br>
>> rank 2(before): x value is 1591464488, -1, 4994901, 0<br>
>> rank 3(before): x value is 1851622184, -1, 4994901, 0<br>
>> rank 0(after): x value is 2, 0, 10, 3<br>
>> rank 3(after): x value is 2, 0, 10, 3<br>
>> rank 1(after): x value is 2, 0, 10, 3<br>
>> rank 2(after): x value is 2, 0, 10, 3<br>
>><br>
>> but some time is shows as:<br>
>> -- processor <a href="http://iocfccd3.aps.anl.gov" target="_blank">iocfccd3.aps.anl.gov</a> <<a href="http://iocfccd3.aps.anl.gov" target="_blank">http://iocfccd3.aps.anl.gov</a>>, rank 0<br>
>><br>
>> out of 4 processors<br>
>> rank 0(before): x value is 2, 0, 10, 3<br>
>> rank 0(after): x value is 2, 0, 10, 3<br>
>> -- processor <a href="http://iocfccd3.aps.anl.gov" target="_blank">iocfccd3.aps.anl.gov</a> <<a href="http://iocfccd3.aps.anl.gov" target="_blank">http://iocfccd3.aps.anl.gov</a>>, rank 1<br>
>><br>
>> out of 4 processors<br>
>> rank 1(before): x value is -464731256, -1, 4994901, 0<br>
>> rank 1(after): x value is 2, 0, 10, 3<br>
>> -- processor <a href="http://iocfccd3.aps.anl.gov" target="_blank">iocfccd3.aps.anl.gov</a> <<a href="http://iocfccd3.aps.anl.gov" target="_blank">http://iocfccd3.aps.anl.gov</a>>, rank 2<br>
>><br>
>> out of 4 processors<br>
>> rank 2(before): x value is 1863042488, -1, 4994901, 0<br>
>> rank 2(after): x value is 2, 0, 10, 3<br>
>> -- processor <a href="http://iocfccd3.aps.anl.gov" target="_blank">iocfccd3.aps.anl.gov</a> <<a href="http://iocfccd3.aps.anl.gov" target="_blank">http://iocfccd3.aps.anl.gov</a>>, rank 3<br>
>><br>
>> out of 4 processors<br>
>> rank 3(before): x value is 1721065144, -1, 4994901, 0<br>
>> rank 3(after): x value is 2, 0, 10, 3<br>
>><br>
>> or<br>
>> -- processor <a href="http://iocfccd3.aps.anl.gov" target="_blank">iocfccd3.aps.anl.gov</a> <<a href="http://iocfccd3.aps.anl.gov" target="_blank">http://iocfccd3.aps.anl.gov</a>>, rank 0<br>
>> out of 4 processors<br>
>> -- processor <a href="http://iocfccd3.aps.anl.gov" target="_blank">iocfccd3.aps.anl.gov</a> <<a href="http://iocfccd3.aps.anl.gov" target="_blank">http://iocfccd3.aps.anl.gov</a>>, rank 1<br>
>><br>
>> out of 4 processors<br>
>> rank 1(before): x value is -1883169624, -1, 4994901, 0<br>
>> -- processor <a href="http://iocfccd3.aps.anl.gov" target="_blank">iocfccd3.aps.anl.gov</a> <<a href="http://iocfccd3.aps.anl.gov" target="_blank">http://iocfccd3.aps.anl.gov</a>>, rank 2<br>
>><br>
>> out of 4 processors<br>
>> rank 2(before): x value is -451256152, -1, 4994901, 0<br>
>> -- processor <a href="http://iocfccd3.aps.anl.gov" target="_blank">iocfccd3.aps.anl.gov</a> <<a href="http://iocfccd3.aps.anl.gov" target="_blank">http://iocfccd3.aps.anl.gov</a>>, rank 3<br>
>><br>
>> out of 4 processors<br>
>> rank 3(before): x value is 1715067240, -1, 4994901, 0<br>
>> rank 0(before): x value is 2, 0, 10, 3<br>
>> rank 0(after): x value is 2, 0, 10, 3<br>
>> rank 1(after): x value is 2, 0, 10, 3<br>
>> rank 2(after): x value is 2, 0, 10, 3<br>
>> rank 3(after): x value is 2, 0, 10, 3<br>
>><br>
>> it is all randomly, I am not sure where is the problem.<br>
>><br>
>> The second issue is I use MPI_Datatype to create a MPI struct for<br>
>> broadcast. However, as the program shown above, if I change the struct:<br>
>> typedef struct<br>
>> {<br>
>> int a;<br>
>> char b;<br>
>> int c;<br>
>> *int d;*<br>
>><br>
>> } foo;<br>
>> as<br>
>> typedef struct<br>
>> {<br>
>> int a;<br>
>> char b;<br>
>> int c;<br>
>> *double d*;<br>
>><br>
>> } foo;<br>
>><br>
>> I found the result is:<br>
>> -- processor <a href="http://ephesus.ece.iit.edu" target="_blank">ephesus.ece.iit.edu</a> <<a href="http://ephesus.ece.iit.edu" target="_blank">http://ephesus.ece.iit.edu</a>>, rank 1<br>
>> out of 4 processors<br>
>> -- processor <a href="http://ephesus.ece.iit.edu" target="_blank">ephesus.ece.iit.edu</a> <<a href="http://ephesus.ece.iit.edu" target="_blank">http://ephesus.ece.iit.edu</a>>, rank 2<br>
>> out of 4 processors<br>
>> -- processor <a href="http://ephesus.ece.iit.edu" target="_blank">ephesus.ece.iit.edu</a> <<a href="http://ephesus.ece.iit.edu" target="_blank">http://ephesus.ece.iit.edu</a>>, rank 3<br>
>> out of 4 processors<br>
>> -- processor <a href="http://ephesus.ece.iit.edu" target="_blank">ephesus.ece.iit.edu</a> <<a href="http://ephesus.ece.iit.edu" target="_blank">http://ephesus.ece.iit.edu</a>>, rank 0<br>
>><br>
>> out of 4 processors<br>
>> rank 0(before): x value is 2, 0, 10, 3.250000<br>
>> rank 3(before): x value is 0, 0, 547474368, 0.000000<br>
>> rank 1(before): x value is 0, 0, 547474368, 0.000000<br>
>> rank 2(before): x value is 0, 0, 547474368, 0.000000<br>
>> rank 0(after): x value is 2, 0, 10, 3.250000<br>
>> *rank 2(after): x value is 2, 0, 10, 0.000000 <- should be 3.25<br>
>><br>
>> rank 3(after): x value is 2, 0, 10, 0.000000 <- should be 3.25<br>
>> rank 1(after): x value is 2, 0, 10, 0.000000**<- should be 3.25<br>
>><br>
>> *<br>
>> Do you have any clues on why this happened? Thanks a lot!<br>
>><br>
>> --<br>
>> Best Regards,<br>
>> Sufeng Niu<br>
>> ECASP lab, ECE department, Illinois Institute of Technology<br>
>> Tel: 312-731-7219<br>
>><br>
>><br>
>> _______________________________________________<br>
>> discuss mailing list <a href="mailto:discuss@mpich.org">discuss@mpich.org</a><br>
>> To manage subscription options or unsubscribe:<br>
>> <a href="https://lists.mpich.org/mailman/listinfo/discuss" target="_blank">https://lists.mpich.org/mailman/listinfo/discuss</a><br>
>><br>
><br>
> --<br>
> Pavan Balaji<br>
> <a href="http://www.mcs.anl.gov/~balaji" target="_blank">http://www.mcs.anl.gov/~balaji</a><br>
> _______________________________________________<br>
> discuss mailing list <a href="mailto:discuss@mpich.org">discuss@mpich.org</a><br>
> To manage subscription options or unsubscribe:<br>
> <a href="https://lists.mpich.org/mailman/listinfo/discuss" target="_blank">https://lists.mpich.org/mailman/listinfo/discuss</a><br>
<br>
<br>
<br>
--<br>
</div></div><span class="HOEnZb"><font color="#888888">Jeff Hammond<br>
<a href="mailto:jeff.science@gmail.com">jeff.science@gmail.com</a><br>
</font></span></blockquote></div><br><br clear="all"><div><br></div>-- <br>Best Regards,<div>Sufeng Niu</div><div>ECASP lab, ECE department, Illinois Institute of Technology</div><div>Tel: 312-731-7219</div>
</div></div></div>