<meta http-equiv="Content-Type" content="text/html; charset=gb2312"><div style="line-height:1.7;color:#000000;font-size:14px;font-family:Arial"><div>Thanks for your help. With your configuration, I get the expected result on the same machine.</div><div><br></div><div>Previous configuration:</div><div>====</div><div>1><span class="Apple-tab-span" style="white-space:pre"> </span>--prefix=/home/nbtest/develop/mpich30 --with-device=ch3:nemesis CFLAGS=-fPIC</div><div>2> --prefix=/home/nbtest/develop/mpich30 --with-device=ch3:nemesis --enable-romio --enable-nemesis-dbg-localoddeven --enable-g=all --enable-fast=O0 --enable-fortran=f77</div><div><br></div><div>Current configuration:</div><div>====</div><div>--prefix=/home/nbtest/develop/mpich30 --with-device=ch3:nemesis --enable-romio --enable-nemesis-dbg-localoddeven --enable-g=all --enable-fast=O0 --enable-fortran=f77,fc CC=gcc CXX=g++ F77=gfortran FC=gfortran</div><div></div><div id="divNeteaseMailCard"></div><br>At 2014-10-20 10:39:20, "Junchao Zhang" <jczhang@mcs.anl.gov> wrote:<br> <blockquote id="isReplyContent" style="PADDING-LEFT: 1ex; MARGIN: 0px 0px 0px 0.8ex; BORDER-LEFT: #ccc 1px solid"><div dir="ltr">Try to run on another machine, you may get different results. My configure is <div><div>$ mpichversion</div><div>MPICH Version: <span class="" style="white-space:pre"> </span>3.1.3</div><div>MPICH Release date:<span class="" style="white-space:pre"> </span>unreleased development copy</div><div>MPICH Device: <span class="" style="white-space:pre"> </span>ch3:nemesis</div><div>MPICH configure: <span class="" style="white-space:pre"> </span>--enable-romio --enable-nemesis-dbg-localoddeven --enable-g=all --enable-fast=O0 --enable-fortran=f77,fc CC=gcc CXX=g++ F77=gfortran FC=gfortran</div><div>MPICH CC: <span class="" style="white-space:pre"> </span>gcc -g -O0</div><div>MPICH CXX: <span class="" style="white-space:pre"> </span>g++ -g -O0</div><div>MPICH F77: <span class="" style="white-space:pre"> </span>gfortran -g -O0</div><div>MPICH FC: <span class="" style="white-space:pre"> </span>gfortran -g -O0</div></div></div><div class="gmail_extra"><br clear="all"><div><div dir="ltr">--Junchao Zhang</div></div>
<br><div class="gmail_quote">On Sun, Oct 19, 2014 at 9:23 PM, myself <span dir="ltr"><<a href="mailto:chcdlf@126.com" target="_blank">chcdlf@126.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="line-height:1.7;color:#000000;font-size:14px;font-family:Arial"><br><span style="line-height:23.7999992370605px">I run several times of the program, the results are as the same as before. </span>Could you tell me your <span style="line-height:23.7999992370605px">MPICH</span> version and compiling parameters? The version of my MPICH is 3.1.3, I compile it as <br><div> </div><div>$ mpirun --version</div><div>HYDRA build details:</div><div> Version: 3.1.3</div><div> Release Date: Wed Oct 8 09:37:19 CDT 2014</div><div> CC: gcc -fPIC </div><div> CXX: g++ </div><div> F77: gfortran </div><div> F90: gfortran </div><div> Configure options: '--disable-option-checking' '--prefix=/usr/local/mpich3' '--with-device=ch3:nemesis' 'CFLAGS=-fPIC -O2' '--disable-fortran' '--cache-file=/dev/null' '--srcdir=.' 'CC=gcc' 'LDFLAGS= ' 'LIBS=-lpthread ' 'CPPFLAGS= -I/home/valder/Downloads/Software/mpi/mpich-3.1.3/src/mpl/include -I/home/valder/Downloads/Software/mpi/mpich-3.1.3/src/mpl/include -I/home/valder/Downloads/Software/mpi/mpich-3.1.3/src/openpa/src -I/home/valder/Downloads/Software/mpi/mpich-3.1.3/src/openpa/src -D_REENTRANT -I/home/valder/Downloads/Software/mpi/mpich-3.1.3/src/mpi/romio/include'</div><div> Process Manager: pmi</div><div> Launchers available: ssh rsh fork slurm ll lsf sge manual persist</div><div> Topology libraries available: hwloc</div><div> Resource management kernels available: user slurm ll lsf sge pbs cobalt</div><div> Checkpointing libraries available: </div><div> Demux engines available: poll select</div><div> </div><div>the whole program is </div><div><span style="line-height:23.7999992370605px">=========</span></div><div><span style="line-height:23.7999992370605px"><br></span></div><div><span class=""><div>#include "mpi.h"</div><div>#include <stdio.h></div><div>#include <stdlib.h></div><div>#include <sys/time.h></div><div><br></div><div>typedef unsigned char byte;</div><div><br></div><div>int main(int argc, char *argv[]){</div></span><div> int rank,size;</div><span class=""><div> byte buf[10][10];</div><div> MPI_Request req[10];</div><div> MPI_Status stat[10];</div><div> int outcount;</div><div> int array_of_indices[10];</div><div> int i;</div></span><div> int complete = 0;</div><span class=""><div><br></div><div> MPI_Init(&argc, &argv);</div><div> MPI_Comm_size(MPI_COMM_WORLD, &size);</div><div> MPI_Comm_rank(MPI_COMM_WORLD, &rank);</div><div><br></div><div> if(rank==0) {</div><div> for(i =0; i< 10;i++){</div><div> MPI_Isend(buf[i], 10, MPI_BYTE, 1, 123, MPI_COMM_WORLD, &req[i]);</div><div> }</div><div> printf("post send\n");</div><div> }else {</div><div> for(i=0; i< 10;i++){</div><div> MPI_Irecv(buf[i], 10, MPI_BYTE, 0, 123, MPI_COMM_WORLD, &req[i]);</div><div> }</div><div> printf("post recv\n");</div><div> }</div><div><br></div><div> for (;;){</div><div> MPI_Waitsome(10, req, &outcount, array_of_indices, stat);</div><div> complete += outcount;</div><div> printf("rank %d %d index %d\n", rank, outcount, array_of_indices[0]);</div><div> if(complete == 10)</div><div> break;</div><div> }</div><div> printf("rank %d complete\n", rank);</div><div> MPI_Finalize();</div><div>}</div></span></div><div><span style="line-height:23.7999992370605px">=========</span></div><div><span style="line-height:23.7999992370605px"><br></span></div><div>I compile it as </div><div>$ mpicc -o tests testsome.c </div><div><br></div><div>and run as </div><span class=""><div>$ mpirun -n 2 ./tests <br></div><div><br></div><div><br></div></span><div><div class="h5">At 2014-10-20 09:02:39, "Junchao Zhang" <<a href="mailto:jczhang@mcs.anl.gov" target="_blank">jczhang@mcs.anl.gov</a>> wrote:<br> <blockquote style="PADDING-LEFT:1ex;MARGIN:0px 0px 0px 0.8ex;BORDER-LEFT:#ccc 1px solid"><div dir="ltr">I think it is random. For example, I ran your code and its output was:<div><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div><div><div>post recv</div></div></div><div><div><div>post send</div></div></div><div><div><div>rank 0 10 index 0</div></div></div><div><div><div>rank 0 complete</div></div></div><div><div><div>rank 1 1 index 0</div></div></div><div><div><div>rank 1 9 index 1</div></div></div><div><div><div>rank 1 complete</div></div></div><div><br></div></blockquote><div><div><div class="gmail_extra"><div><div dir="ltr">--Junchao Zhang</div></div>
<br><div class="gmail_quote">On Sun, Oct 19, 2014 at 9:48 AM, myself <span dir="ltr"><<a href="mailto:chcdlf@126.com" target="_blank">chcdlf@126.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div style="line-height:1.7;color:rgb(0,0,0);font-size:14px;font-family:Arial"><div><span style="line-height:23.7999992370605px">Because I want to understand the behavior of MPI_Waitsome, I change the program as fellow. MPI can waitsome 10 Isend request once time, while waitsome 1 Irecv request each time. Why waitsome-Irecv can not be more than 1 as I expect?</span></div><div><span style="line-height:23.7999992370605px">=====</span></div><div> MPI_Init(&argc, &argv);</div><div> MPI_Comm_size(MPI_COMM_WORLD, &size);</div><div> MPI_Comm_rank(MPI_COMM_WORLD, &rank);</div><span><div> </div><div> if(rank==0) {</div><div> for(i =0; i< 10;i++){</div><div> MPI_Isend(buf[i], 10, MPI_BYTE, 1, 123, MPI_COMM_WORLD, &req[i]);</div><div> }</div><div> printf("post send\n");</div></span><span><div> }else {</div><div> for(i=0; i< 10;i++){</div><div> MPI_Irecv(buf[i], 10, MPI_BYTE, 0, 123, MPI_COMM_WORLD, &req[i]);</div><div> }</div><div> printf("post recv\n");</div><div> }</div><div> </div></span><div> for (;;){</div><span><div> MPI_Waitsome(10, req, &outcount, array_of_indices, stat);</div></span><div> complete += outcount;</div><div> printf("rank %d %d index %d\n", rank, outcount, array_of_indices[0]);</div><div> if(complete == 10)</div><div> break;</div><div> }</div><div> printf("rank %d complete\n", rank);</div><div> MPI_Finalize();</div><span><div>=====</div><div>$ mpirun -n 2 ./tests</div><div>post send</div></span><div>post recv</div><div>rank 0 10 index 0</div><div>rank 0 complete</div><div>rank 1 1 index 0</div><div>rank 1 1 index 1</div><div>rank 1 1 index 2</div><div>rank 1 1 index 3</div><div>rank 1 1 index 4</div><div>rank 1 1 index 5</div><div>rank 1 1 index 6</div><div>rank 1 1 index 7</div><div>rank 1 1 index 8</div><div>rank 1 1 index 9</div><div>rank 1 complete</div><div><div><div><br></div><br><br><br><div></div><div></div><br>At 2014-10-19 22:36:25, "Bland, Wesley B." <<a href="mailto:wbland@anl.gov" target="_blank">wbland@anl.gov</a>> wrote:<br> <blockquote style="padding-left:1ex;margin:0px 0px 0px 0.8ex;border-left-color:rgb(204,204,204);border-left-width:1px;border-left-style:solid">
<div>You need to move your waitany outside of the if statement so both ranks execute it. <br>
<br>
<br>
</div>
<div><br>
On Oct 19, 2014, at 9:33 AM, myself <<a href="mailto:chcdlf@126.com" target="_blank">chcdlf@126.com</a>> wrote:<br>
<br>
</div>
<blockquote type="cite">
<div>
<div style="line-height:1.7;color:rgb(0,0,0);font-size:14px;font-family:Arial">
<div>Here is my test program</div>
<br>
<span style="line-height:23.7999992370605px">======</span>
<div><br>
</div>
<div>
<div> #include "mpi.h"</div>
<div> #include <stdio.h></div>
<div> #include <stdlib.h></div>
<div> #include <sys/time.h></div>
<div> </div>
<div>
<div> typedef unsigned char byte;</div>
<div> </div>
<div> int main(int argc, char *argv[]){</div>
<div> MPI_Init(&argc, &argv);</div>
<div> int rank,size;</div>
<div> MPI_Comm_size(MPI_COMM_WORLD, &size);</div>
<div> MPI_Comm_rank(MPI_COMM_WORLD, &rank);</div>
<div> byte buf[10][10];</div>
<div> MPI_Request req[10];</div>
<div> MPI_Status stat[10];</div>
<div> int outcount;</div>
<div> int array_of_indices[10];</div>
<div> int i;</div>
<div> </div>
<div> if(rank==0) {</div>
<div> for(i =0; i< 10;i++){</div>
<div> MPI_Isend(buf[i], 10, MPI_BYTE, 1, 123, MPI_COMM_WORLD, &req[i]);</div>
<div> }</div>
<div> printf("post send\n");</div>
<div> MPI_Waitall(10, req, stat);</div>
<div> printf("send over\n");</div>
<div> }else {</div>
<div> for(i=0; i< 10;i++){</div>
<div> MPI_Irecv(buf[i], 10, MPI_BYTE, 0, 123, MPI_COMM_WORLD, &req[i]);</div>
<div> }</div>
<div> </div>
<div> printf("post recv\n");</div>
<div> for (i=0;i<10;i++){</div>
<div> MPI_Waitsome(10, req, &outcount, array_of_indices, stat);</div>
<div> printf("%d index %d\n", outcount, array_of_indices[0]);</div>
<div> }</div>
<div> }</div>
<div> MPI_Finalize();</div>
<div> }</div>
</div>
</div>
<div><br>
</div>
<div>======</div>
<div><span style="line-height:1.7"> get the result like this</span></div>
<div>======</div>
<div>
<div>
<div>$ mpirun -n 2 ./tests</div>
<div>post send</div>
<div>send over</div>
<div>post recv</div>
<div>1 index 0</div>
<div>1 index 1</div>
<div>1 index 2</div>
<div>1 index 3</div>
<div>1 index 4</div>
<div>1 index 5</div>
<div>1 index 6</div>
<div>1 index 7</div>
<div>1 index 8</div>
<div>1 index 9</div>
</div>
</div>
<div><br>
</div>
<div><span style="line-height:23.7999992370605px">======</span></div>
<div><span style="line-height:23.7999992370605px"><br>
</span></div>
<div><span style="line-height:23.7999992370605px">I think I should get some completed receive connection, not just 1.</span></div>
<div><br>
</div>
<div><br>
</div>
</div>
<br>
<br>
<span title="neteasefooter"><span></span></span></div>
</blockquote>
<blockquote type="cite">
<div><span>_______________________________________________</span><br>
<span>discuss mailing list <a href="mailto:discuss@mpich.org" target="_blank">discuss@mpich.org</a></span><br>
<span>To manage subscription options or unsubscribe:</span><br>
<span><a href="https://lists.mpich.org/mailman/listinfo/discuss" target="_blank">https://lists.mpich.org/mailman/listinfo/discuss</a></span></div>
</blockquote>
</blockquote></div></div></div><br><br><span title="neteasefooter"><span></span></span><br>_______________________________________________<br>
discuss mailing list <a href="mailto:discuss@mpich.org" target="_blank">discuss@mpich.org</a><br>
To manage subscription options or unsubscribe:<br>
<a href="https://lists.mpich.org/mailman/listinfo/discuss" target="_blank">https://lists.mpich.org/mailman/listinfo/discuss</a><br></blockquote></div><br></div></div></div></div></div>
</blockquote></div></div></div><br><br><span title="neteasefooter"><span></span></span><br>_______________________________________________<br>
discuss mailing list <a href="mailto:discuss@mpich.org">discuss@mpich.org</a><br>
To manage subscription options or unsubscribe:<br>
<a href="https://lists.mpich.org/mailman/listinfo/discuss" target="_blank">https://lists.mpich.org/mailman/listinfo/discuss</a><br></blockquote></div><br></div>
</blockquote></div><br><br><span title="neteasefooter"><span id="netease_mail_footer"></span></span>