[mpich-devel] MPICH hangs in MPI_Waitall when MPI_Cancel is used

[Rob Latham]

On 06/04/2015 12:19 PM, Jeff Hammond wrote:
> Thanks for pointing that out.  It runs correctly now.  Sorry for the
> stupid question.

  it just so happens, Jeff, that they've been spending a lot of time 
debugging cancel send operations for all our various devices and so 
"cancel semantics" are (moreso than usual) quite warm in the cache.

==rob

> On Thu, Jun 4, 2015 at 11:49 AM, Halim Amer <aamer at anl.gov> wrote:
>> Hi Jeff,
>>
>> I don't think it is a correct program. If the send is correctly canceled
>> then the origin has to satisfy the destination with another send. The hang
>> is an expected result.
>>
>> This is what the standard says (P102):
>>
>> "...or that the send is successfully cancelled, in which case no part of the
>> message was received at the destination. Then, any matching receive has to
>> be satisfied by another send."
>>
>> --Halim
>>
>> Abdelhalim Amer (Halim)
>> Postdoctoral Appointee
>> MCS Division
>> Argonne National Laboratory
>>
>>
>> On 6/4/15 9:21 AM, Jeff Hammond wrote:
>>>
>>> I can't tell for sure if this is a correct program, but multiple
>>> members of the MPI Forum suggested it is.
>>>
>>> If it is a correct program, it appears to expose a bug in MPICH,
>>> because the MPI_Waitall hangs.
>>>
>>> Thanks,
>>>
>>> Jeff
>>>
>>> $ mpicc -g -Wall -std=c99 cancel-sucks.c && mpiexec -n 4 ./a.out
>>>
>>> $ mpichversion
>>> MPICH Version:    3.2b1
>>> MPICH Release date: unreleased development copy
>>> MPICH Device:    ch3:nemesis
>>> MPICH configure: CC=gcc-4.9 CXX=g++-4.9 FC=gfortran-4.9
>>> F77=gfortran-4.9 --enable-cxx --enable-fortran
>>> --enable-threads=runtime --enable-g=dbg --with-pm=hydra
>>> --prefix=/opt/mpich/dev/gcc/default --enable-wrapper-rpath
>>> --enable-static --enable-shared
>>> MPICH CC: gcc-4.9    -g -O2
>>> MPICH CXX: g++-4.9   -g -O2
>>> MPICH F77: gfortran-4.9   -g -O2
>>> MPICH FC: gfortran-4.9   -g -O2
>>>
>>>
>>> #include <stdio.h>
>>> #include <stdlib.h>
>>> #include <mpi.h>
>>>
>>> const int n=1000;
>>>
>>> int main(void)
>>> {
>>>       MPI_Init(NULL,NULL);
>>>
>>>       int size, rank;
>>>       MPI_Comm_size(MPI_COMM_WORLD, &size);
>>>       MPI_Comm_rank(MPI_COMM_WORLD, &rank);
>>>       if (size<2) {
>>>           printf("You must use 2 or more processes!\n");
>>>           MPI_Finalize();
>>>           exit(1);
>>>       }
>>>
>>>       MPI_Request reqs[2*n];
>>>
>>>       int target = (rank+1)%size;
>>>       for (int i=0; i<n; i++) {
>>>           MPI_Issend(NULL,0,MPI_BYTE,target,0,MPI_COMM_WORLD,&(reqs[i]));
>>>       }
>>>
>>>       srand((unsigned)(rank+MPI_Wtime()));
>>>       int slot = rand()%n;
>>>       printf("Cancelling send %d.\n", slot);
>>>       MPI_Cancel(&reqs[slot]);
>>>
>>> #if 1
>>>       MPI_Barrier(MPI_COMM_WORLD);
>>> #endif
>>>
>>>       int origin = (rank==0) ? (size-1) : (rank-1);
>>>       for (int i=0; i<n; i++) {
>>>           MPI_Irecv(NULL,0,MPI_BYTE,origin,0,MPI_COMM_WORLD,&(reqs[n+i]));
>>>       }
>>>
>>>       MPI_Status stats[2*n];
>>>       MPI_Waitall(2*n,reqs,stats);
>>>
>>>       for (int i=0; i<n; i++) {
>>>           int flag;
>>>           MPI_Test_cancelled(&(stats[i]),&flag);
>>>           if (flag) {
>>>               printf("Status %d indicates cancel was successful.\n", i);
>>>           }
>>>       }
>>>
>>>       MPI_Finalize();
>>>       return 0;
>>> }
>>>
>>>
>> _______________________________________________
>> To manage subscription options or unsubscribe:
>> https://lists.mpich.org/mailman/listinfo/devel
>
>
>

-- 
Rob Latham
Mathematics and Computer Science Division
Argonne National Lab, IL USA