[mpich-discuss] gather, send

Rajeev Thakur thakur at mcs.anl.gov
Wed May 1 19:34:56 CDT 2013


The gather is sending everything to real rank 1. Since you have incremented irank, you need to print the values at irank=2.

On May 1, 2013, at 7:31 PM, Ryan Crocker wrote:

> my call for the memory allocation for the buffers:
> 
>  call parallel_min_int_0d(imaxo_*jmaxo_*kmaxo_,max_buff)
>  allocate(send_buffer(max_buff))
>  max_global_buff = max_buff*nproc
>  allocate(gsend_buffer  (max_global_buff))
> 
> in my mpi init subroutine:
> 
> 
>  call MPI_INIT(ierr)
>  call MPI_COMM_RANK(MPI_COMM_WORLD,irank,ierr)
>  call MPI_COMM_SIZE(MPI_COMM_WORLD,nproc,ierr) 
>  irank = irank+1
>  iroot = 1
> 
> 
> On May 1, 2013, at 5:21 PM, Rajeev Thakur wrote:
> 
>> Is the send_buffer allocated to be large enough to hold max_buff on all ranks?
>> 
>> Is gsend_buffer large enough to hold max_buff * nprocs?
>> 
>> Is the rank 1, not rank 0, the root?
>> 
>> On May 1, 2013, at 7:16 PM, Ryan Crocker wrote:
>> 
>>> i'm just checking just the gather call, and the gsend_buffer from the gather call is still empty:
>>> 
>>> max_buff is now the largest core domain.
>>> 
>>> print*,max_buff ->        24750
>>> 
>>> call MPI_GATHER(send_buffer, max_buff, MPI_REAL, gsend_buffer, max_buff, MPI_REAL, iroot, comm, ierr)
>>> 
>>> a print out of the send_buffer and the gsend_buffer where it should have non-zero terms.
>>> 
>>> print*,irank,i,j,k,send_buffer(i,j,k),gsent_buffer(i,j,k) ->
>>> 
>>>         1           6          50           6   0.0000000000000000        0.0000000000000000     
>>>         1           7          50           6   0.0000000000000000        0.0000000000000000     
>>>         1           8          50           6   0.0000000000000000        0.0000000000000000     
>>>         1           9          50           6   0.0000000000000000        0.0000000000000000     
>>>         1          10          50           6   0.0000000000000000        0.0000000000000000     
>>>         1          11          50           6   0.0000000000000000        0.0000000000000000     
>>>         1          12          50           6   0.0000000000000000        0.0000000000000000     
>>>         1          13          50           6   0.0000000000000000        0.0000000000000000     
>>>         1          14          50           6   0.0000000000000000        0.0000000000000000     
>>>         1          15          50           6   0.0000000000000000        0.0000000000000000     
>>>         1          16          50           6   0.0000000000000000        0.0000000000000000     
>>>         1          17          50           6   0.0000000000000000        0.0000000000000000     
>>>         1          18          50           6   0.0000000000000000        0.0000000000000000     
>>>         1          19          50           6   0.0000000000000000        0.0000000000000000     
>>>         1          20          50           6   0.0000000000000000        0.0000000000000000     
>>>         1          21          50           6   0.0000000000000000        0.0000000000000000     
>>>         1          22          50           6   0.0000000000000000        0.0000000000000000     
>>>         1          23          50           6   0.0000000000000000        0.0000000000000000     
>>>         1          24          50           6   0.0000000000000000        0.0000000000000000     
>>>         1          25          50           6   0.0000000000000000        0.0000000000000000   
>>>         1           6          51           6   1.0000000000000000        0.0000000000000000     
>>>         1           7          51           6   1.0000000000000000        0.0000000000000000     
>>>         1           8          51           6   1.0000000000000000        0.0000000000000000     
>>>         1           9          51           6   1.0000000000000000        0.0000000000000000     
>>>         1          10          51           6   1.0000000000000000        0.0000000000000000     
>>>         1          11          51           6   1.0000000000000000        0.0000000000000000     
>>>         1          12          51           6   1.0000000000000000        0.0000000000000000     
>>>         1          13          51           6   1.0000000000000000        0.0000000000000000     
>>>         1          14          51           6   1.0000000000000000        0.0000000000000000     
>>>         1          15          51           6   1.0000000000000000        0.0000000000000000     
>>>         1          16          51           6   1.0000000000000000        0.0000000000000000     
>>>         1          17          51           6   1.0000000000000000        0.0000000000000000     
>>>         1          18          51           6   1.0000000000000000        0.0000000000000000     
>>>         1          19          51           6   1.0000000000000000        0.0000000000000000     
>>>         1          20          51           6   1.0000000000000000        0.0000000000000000     
>>>         1          21          51           6   1.0000000000000000        0.0000000000000000     
>>>         1          22          51           6   1.0000000000000000        0.0000000000000000     
>>>         1          23          51           6   1.0000000000000000        0.0000000000000000     
>>>         1          24          51           6   1.0000000000000000        0.0000000000000000     
>>>         1          25          51           6   1.0000000000000000        0.0000000000000000     
>>>         1           6          52           6   0.0000000000000000        0.0000000000000000     
>>>         1           7          52           6   0.0000000000000000        0.0000000000000000     
>>>         1           8          52           6   0.0000000000000000        0.0000000000000000     
>>>         1           9          52           6   0.0000000000000000        0.0000000000000000     
>>>         1          10          52           6   0.0000000000000000        0.0000000000000000     
>>>         1          11          52           6   0.0000000000000000        0.0000000000000000     
>>>         1          12          52           6   0.0000000000000000        0.0000000000000000     
>>>         1          13          52           6   0.0000000000000000        0.0000000000000000     
>>>         1          14          52           6   0.0000000000000000        0.0000000000000000     
>>>         1          15          52           6   0.0000000000000000        0.0000000000000000     
>>>         1          16          52           6   0.0000000000000000        0.0000000000000000     
>>>         1          17          52           6   0.0000000000000000        0.0000000000000000     
>>>         1          18          52           6   0.0000000000000000        0.0000000000000000     
>>>         1          19          52           6   0.0000000000000000        0.0000000000000000     
>>>         1          20          52           6   0.0000000000000000        0.0000000000000000     
>>>         1          21          52           6   0.0000000000000000        0.0000000000000000   
>>>         1          22          51           6   1.0000000000000000        0.0000000000000000     
>>>         1          23          51           6   1.0000000000000000        0.0000000000000000     
>>>         1          24          51           6   1.0000000000000000        0.0000000000000000     
>>>         1          25          51           6   1.0000000000000000        0.0000000000000000     
>>>         1           6          52           6   0.0000000000000000        0.0000000000000000     
>>>         1           7          52           6   0.0000000000000000        0.0000000000000000     
>>>         1           8          52           6   0.0000000000000000        0.0000000000000000     
>>>         1           9          52           6   0.0000000000000000        0.0000000000000000     
>>>         1          10          52           6   0.0000000000000000        0.0000000000000000     
>>>         1          11          52           6   0.0000000000000000        0.0000000000000000     
>>>         1          12          52           6   0.0000000000000000        0.0000000000000000     
>>>         1          13          52           6   0.0000000000000000        0.0000000000000000     
>>>         1          14          52           6   0.0000000000000000        0.0000000000000000     
>>>         1          15          52           6   0.0000000000000000        0.0000000000000000     
>>>         1          16          52           6   0.0000000000000000        0.0000000000000000     
>>>         1          17          52           6   0.0000000000000000        0.0000000000000000     
>>>         1          18          52           6   0.0000000000000000        0.0000000000000000     
>>>         1          19          52           6   0.0000000000000000        0.0000000000000000     
>>>         1          20          52           6   0.0000000000000000        0.0000000000000000     
>>>         1          21          52           6   0.0000000000000000        0.0000000000000000     
>>>         1          22          52           6   0.0000000000000000        0.0000000000000000     
>>>         1          23          52           6   0.0000000000000000        0.0000000000000000     
>>>         1          24          52           6   0.0000000000000000        0.0000000000000000     
>>>         1          25          52           6   0.0000000000000000        0.0000000000000000     
>>> 
>>> 
>>> On May 1, 2013, at 5:06 PM, Rajeev Thakur wrote:
>>> 
>>>> Just before the call to gather, check the value of send_count and check that send_buffer is not all zeros.
>>>> 
>>>> send_count has to be the same on all ranks. Otherwise you have to use Gatherv.
>>>> 
>>>> On May 1, 2013, at 7:03 PM, Ryan Crocker wrote:
>>>> 
>>>>> that's where i checked it with and it's min and max are still zero.  I just amended it so that each send buffer from the different cores are the same size and when i check it on the root gsend_buffer is still incorrect.
>>>>> 
>>>>> On May 1, 2013, at 5:00 PM, Rajeev Thakur wrote:
>>>>> 
>>>>>> gsend_buffer will be valid only on the root.
>>>>>> 
>>>>>> On May 1, 2013, at 6:55 PM, Ryan Crocker wrote:
>>>>>> 
>>>>>>> I just looked at the gsend_buffer, that's actually all zeros as well, and each send_buffer is not.  So i think my problem is there.
>>>>>>> 
>>>>>>> On May 1, 2013, at 4:47 PM, Rajeev Thakur wrote:
>>>>>>> 
>>>>>>>> The count passed to MPI_Scatter should be the local size, i.e., the amount that gets sent to each process. Looks like what is being passed is the global size.
>>>>>>>> 
>>>>>>>> On May 1, 2013, at 6:21 PM, Ryan Crocker wrote:
>>>>>>>> 
>>>>>>>>> Hi all, 
>>>>>>>>> 
>>>>>>>>> So i don't know what the issue with this code snippet is but i cannot get the scatter call to work.  When i print out the recieve_buffer it comes out as all zeros.   The counters without the underscore are the core domains, with the o, the global domains.
>>>>>>>>> 
>>>>>>>>> rec_count  = (imaxo-imino+1)*(jmaxo-jmino+1)*(kmaxo-kmino+1) 
>>>>>>>>> send_count = (imax_-imin_+1)*(jmax_-jmin_+1)*(kmax_-kmin_+1) 
>>>>>>>>> 
>>>>>>>>> call MPI_GATHER  (send_buffer, send_count, MPI_REAL, gsend_buffer, send_count, MPI_REAL, iroot, comm, ierr)
>>>>>>>>> 
>>>>>>>>> rec_count  = (imaxo-imino+1)*(jmaxo-jmino+1)*(kmaxo-kmino+1) 
>>>>>>>>> send_count = (imaxo-imino+1)*(jmaxo-jmino+1)*(kmaxo-kmino+1) 
>>>>>>>>> 
>>>>>>>>> call MPI_SCATTER (gsend_buffer, send_count, MPI_REAL, recieve_buffer, rec_count, MPI_REAL, iroot, comm, ierr)
>>>>>>>>> 
>>>>>>>>> thanks for the help.
>>>>>>>>> 
>>>>>>>>> Ryan Crocker
>>>>>>>>> University of Vermont, School of Engineering
>>>>>>>>> Mechanical Engineering Department
>>>>>>>>> rcrocker at uvm.edu
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> _______________________________________________
>>>>>>>>> discuss mailing list     discuss at mpich.org
>>>>>>>>> To manage subscription options or unsubscribe:
>>>>>>>>> https://lists.mpich.org/mailman/listinfo/discuss
>>>>>>>> 
>>>>>>>> _______________________________________________
>>>>>>>> discuss mailing list     discuss at mpich.org
>>>>>>>> To manage subscription options or unsubscribe:
>>>>>>>> https://lists.mpich.org/mailman/listinfo/discuss
>>>>>>> 
>>>>>>> Ryan Crocker
>>>>>>> University of Vermont, School of Engineering
>>>>>>> Mechanical Engineering Department
>>>>>>> rcrocker at uvm.edu
>>>>>>> 315-212-7331
>>>>>>> 
>>>>>>> _______________________________________________
>>>>>>> discuss mailing list     discuss at mpich.org
>>>>>>> To manage subscription options or unsubscribe:
>>>>>>> https://lists.mpich.org/mailman/listinfo/discuss
>>>>>> 
>>>>>> _______________________________________________
>>>>>> discuss mailing list     discuss at mpich.org
>>>>>> To manage subscription options or unsubscribe:
>>>>>> https://lists.mpich.org/mailman/listinfo/discuss
>>>>> 
>>>>> Ryan Crocker
>>>>> University of Vermont, School of Engineering
>>>>> Mechanical Engineering Department
>>>>> rcrocker at uvm.edu
>>>>> 315-212-7331
>>>>> 
>>>>> _______________________________________________
>>>>> discuss mailing list     discuss at mpich.org
>>>>> To manage subscription options or unsubscribe:
>>>>> https://lists.mpich.org/mailman/listinfo/discuss
>>>> 
>>>> _______________________________________________
>>>> discuss mailing list     discuss at mpich.org
>>>> To manage subscription options or unsubscribe:
>>>> https://lists.mpich.org/mailman/listinfo/discuss
>>> 
>>> Ryan Crocker
>>> University of Vermont, School of Engineering
>>> Mechanical Engineering Department
>>> rcrocker at uvm.edu
>>> 315-212-7331
>>> 
>>> _______________________________________________
>>> discuss mailing list     discuss at mpich.org
>>> To manage subscription options or unsubscribe:
>>> https://lists.mpich.org/mailman/listinfo/discuss
>> 
>> _______________________________________________
>> discuss mailing list     discuss at mpich.org
>> To manage subscription options or unsubscribe:
>> https://lists.mpich.org/mailman/listinfo/discuss
> 
> Ryan Crocker
> University of Vermont, School of Engineering
> Mechanical Engineering Department
> rcrocker at uvm.edu
> 315-212-7331
> 
> _______________________________________________
> discuss mailing list     discuss at mpich.org
> To manage subscription options or unsubscribe:
> https://lists.mpich.org/mailman/listinfo/discuss




More information about the discuss mailing list