[mpich-discuss] gather, send
Rajeev Thakur
thakur at mcs.anl.gov
Wed May 1 19:34:56 CDT 2013
The gather is sending everything to real rank 1. Since you have incremented irank, you need to print the values at irank=2.
On May 1, 2013, at 7:31 PM, Ryan Crocker wrote:
> my call for the memory allocation for the buffers:
>
> call parallel_min_int_0d(imaxo_*jmaxo_*kmaxo_,max_buff)
> allocate(send_buffer(max_buff))
> max_global_buff = max_buff*nproc
> allocate(gsend_buffer (max_global_buff))
>
> in my mpi init subroutine:
>
>
> call MPI_INIT(ierr)
> call MPI_COMM_RANK(MPI_COMM_WORLD,irank,ierr)
> call MPI_COMM_SIZE(MPI_COMM_WORLD,nproc,ierr)
> irank = irank+1
> iroot = 1
>
>
> On May 1, 2013, at 5:21 PM, Rajeev Thakur wrote:
>
>> Is the send_buffer allocated to be large enough to hold max_buff on all ranks?
>>
>> Is gsend_buffer large enough to hold max_buff * nprocs?
>>
>> Is the rank 1, not rank 0, the root?
>>
>> On May 1, 2013, at 7:16 PM, Ryan Crocker wrote:
>>
>>> i'm just checking just the gather call, and the gsend_buffer from the gather call is still empty:
>>>
>>> max_buff is now the largest core domain.
>>>
>>> print*,max_buff -> 24750
>>>
>>> call MPI_GATHER(send_buffer, max_buff, MPI_REAL, gsend_buffer, max_buff, MPI_REAL, iroot, comm, ierr)
>>>
>>> a print out of the send_buffer and the gsend_buffer where it should have non-zero terms.
>>>
>>> print*,irank,i,j,k,send_buffer(i,j,k),gsent_buffer(i,j,k) ->
>>>
>>> 1 6 50 6 0.0000000000000000 0.0000000000000000
>>> 1 7 50 6 0.0000000000000000 0.0000000000000000
>>> 1 8 50 6 0.0000000000000000 0.0000000000000000
>>> 1 9 50 6 0.0000000000000000 0.0000000000000000
>>> 1 10 50 6 0.0000000000000000 0.0000000000000000
>>> 1 11 50 6 0.0000000000000000 0.0000000000000000
>>> 1 12 50 6 0.0000000000000000 0.0000000000000000
>>> 1 13 50 6 0.0000000000000000 0.0000000000000000
>>> 1 14 50 6 0.0000000000000000 0.0000000000000000
>>> 1 15 50 6 0.0000000000000000 0.0000000000000000
>>> 1 16 50 6 0.0000000000000000 0.0000000000000000
>>> 1 17 50 6 0.0000000000000000 0.0000000000000000
>>> 1 18 50 6 0.0000000000000000 0.0000000000000000
>>> 1 19 50 6 0.0000000000000000 0.0000000000000000
>>> 1 20 50 6 0.0000000000000000 0.0000000000000000
>>> 1 21 50 6 0.0000000000000000 0.0000000000000000
>>> 1 22 50 6 0.0000000000000000 0.0000000000000000
>>> 1 23 50 6 0.0000000000000000 0.0000000000000000
>>> 1 24 50 6 0.0000000000000000 0.0000000000000000
>>> 1 25 50 6 0.0000000000000000 0.0000000000000000
>>> 1 6 51 6 1.0000000000000000 0.0000000000000000
>>> 1 7 51 6 1.0000000000000000 0.0000000000000000
>>> 1 8 51 6 1.0000000000000000 0.0000000000000000
>>> 1 9 51 6 1.0000000000000000 0.0000000000000000
>>> 1 10 51 6 1.0000000000000000 0.0000000000000000
>>> 1 11 51 6 1.0000000000000000 0.0000000000000000
>>> 1 12 51 6 1.0000000000000000 0.0000000000000000
>>> 1 13 51 6 1.0000000000000000 0.0000000000000000
>>> 1 14 51 6 1.0000000000000000 0.0000000000000000
>>> 1 15 51 6 1.0000000000000000 0.0000000000000000
>>> 1 16 51 6 1.0000000000000000 0.0000000000000000
>>> 1 17 51 6 1.0000000000000000 0.0000000000000000
>>> 1 18 51 6 1.0000000000000000 0.0000000000000000
>>> 1 19 51 6 1.0000000000000000 0.0000000000000000
>>> 1 20 51 6 1.0000000000000000 0.0000000000000000
>>> 1 21 51 6 1.0000000000000000 0.0000000000000000
>>> 1 22 51 6 1.0000000000000000 0.0000000000000000
>>> 1 23 51 6 1.0000000000000000 0.0000000000000000
>>> 1 24 51 6 1.0000000000000000 0.0000000000000000
>>> 1 25 51 6 1.0000000000000000 0.0000000000000000
>>> 1 6 52 6 0.0000000000000000 0.0000000000000000
>>> 1 7 52 6 0.0000000000000000 0.0000000000000000
>>> 1 8 52 6 0.0000000000000000 0.0000000000000000
>>> 1 9 52 6 0.0000000000000000 0.0000000000000000
>>> 1 10 52 6 0.0000000000000000 0.0000000000000000
>>> 1 11 52 6 0.0000000000000000 0.0000000000000000
>>> 1 12 52 6 0.0000000000000000 0.0000000000000000
>>> 1 13 52 6 0.0000000000000000 0.0000000000000000
>>> 1 14 52 6 0.0000000000000000 0.0000000000000000
>>> 1 15 52 6 0.0000000000000000 0.0000000000000000
>>> 1 16 52 6 0.0000000000000000 0.0000000000000000
>>> 1 17 52 6 0.0000000000000000 0.0000000000000000
>>> 1 18 52 6 0.0000000000000000 0.0000000000000000
>>> 1 19 52 6 0.0000000000000000 0.0000000000000000
>>> 1 20 52 6 0.0000000000000000 0.0000000000000000
>>> 1 21 52 6 0.0000000000000000 0.0000000000000000
>>> 1 22 51 6 1.0000000000000000 0.0000000000000000
>>> 1 23 51 6 1.0000000000000000 0.0000000000000000
>>> 1 24 51 6 1.0000000000000000 0.0000000000000000
>>> 1 25 51 6 1.0000000000000000 0.0000000000000000
>>> 1 6 52 6 0.0000000000000000 0.0000000000000000
>>> 1 7 52 6 0.0000000000000000 0.0000000000000000
>>> 1 8 52 6 0.0000000000000000 0.0000000000000000
>>> 1 9 52 6 0.0000000000000000 0.0000000000000000
>>> 1 10 52 6 0.0000000000000000 0.0000000000000000
>>> 1 11 52 6 0.0000000000000000 0.0000000000000000
>>> 1 12 52 6 0.0000000000000000 0.0000000000000000
>>> 1 13 52 6 0.0000000000000000 0.0000000000000000
>>> 1 14 52 6 0.0000000000000000 0.0000000000000000
>>> 1 15 52 6 0.0000000000000000 0.0000000000000000
>>> 1 16 52 6 0.0000000000000000 0.0000000000000000
>>> 1 17 52 6 0.0000000000000000 0.0000000000000000
>>> 1 18 52 6 0.0000000000000000 0.0000000000000000
>>> 1 19 52 6 0.0000000000000000 0.0000000000000000
>>> 1 20 52 6 0.0000000000000000 0.0000000000000000
>>> 1 21 52 6 0.0000000000000000 0.0000000000000000
>>> 1 22 52 6 0.0000000000000000 0.0000000000000000
>>> 1 23 52 6 0.0000000000000000 0.0000000000000000
>>> 1 24 52 6 0.0000000000000000 0.0000000000000000
>>> 1 25 52 6 0.0000000000000000 0.0000000000000000
>>>
>>>
>>> On May 1, 2013, at 5:06 PM, Rajeev Thakur wrote:
>>>
>>>> Just before the call to gather, check the value of send_count and check that send_buffer is not all zeros.
>>>>
>>>> send_count has to be the same on all ranks. Otherwise you have to use Gatherv.
>>>>
>>>> On May 1, 2013, at 7:03 PM, Ryan Crocker wrote:
>>>>
>>>>> that's where i checked it with and it's min and max are still zero. I just amended it so that each send buffer from the different cores are the same size and when i check it on the root gsend_buffer is still incorrect.
>>>>>
>>>>> On May 1, 2013, at 5:00 PM, Rajeev Thakur wrote:
>>>>>
>>>>>> gsend_buffer will be valid only on the root.
>>>>>>
>>>>>> On May 1, 2013, at 6:55 PM, Ryan Crocker wrote:
>>>>>>
>>>>>>> I just looked at the gsend_buffer, that's actually all zeros as well, and each send_buffer is not. So i think my problem is there.
>>>>>>>
>>>>>>> On May 1, 2013, at 4:47 PM, Rajeev Thakur wrote:
>>>>>>>
>>>>>>>> The count passed to MPI_Scatter should be the local size, i.e., the amount that gets sent to each process. Looks like what is being passed is the global size.
>>>>>>>>
>>>>>>>> On May 1, 2013, at 6:21 PM, Ryan Crocker wrote:
>>>>>>>>
>>>>>>>>> Hi all,
>>>>>>>>>
>>>>>>>>> So i don't know what the issue with this code snippet is but i cannot get the scatter call to work. When i print out the recieve_buffer it comes out as all zeros. The counters without the underscore are the core domains, with the o, the global domains.
>>>>>>>>>
>>>>>>>>> rec_count = (imaxo-imino+1)*(jmaxo-jmino+1)*(kmaxo-kmino+1)
>>>>>>>>> send_count = (imax_-imin_+1)*(jmax_-jmin_+1)*(kmax_-kmin_+1)
>>>>>>>>>
>>>>>>>>> call MPI_GATHER (send_buffer, send_count, MPI_REAL, gsend_buffer, send_count, MPI_REAL, iroot, comm, ierr)
>>>>>>>>>
>>>>>>>>> rec_count = (imaxo-imino+1)*(jmaxo-jmino+1)*(kmaxo-kmino+1)
>>>>>>>>> send_count = (imaxo-imino+1)*(jmaxo-jmino+1)*(kmaxo-kmino+1)
>>>>>>>>>
>>>>>>>>> call MPI_SCATTER (gsend_buffer, send_count, MPI_REAL, recieve_buffer, rec_count, MPI_REAL, iroot, comm, ierr)
>>>>>>>>>
>>>>>>>>> thanks for the help.
>>>>>>>>>
>>>>>>>>> Ryan Crocker
>>>>>>>>> University of Vermont, School of Engineering
>>>>>>>>> Mechanical Engineering Department
>>>>>>>>> rcrocker at uvm.edu
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> discuss mailing list discuss at mpich.org
>>>>>>>>> To manage subscription options or unsubscribe:
>>>>>>>>> https://lists.mpich.org/mailman/listinfo/discuss
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> discuss mailing list discuss at mpich.org
>>>>>>>> To manage subscription options or unsubscribe:
>>>>>>>> https://lists.mpich.org/mailman/listinfo/discuss
>>>>>>>
>>>>>>> Ryan Crocker
>>>>>>> University of Vermont, School of Engineering
>>>>>>> Mechanical Engineering Department
>>>>>>> rcrocker at uvm.edu
>>>>>>> 315-212-7331
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> discuss mailing list discuss at mpich.org
>>>>>>> To manage subscription options or unsubscribe:
>>>>>>> https://lists.mpich.org/mailman/listinfo/discuss
>>>>>>
>>>>>> _______________________________________________
>>>>>> discuss mailing list discuss at mpich.org
>>>>>> To manage subscription options or unsubscribe:
>>>>>> https://lists.mpich.org/mailman/listinfo/discuss
>>>>>
>>>>> Ryan Crocker
>>>>> University of Vermont, School of Engineering
>>>>> Mechanical Engineering Department
>>>>> rcrocker at uvm.edu
>>>>> 315-212-7331
>>>>>
>>>>> _______________________________________________
>>>>> discuss mailing list discuss at mpich.org
>>>>> To manage subscription options or unsubscribe:
>>>>> https://lists.mpich.org/mailman/listinfo/discuss
>>>>
>>>> _______________________________________________
>>>> discuss mailing list discuss at mpich.org
>>>> To manage subscription options or unsubscribe:
>>>> https://lists.mpich.org/mailman/listinfo/discuss
>>>
>>> Ryan Crocker
>>> University of Vermont, School of Engineering
>>> Mechanical Engineering Department
>>> rcrocker at uvm.edu
>>> 315-212-7331
>>>
>>> _______________________________________________
>>> discuss mailing list discuss at mpich.org
>>> To manage subscription options or unsubscribe:
>>> https://lists.mpich.org/mailman/listinfo/discuss
>>
>> _______________________________________________
>> discuss mailing list discuss at mpich.org
>> To manage subscription options or unsubscribe:
>> https://lists.mpich.org/mailman/listinfo/discuss
>
> Ryan Crocker
> University of Vermont, School of Engineering
> Mechanical Engineering Department
> rcrocker at uvm.edu
> 315-212-7331
>
> _______________________________________________
> discuss mailing list discuss at mpich.org
> To manage subscription options or unsubscribe:
> https://lists.mpich.org/mailman/listinfo/discuss
More information about the discuss
mailing list