[mpich-discuss] The max amount of data each process can send in MPI_Alltoallv

Jeff Hammond jeff.science at gmail.com
Sun Dec 15 20:07:06 CST 2013


On Sun, Dec 15, 2013 at 7:21 PM, Pavan Balaji <balaji at mcs.anl.gov> wrote:
>
> On Dec 16, 2013, at 9:49 AM, Huiwei Lu <huiweilu at mcs.anl.gov> wrote:
>> @JT, yes, if the data continue to grow and there is no proper way to do it in MPI type, I will have to partition the buffer to smaller trunks.
>
> MPI (standard) can send infinitely large data using Rajeev’s suggestion.  The internal MPICH stack can allow up to 64-bits (16 exabytes) because of its internal bookkeeping.  When people start sending more than 16 exabytes in a single message, we’ll fix that.  :-)

Let us define Big Data to be the domain of computing wherein 64b
indexing is not sufficient :-)

Jeff

-- 
Jeff Hammond
jeff.science at gmail.com



More information about the discuss mailing list