[mpich-devel] MPI_Dims_create() fails for nproc >1000000

[Jeff Hammond]

Indeed, I also wrote a C program to generate primes and found that it was fast enough to justify generating the list on-the-fly for every call.

No one expects MPI_Cart_create to be optimized. And I favor saving space at the cost of time when it comes to text/data/bss (wherever static arrays go).

And in how many cases does prime generation take longer than the context id allreduce, etc.?

Jeff

Sent from my iPhone

> On Dec 19, 2013, at 6:58 PM, Pavan Balaji <balaji at mcs.anl.gov> wrote:
> 
> 
> Increasing the number of indices to 4792 is the faster approach, if we are looking for a quick solution.  It’s also safer, since the code is a bit tricky to get right.
> 
>  — Pavan
> 
>> On Dec 20, 2013, at 8:50 AM, Pavan Balaji <balaji at mcs.anl.gov> wrote:
>> 
>> 
>> The mixed approach sounds good (it’s fast for “small systems”).
>> 
>> But for completeness I should point out a couple of things:
>> 
>> 1. If we want to do this statically, the maximum number of MPI processes in an application is MAX_INT.  That requires 4792 indices (19KB). So doing a "full set" of primes is not too space consuming.
>> 
>> 2. If we want to do this at runtime (presumably at INIT time), my super-inefficient sequential script with gcc and no optimizations took 40 milliseconds to run.  So it should not be too bad time wise either.
>> 
>> — Pavan
>> 
>>> On Dec 20, 2013, at 2:24 AM, Antonio J. Peña <apenya at anl.gov> wrote:
>>> 
>>> 
>>> What about a mixed approach? If we've received no complaints so far that means that the current LUT is fine in most cases. We could fall into runtime computations if it's larger that that value. That will introduce a forced check, but the overhead will be lower than always computing, while the memory consumption would keep the same and the values not limited.
>>> 
>>> Toni
>>> 
>>> 
>>>> On 12/19/2013 11:13 AM, Jeff Hammond wrote:
>>>> I started working on a patch, but I'm not sure if it is the right
>>>> thing to just encode a longer static array of primes.
>>>> 
>>>> Any suggestions on how to proceed?  Should I generate the array of
>>>> primes on-the-fly (once per MPI instance) up to the max possible
>>>> amount?
>>>> 
>>>> Jeff
>>>> 
>>>> 
>>>> ---------- Forwarded message ----------
>>>> From: Jeff Hammond <jeff.science at gmail.com>
>>>> Date: Thu, Dec 19, 2013 at 10:55 AM
>>>> Subject: Re: [mpich-discuss] MPI_Dims_create() fails for nproc >1000000
>>>> To: discuss at mpich.org
>>>> 
>>>> 
>>>> Can you provide a short test that reproduces the issue you are seeing?
>>>> I will add a Trac ticket for it.
>>>> 
>>>> Thanks,
>>>> 
>>>> Jeff
>>>> 
>>>>> On Thu, Dec 19, 2013 at 9:49 AM, Andreas Schäfer <gentryx at gmx.de> wrote:
>>>>> Hi,
>>>>> 
>>>>> I noticed that MPICH's MPI_Dims_create() fails with a floating point
>>>>> exception, if the number of processes is larger than 10^6. Apparently
>>>>> the reason is that the prime factors are hard coded into the code and
>>>>> are limited to 1000 (or sqrt(10^6)). Open MPI's implementation has no
>>>>> such limitation.
>>>>> 
>>>>> Best
>>>>> -Andreas
>>>>> 
>>>>> 
>>>>> --
>>>>> ==========================================================
>>>>> Andreas Schäfer
>>>>> HPC and Grid Computing
>>>>> Chair of Computer Science 3
>>>>> Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany
>>>>> +49 9131 85-27910
>>>>> PGP/GPG key via keyserver
>>>>> http://www.libgeodecomp.org
>>>>> ==========================================================
>>>>> 
>>>>> (\___/)
>>>>> (+'.'+)
>>>>> (")_(")
>>>>> This is Bunny. Copy and paste Bunny into your
>>>>> signature to help him gain world domination!
>>>>> 
>>>>> _______________________________________________
>>>>> discuss mailing list     discuss at mpich.org
>>>>> To manage subscription options or unsubscribe:
>>>>> https://lists.mpich.org/mailman/listinfo/discuss
>>>> 
>>>> 
>>>> --
>>>> Jeff Hammond
>>>> jeff.science at gmail.com
>>> 
>>> 
>>> -- 
>>> Antonio J. Peña
>>> Postdoctoral Appointee
>>> Mathematics and Computer Science Division
>>> Argonne National Laboratory
>>> 9700 South Cass Avenue, Bldg. 240, Of. 3148
>>> Argonne, IL 60439-4847
>>> apenya at mcs.anl.gov
>>> www.mcs.anl.gov/~apenya
>> 
>> --
>> Pavan Balaji
>> http://www.mcs.anl.gov/~balaji
> 
> --
> Pavan Balaji
> http://www.mcs.anl.gov/~balaji