[mpich-devel] ROMIO collective i/o memory use

[Rob Ross]

Hi,

Let's separate collective I/O the concept from collective I/O the implementation in ROMIO.

Lots of things we *could* do:
- switch to a different internal representation of noncontiguous regions, perhaps dramatically decreasing memory allocation when exchanging regions
- switch to a different collective I/O algorithm that uses some hierarchy to reduce number of processes for which space must be allocated
- don't globally coordinate, but rather coordinate more loosely to reduce communication and memory allocation requirements
- …

I'm sure we can come up with more ideas, some of which would be new research and some of which would be mostly engineering. All relatively major activities. Thus my question :).

Rob

On May 6, 2013, at 2:30 PM, Bob Cernohous wrote:

> 
> > From: Rob Ross <rross at mcs.anl.gov> 
> > 
> > Should we consider this as interest in working on this problem on 
> > the IBM side :)? -- Rob
> 
> Say what?! ;) 
> 
> Meaning we can get rid of all o(p) allocations?  Not sure how you do internal collectives on behalf of the app/mpi-io without at least some of those.  I was looking more for agreement that collective i/o is 'what it is'... and maybe some idea if we just have some known limitations on scaling it.  Yes, that BG alltoallv is a bigger problem that we can avoid with an env var -- is that just going to have to be 'good enough'?  (I think that Jeff P wrote that on BG/P and got good performance with that alltoallv.  Trading memory for performance, not unusual, and at least it's selectable.)