[mpich-devel] ROMIO collective i/o memory use
rross at mcs.anl.gov
Mon May 6 15:03:52 CDT 2013
Let's separate collective I/O the concept from collective I/O the implementation in ROMIO.
Lots of things we *could* do:
- switch to a different internal representation of noncontiguous regions, perhaps dramatically decreasing memory allocation when exchanging regions
- switch to a different collective I/O algorithm that uses some hierarchy to reduce number of processes for which space must be allocated
- don't globally coordinate, but rather coordinate more loosely to reduce communication and memory allocation requirements
I'm sure we can come up with more ideas, some of which would be new research and some of which would be mostly engineering. All relatively major activities. Thus my question :).
On May 6, 2013, at 2:30 PM, Bob Cernohous wrote:
> > From: Rob Ross <rross at mcs.anl.gov>
> > Should we consider this as interest in working on this problem on
> > the IBM side :)? -- Rob
> Say what?! ;)
> Meaning we can get rid of all o(p) allocations? Not sure how you do internal collectives on behalf of the app/mpi-io without at least some of those. I was looking more for agreement that collective i/o is 'what it is'... and maybe some idea if we just have some known limitations on scaling it. Yes, that BG alltoallv is a bigger problem that we can avoid with an env var -- is that just going to have to be 'good enough'? (I think that Jeff P wrote that on BG/P and got good performance with that alltoallv. Trading memory for performance, not unusual, and at least it's selectable.)
More information about the devel