[mpich-devel] ROMIO for Lustre

[Rob Latham]

On Sat, Sep 21, 2013 at 11:21:19PM -0500, Jaln wrote:
> Hi everyone,
> 
> I'm not sure, whether the lustre or the MPI forum is the right place for my
> question.

both, i guess :>

> The question is about the ROMIO optimization on Lustre,
> In one SC'08 paper,
> http://users.eecs.northwestern.edu/~wkliao/PAPERS/fd_sc08_revised.pdf<https://mail.ttu.edu/owa/redir.aspx?C=yUmbVUH4hUWLFEWFA2GcoiKOEhnhitAIatZfGT92-aN2MTXitjDjPgfE9EfJkJF9q3XAaOQ_iME.&URL=http%3a%2f%2fusers.eecs.northwestern.edu%2f%7ewkliao%2fPAPERS%2ffd_sc08_revised.pdf>
> , it's said that the way ROMIO assigns the file domains to I/O aggregators
> will not make two aggregators access the same OST.
> 
> In my understanding, this means, the data locality on Lustre layer has been
> taken care of in the ROMIO, such that the aggregators will not
> have competition on the same OST.
> 
> My question is "is this optimization used in all current lustre system,
> e.g., Hopper at NERSC?"

Wei-keng never contributed the specific ROMIO optimizations he discussed in
the SC 08 paper, but his work did spur a lot of community discussion
and contributions.  

Emoly Lu contributed a bunch of Lustre ADIO driver work, which Pascal
Deveze and Martin Pokorny improved upon.   MPICH-1.3 and newer contain
these improvements.

David Knaak from Cray implemented his own improvements.  Cray's MPI-IO
is based on ROMIO but the cray modifications are proprietary. MPT-3.2
and newer contain lustre-specific optimizations.

The community has been quiet with respect to Lustre MPI-IO work since
then.  I hope that's because everything "just works". 

==rob

-- 
Rob Latham
Mathematics and Computer Science Division
Argonne National Lab, IL USA