[mpich-discuss] reading from a read only directory

[Geoffrey Irving]

On Tue, Dec 3, 2013 at 2:10 PM, Rob Latham <robl at mcs.anl.gov> wrote:
> On Tue, Dec 03, 2013 at 12:58:53PM -0800, Geoffrey Irving wrote:
>> > Mea culpa.  MPI_File_read_ordered is implemented in "stupid mode"
>> > right now, and no one ever complained -- we don't pay much attention
>> > to shared file pointers.  Tell me more about how you are relying on
>> > ordered mode, please.
>>
>> I'm dividing a large file into nearly equal size chunks and slurping
>> the entire file into RAM.  Then I do a bunch of rearrangement in
>> memory.  I could just as easily use MPI_File_read_at_all, but had
>> assumed that ordered was better because my known intervals happen to
>> be ordered and contiguous.  Should I be using MPI_File_read_at_all
>> instead, and treating the ordered routine as vestigial?
>
> Yeah, the ordered mode routines, to borrow the "self-consistent MPI"
> model, are more strict than MPI_File_read_at_all -- not only are all
> processes participating, but they are participating in I/O that must
> end up in the file in rank-order.

Thanks, I'll switch to ..._read_at_all.

> This is probably not a significant portion of your runtime, but have
> you considered combining mpi memory datatypes with
> mpi_file_read_at_all to get the read and the memory rearrangement in
> one go?

The rearrangement step is 2 orders of magnitude faster than the I/O,
so it doesn't matter in this case, but datatypes do seem cleaner for
future versions of similar code.

>> Okay, I was basically assuming that if I used a collective routine, it
>> would be smart about accessing the file system as few times as
>> possible.  If this isn't true I should unqueue my job now and switch
>> to MPI_File_read_at_all.  Is this right?  Should I also use
>> MPI_File_write_at_all instead of the shared fp collective write
>> routine?
>
> Yeah, MPI_File_write_at_all and MPI_File_read_at_all will be the smart
> clever routines you are looking for -- though on the Cray they may
> need a bit of tuning.  I *think* "good mode" is now the default, but I
> don't have a lot of experience with Cray's modifications to ROMIO.

Good to know.  Thanks!

Geoffrey