[mpich-discuss] reading from a read only directory

[Rob Latham]

On Tue, Dec 03, 2013 at 12:58:53PM -0800, Geoffrey Irving wrote:
> > Mea culpa.  MPI_File_read_ordered is implemented in "stupid mode"
> > right now, and no one ever complained -- we don't pay much attention
> > to shared file pointers.  Tell me more about how you are relying on
> > ordered mode, please.
> 
> I'm dividing a large file into nearly equal size chunks and slurping
> the entire file into RAM.  Then I do a bunch of rearrangement in
> memory.  I could just as easily use MPI_File_read_at_all, but had
> assumed that ordered was better because my known intervals happen to
> be ordered and contiguous.  Should I be using MPI_File_read_at_all
> instead, and treating the ordered routine as vestigial?

Yeah, the ordered mode routines, to borrow the "self-consistent MPI"
model, are more strict than MPI_File_read_at_all -- not only are all
processes participating, but they are participating in I/O that must
end up in the file in rank-order.

This is probably not a significant portion of your runtime, but have
you considered combining mpi memory datatypes with
mpi_file_read_at_all to get the read and the memory rearrangement in
one go?

> Okay, I was basically assuming that if I used a collective routine, it
> would be smart about accessing the file system as few times as
> possible.  If this isn't true I should unqueue my job now and switch
> to MPI_File_read_at_all.  Is this right?  Should I also use
> MPI_File_write_at_all instead of the shared fp collective write
> routine?

Yeah, MPI_File_write_at_all and MPI_File_read_at_all will be the smart
clever routines you are looking for -- though on the Cray they may
need a bit of tuning.  I *think* "good mode" is now the default, but I
don't have a lot of experience with Cray's modifications to ROMIO.

==rob

> > I wrote a little ordered mode shim you can link into your program.
> > Looks like I last seriously did anything with it back in 2008 ?  I
> > cleaned up some of the warnings and made it use MPI_Count (remove
> > HAVE_MPI_TYPE_SIZE_X from the makefile if you're on an old system).
> > Caveat: I think the last time I ran this code was on Argonne's Blue
> > Gene /L.  Bear in mind the adage about tested code and broken code.
> 
> Thanks, though in my case switching away from the shared file pointer
> routine is easy, so I don't need the shim.
> 
> Geoffrey

-- 
Rob Latham
Mathematics and Computer Science Division
Argonne National Lab, IL USA