[mpich-discuss] Single node IO Aggregator setup

Rob Latham robl at mcs.anl.gov
Tue Jun 21 13:08:20 CDT 2016

On 06/21/2016 12:48 PM, Jon Povich wrote:
> I'm trying to simulate a cluster setup where only the I/O aggregators
> have access to the working directory. Is this feasible to do with romio
> hints?
> Setup:
>    A) Work dir is on Node0's local hard drive.
>    B) Remote Node1 has no access to Node0's hard drive
>    C) Run a case where only rank 0 on Node0 serves as the I/O aggregator

that should work as long as you only use collective operations.  You'll 
trigger the "deferred open" optimization.


Your hints are close.

>     MPI_Info_create(&info);
>     MPI_Info_set(info, "cb_nodes", "1");
>     MPI_Info_set(info, "no_indep_rw", "true");

The hint is actually "romio_no_indep_rw".   Where did you find 
"no_indep_rw" documented?  I'd like to get that fixed.

I'd adjust one more hint: the "cb_config_list" hint.  By default it is 
"*:1", which means select one MPI process from each node to act as 
aggregator.  That's *probably* node0/crane/whatever, but you can ensure 
the desired node is selected by being more explicit.

I'd make your cb_config_list look like "crane:1" or "node0:1" or 
whatever MPI_Get_processor_name returns.

In 2014 we went on a deferred open cleanup and fixed a few bugs. I hope 
you are using MPICH-3.1.3 or newer.

discuss mailing list     discuss at mpich.org
To manage subscription options or unsubscribe:

More information about the discuss mailing list