[mpich-discuss] mpi_shared memory

[Arash sharify]

Hello all
my name is Arash Hajisharifi and I am a phd student in computational fluid dynamics, Iam working on a code which has been parallelizaed with MPI.
 I need to improve the parallelisation to reach better performance.
I have a 3D domain which has decomposed as 2D domain decomposition and each rank has a stencil of the domain.

In my case I have a large 3D array for example Array_total(nx,ny,nz), after decomposition each rank has a part this array:#0 : array_partial(nx,ny/4,nz/4) in rank 0#1  :array_partial(nx,ny/4,nz/4) in rank 1
...

#15  :array_partial(nx,ny/4,nz/4) in rank 15

After some operation, each rank should see Array_partial of every rank, I could not use allgather for each rank because Array_total is so large (about 10 GIG) to gather it inside the local buffer of each rank and can not fit inside and the only way is mpi_shared memory by which each rank shared its part .
I know the idea of mpi_shared memory to creat a window among the processes, this idea could be feasible for the processes in one node which the ranks have a shared memory, but it is also possible to creat a window for the processes between 2 nodes? I mean creating one window among all the processes in node 1 and node 2?
Unfortunately I am searching for an easy example or a good documentation specially in Fortran  for 2 weeks but there is nothing,i don't know how is it possible to not find any thing for one mpi functinlaity which has released 3 years ago and it is not new anymore.

May I ask you kindly to guid me or send me some simple example to understand the idea?
looking forward to hearing from you,
Best Regards,Arash Hajisharifi
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mpich.org/pipermail/discuss/attachments/20190524/01f1fc02/attachment.html>