[mpich-devel] small MPI_Win_allocate

[Balaji, Pavan]

Yes, TLB would be a concern, particularly because shared memory (mmap backed by a file) uses regular-sized pages, not large pages.

The shared memory pool is a bit messy because we have to remember how much memory in each page was already given away and deal with fragmentation, etc.

A simpler "optimization" might be to turn off noncontiguity after some number of window allocations.

  -- Pavan

> On May 17, 2016, at 2:31 PM, Jeff Hammond <jeff.science at gmail.com> wrote:
> 
> I always ask for noncontig window allocations.  I use as many RMA info keys as possible :-)
> 
> I am not so worried about running out of VA bits, although 48 bits don't go as far as they used to, as I am about TLB slots.
> 
> However, this problem can be solved above MPI, it just requires me to do it in every MPI-3 code I write.
> 
> Jeff
> 
> On Tue, May 17, 2016 at 12:20 PM, Balaji, Pavan <balaji at anl.gov> wrote:
> 
> It does allocate from shared memory irrespective of the size.  It's not page-aligned unless you ask for noncontig allocations.  With respect to wasted VM space, you'll run out of window contexts much before you run out of VM space, so I'm not sure this is a worthwhile optimization.
> 
>   -- Pavan
> 
> > On May 12, 2016, at 4:54 PM, Jeff Hammond <jeff.science at gmail.com> wrote:
> >
> > If I call MPI_Win_allocate for e.g. 8 bytes, does it allocate shared memory every time, and thus potentially allocate with page alignment, which could cause a lot of empty VM space?
> >
> > Perhaps I do not understand POSIX shared memory well enough, but it strikes me that it might be a worthwhile optimization to have a memory pool for small-size MPI_Win_allocate(_shared) calls.
> >
> > Thanks,
> >
> > Jeff
> >
> > --
> > Jeff Hammond
> > jeff.science at gmail.com
> > http://jeffhammond.github.io/
> > _______________________________________________
> > To manage subscription options or unsubscribe:
> > https://lists.mpich.org/mailman/listinfo/devel
> 
> _______________________________________________
> To manage subscription options or unsubscribe:
> https://lists.mpich.org/mailman/listinfo/devel
> 
> 
> 
> -- 
> Jeff Hammond
> jeff.science at gmail.com
> http://jeffhammond.github.io/
> _______________________________________________
> To manage subscription options or unsubscribe:
> https://lists.mpich.org/mailman/listinfo/devel