[mpich-devel] memory profiling HOWTO?

[Dave Goodell]

There are two kinds of memory that MPICH with ch3:nemesis allocates:

1) regular heap memory via malloc (with interposition possible via MPIU_Malloc and friends)

2) shared memory via one of the usual mechanisms (mmap, shm_open, etc.)

You can determine the current amount of #1 by calling "MPIU_trspace" as long as you configure with "--enable-g=mem" (or some superset thereof).  You'll get some additional performance overhead when allocating/freeing memory if you configure this way, since MPICH is doing some additional allocation and bookkeeping in order to track this information for you.  If you need a lower overhead method for doing this, you'll need to modify/intercept MPIU_Malloc and friends yourself.  You'll need an "extern" declaration for "MPIU_trspace" in order to call it from your test program.

You can use the attached (untested against HEAD) patch to give you a hook for #2.  You'll need to throw down an "extern" declaration for the routine in your test program as well.

-Dave

-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-WIP-MPIDI_CH3I_Seg_get_size-shared-mem-profiling-rou.patch
Type: application/octet-stream
Size: 1168 bytes
Desc: not available
URL: <http://lists.mpich.org/pipermail/devel/attachments/20130223/ba123f04/attachment.obj>
-------------- next part --------------


On Feb 22, 2013, at 9:28 AM CST, Jim Dinan wrote:

> Jeff,
> 
> Which memory are you interested in -- memory the application allocates, or memory that MPICH allocates?  If the former, you can use the PMPI interface to hook MPI_Alloc/Free_mem.  If you are interested in MPICH internal memory use, Dave can probably give the most informed answer.  I would probably just modify MPIU_Malloc.
> 
> ~Jim.
> 
> On 2/22/13 8:44 AM, Jeff Hammond wrote:
>> Is there any documentation of how to properly instrument MPICH with
>> memory profiling?  I understand MPIU_MALLOC allows me to do this.
>> 
>> My goal is to merely print out every time memory is allocated larger
>> than a particular size.  I'd also like the rank to show up in the
>> output.
>> 
>> Please RTM me if the M exists.  Source comments are also a good M if
>> they are correct and sufficiently complete.
>> 
>> Thanks!
>> 
>> Jeff
>>