[mpich-devel] memory profiling HOWTO?

[Jeff Hammond]

I think I need to override MPIU_Malloc myself for this but I will also
try --enable-g=mem and MPIU_trspace to see how the information I get
from that lines up with what I get from the BGQ SPI (which can track
everything: heap, stack, shm, ...).

Thanks for all the pointers.

Jeff

On Sat, Feb 23, 2013 at 10:16 AM, Dave Goodell <goodell at mcs.anl.gov> wrote:
> There are two kinds of memory that MPICH with ch3:nemesis allocates:
>
> 1) regular heap memory via malloc (with interposition possible via MPIU_Malloc and friends)
>
> 2) shared memory via one of the usual mechanisms (mmap, shm_open, etc.)
>
> You can determine the current amount of #1 by calling "MPIU_trspace" as long as you configure with "--enable-g=mem" (or some superset thereof).  You'll get some additional performance overhead when allocating/freeing memory if you configure this way, since MPICH is doing some additional allocation and bookkeeping in order to track this information for you.  If you need a lower overhead method for doing this, you'll need to modify/intercept MPIU_Malloc and friends yourself.  You'll need an "extern" declaration for "MPIU_trspace" in order to call it from your test program.
>
> You can use the attached (untested against HEAD) patch to give you a hook for #2.  You'll need to throw down an "extern" declaration for the routine in your test program as well.
>
> -Dave
>
>
>
>
> On Feb 22, 2013, at 9:28 AM CST, Jim Dinan wrote:
>
>> Jeff,
>>
>> Which memory are you interested in -- memory the application allocates, or memory that MPICH allocates?  If the former, you can use the PMPI interface to hook MPI_Alloc/Free_mem.  If you are interested in MPICH internal memory use, Dave can probably give the most informed answer.  I would probably just modify MPIU_Malloc.
>>
>> ~Jim.
>>
>> On 2/22/13 8:44 AM, Jeff Hammond wrote:
>>> Is there any documentation of how to properly instrument MPICH with
>>> memory profiling?  I understand MPIU_MALLOC allows me to do this.
>>>
>>> My goal is to merely print out every time memory is allocated larger
>>> than a particular size.  I'd also like the rank to show up in the
>>> output.
>>>
>>> Please RTM me if the M exists.  Source comments are also a good M if
>>> they are correct and sufficiently complete.
>>>
>>> Thanks!
>>>
>>> Jeff
>>>
>
>



-- 
Jeff Hammond
Argonne Leadership Computing Facility
University of Chicago Computation Institute
jhammond at alcf.anl.gov / (630) 252-5381
http://www.linkedin.com/in/jeffhammond
https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond