[mpich-discuss] How To Limit Memory for each MPI Process

[Bowen Yu]

Hi,


It's easy to control the memory usage of Java containers. Specify java parameters like -Xmx when launch the containers and JVM will assure the maximum memory usage. However, limitation on containers is not enough. When the Application Master invokes mpiexec, Hydra will spawn proxy to all the nodes needed by this application via ssh. All the processes are forked by Hydra Process Manager Proxies, which is not controlled by the container. 


I don't know if MPI has mechanism to limit memory for each MPI processes. What I really mean is like I specify MAX_MEMORY_USAGE_PER_PROCESS in environment variable, and I am assured that none of the MPI processes will exceed that amount. 




------------------ Original ------------------
From:  "Lu, Huiwei";<huiweilu at mcs.anl.gov>;
Date:  Thu, Sep 11, 2014 08:49 PM
To:  "discuss at mpich.org"<discuss at mpich.org>; 

Subject:  Re: [mpich-discuss] How To Limit Memory for each MPI Process



Hi Steven,

It’s good to know that you have enabled MPICH to work with Hadoop YARN.

So you have a Java container inside a MPI process and want to limit the memory usage of a container. A MPI process is as the same as a system process. And you can limit its resource usage use either ulimit or cgroup. However, if the Java container is not aware of the limit and continue to allocate the memory, it will still exceed the memory usage. I don’t know if I understand the problem correctly. But maybe it’s better to limit the memory usage of a Java container. Is there a way to limit the memory usage of a Java container?

Thanks.
—
Huiwei Lu
Postdoc Appointee
Mathematics and Computer Science Division, Argonne National Laboratory
http://www.mcs.anl.gov/~huiweilu/

On Sep 11, 2014, at 6:55 AM, Bowen Yu <jxb002 at qq.com> wrote:

> Hi,
> 
> I'm developing a application that enables MPICH executables running at Hadoop YARN cluster, and most functionality has been finished: https://github.com/alibaba/mpich2-yarn. This MPICH-YARN uses MPICH-3.1.2 to run MPI executable.
> 
> YARN allocate resources in container, and in one container there are specific amount of memory and CPU virtual cores. MPICH-YARN assumes one MPI process is one-to-one correspondence with one container, so the MPI process' memory should be limited. But I have no idea how. How to do so that when I run mpiexec, each process is running with a limited resource, such as memory, and CPU utilization; and if one of the process' memory exceeds, the MPI whole program fails?
> 
> I know two ways to implement resource limitation in Linux, one is to use system call in programs or ulimit command in shell; the other is to use cgroup kernel module.
> 
> Thanks!
> Steven
> _______________________________________________
> discuss mailing list     discuss at mpich.org
> To manage subscription options or unsubscribe:
> https://lists.mpich.org/mailman/listinfo/discuss

_______________________________________________
discuss mailing list     discuss at mpich.org
To manage subscription options or unsubscribe:
https://lists.mpich.org/mailman/listinfo/discuss
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mpich.org/pipermail/discuss/attachments/20140912/0fd2b6fd/attachment.html>