[mpich-discuss] Custom rank for processes

Zhou, Hui zhouh at anl.gov
Mon Jul 1 15:43:10 CDT 2024 

Hi Niyaz,

With 5 processes and two hosts, by default it is equivalent to "-rankmap (vector,(0,2,5))​" or "-rankmap (vector,0,1,0,1,0)", which is the same with option "-ppn 1"-- a round-robin assignment of one process per host. The result is, rank 0,2,4 on the fist host and rank 1,3 on the second host. Let me know what is your desired rank ordering, I'll help you on the options.

You can use "-bind-to" to specify binding. Unfortunately, you only can specify the same binding policy for all hosts.

The option "-bind-to user:0+2,1+4,3,2" binds the first process to core 0 and 2, the 2nd process to core 1 and 4, the 3rd process to core 3, the 4th process to core 2. If there are only 3 or 2 processes on the node, the later binding specifications are ignored.

--
Hui
________________________________
From: Niyaz Murshed <Niyaz.Murshed at arm.com>
Sent: Monday, July 1, 2024 3:17 PM
To: Zhou, Hui <zhouh at anl.gov>; discuss at mpich.org <discuss at mpich.org>; Jenke, Joachim <jenke at itc.rwth-aachen.de>
Cc: nd <nd at arm.com>
Subject: Re: Custom rank for processes

Thank you Joachim and Hui. -rankmap {rank map} comma separated rank to node id list How is the format if I have 4 processes ? mpirun -prepend-rank -n 5 -hosts 192. 168. 2. 100,192. 168. 2. 200 hello-world How do I add rankmap to the above? Just add
ZjQcmQRYFpfptBannerStart
This Message Is From an External Sender
This message came from outside your organization.

ZjQcmQRYFpfptBannerEnd

Thank you Joachim and Hui.





-rankmap {rank map} comma separated rank to node id list

How is the format if I have 4 processes ?

mpirun -prepend-rank -n 5 -hosts 192.168.2.100,192.168.2.200 hello-world



How do I add rankmap to the above? Just add ranks 0,1,2,3 after –rankmap?



Secondly, can I pin each of these 5 processes to a particular core?

Could you give an example of the “user” option?

user:0+2,1+4,3,2 -- user specified binding



does the above mean rank 0 will run on 2?





From: Zhou, Hui <zhouh at anl.gov>
Date: Monday, July 1, 2024 at 3:10 PM
To: discuss at mpich.org <discuss at mpich.org>, Jenke, Joachim <jenke at itc.rwth-aachen.de>
Cc: Niyaz Murshed <Niyaz.Murshed at arm.com>, nd <nd at arm.com>
Subject: Re: Custom rank for processes

>What Joachim points out is that you can set a rank of 2, 3, 20, 30 if your total number of processes is 4.

Of course, I meant for "can't".

Here are some additional pointers:
* The PR adding the rankmap option: https://urldefense.us/v3/__https://github.com/pmodels/mpich/pull/6213__;!!G_uCfscf7eWS!epGf_fQt4yhkaVMjUxBJbpLSKDAJuJJs1lOlbljAlnkm4Vxs4-c_elFKa5e1BI2K6DoMfQIxu9p-$ <https://urldefense.us/v3/__https://github.com/pmodels/mpich/pull/6213__;!!G_uCfscf7eWS!fsr35O2acR3dYOx9yUx9qiUQCxddjFDP9tll86AG3T_fquNAwewlHkbwYvmPzD1fZ8-mBpIp32aDV_JxND4$>
* Other options for specifying rank ordering and binding:https://urldefense.us/v3/__https://github.com/pmodels/mpich/blob/main/doc/wiki/how_to/Using_the_Hydra_Process_Manager.md__;!!G_uCfscf7eWS!epGf_fQt4yhkaVMjUxBJbpLSKDAJuJJs1lOlbljAlnkm4Vxs4-c_elFKa5e1BI2K6DoMfeERXddg$ <https://urldefense.us/v3/__https://github.com/pmodels/mpich/blob/main/doc/wiki/how_to/Using_the_Hydra_Process_Manager.md__;!!G_uCfscf7eWS!fsr35O2acR3dYOx9yUx9qiUQCxddjFDP9tll86AG3T_fquNAwewlHkbwYvmPzD1fZ8-mBpIp32aD-r3RY3I$>

Hui





________________________________

From: Zhou, Hui <zhouh at anl.gov>
Sent: Monday, July 1, 2024 3:04 PM
To: discuss at mpich.org <discuss at mpich.org>; Jenke, Joachim <jenke at itc.rwth-aachen.de>
Cc: Niyaz Murshed <Niyaz.Murshed at arm.com>; nd <nd at arm.com>
Subject: Re: Custom rank for processes



Hi Niyaz,



Yes, you can use the -rankmap​ option to specify a full rank ordering. The syntax for rankmap string is:

```
/* rankmap string format (used in PMI_process_mapping) in ABNF:

 *

 * mapping = '(' format ',' (block / block_repeats) *[',' (block / block_repeats)] ')'

 * format = 'vector'

 * block = num / '(' num ',' num ',' num ')'

 * block_repeats = '[' block *[',' block] ']x' num

 * num = 1*DIGIT

 *

 * If a block is in 3-tuple of (X,Y,Z), the meaning is:

 *     X - node id start value

 *     Y - number of nodes with size Z

 *     Z - number of processes assigned to each node

 *

 * If a block is a single number id, it denotes the node id, which is equivallen to

 * (id, 1, 1).

 *

 * A block_repeat repeats the bracketed block list by num of times.

 */
```

What Joachim points out is that you can set a rank of 2, 3, 20, 30 if your total number of processes is 4. You can only order the processes.

Hope that helps.



--

Hui

________________________________

From: Niyaz Murshed via discuss <discuss at mpich.org>
Sent: Monday, July 1, 2024 2:43 PM
To: discuss at mpich.org <discuss at mpich.org>; Jenke, Joachim <jenke at itc.rwth-aachen.de>
Cc: Niyaz Murshed <Niyaz.Murshed at arm.com>; nd <nd at arm.com>
Subject: Re: [mpich-discuss] Custom rank for processes



Am actually trying to understand how/if I can pin a rank to a process .. also pin a core to a process. From: Niyaz Murshed via discuss <discuss@ mpich. org> Date: Monday, July 1, 2024 at 2: 34 PM To: Jenke, Joachim <jenke@ itc. rwth-aachen. de>,

ZjQcmQRYFpfptBannerStart

This Message Is From an External Sender

This message came from outside your organization.



ZjQcmQRYFpfptBannerEnd

Am actually trying to understand how/if I can pin a rank to a process .. also pin a core to a process.



From: Niyaz Murshed via discuss <discuss at mpich.org>
Date: Monday, July 1, 2024 at 2:34 PM
To: Jenke, Joachim <jenke at itc.rwth-aachen.de>, discuss at mpich.org <discuss at mpich.org>
Cc: Niyaz Murshed <Niyaz.Murshed at arm.com>, nd <nd at arm.com>
Subject: Re: [mpich-discuss] Custom rank for processes

What is the rankmap used for ? From: Jenke, Joachim <jenke@ itc. rwth-aachen. de> Sent: Monday, July 1, 2024 2: 21: 53 PM To: discuss@ mpich. org <discuss@ mpich. org> Cc: Niyaz Murshed <Niyaz. Murshed@ arm. com>; nd <nd@ arm. com>

ZjQcmQRYFpfptBannerStart

This Message Is From an External Sender

This message came from outside your organization.



ZjQcmQRYFpfptBannerEnd

What is the rankmap used for ?

________________________________

From: Jenke, Joachim <jenke at itc.rwth-aachen.de>
Sent: Monday, July 1, 2024 2:21:53 PM
To: discuss at mpich.org <discuss at mpich.org>
Cc: Niyaz Murshed <Niyaz.Murshed at arm.com>; nd <nd at arm.com>
Subject: Re: Custom rank for processes



MPI defines rank numbers to be consecutive from 0 to size-1. Your suggested numbering will not work.

-Joachim

________________________________

From: Niyaz Murshed via discuss <discuss at mpich.org>
Sent: Monday, July 1, 2024 9:09:41 PM
To: discuss at mpich.org <discuss at mpich.org>
Cc: Niyaz Murshed <Niyaz.Murshed at arm.com>; nd <nd at arm.com>
Subject: [mpich-discuss] Custom rank for processes



Hello, Is it possible to provide custom ranks to processes ? I see a param called “-rankmap” .. Does it provide this option? Is it possible to do something like below : Node1 => 2 processes => rank 2,3 Node2 => 2 processes => rank

ZjQcmQRYFpfptBannerStart

This Message Is From an External Sender

This message came from outside your organization.



ZjQcmQRYFpfptBannerEnd

Hello,

Is it possible to provide custom ranks to processes ?

I see a param called “-rankmap” .. Does it provide this option?



Is it possible to do something like below :

Node1 => 2 processes => rank 2,3

Node2 => 2 processes => rank 20,30



Thanks.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mpich.org/pipermail/discuss/attachments/20240701/d301d387/attachment-0001.html>

More information about the discuss mailing list