[mpich-discuss] How locking on multi-VCI works

[Zhou, Hui]

The total number of VCIs are configured with --with-ch4-max-vcis=#​. The maximum is 64. The default used to be 1, but it is changed to 64 in 4.0b1 release. There is also an option to control the vci assignment method: --enable-ch4-vci-method={communicator,tag,implicit}​. The default is communicator​, with which we assign vci to communicators in a round-robin fashion. If you create communicators consecutively, they are expected to have different VCIs. The other vci-method​s are at the experimental stage. If the communicator method is insufficient for your application, it may be worth a try.  We'd like to understand your use case better before pointing you that way.

I am not exactly understanding your question. The vci locks are local process locks, so if you have N VCIs, you will have N channels for each process. With vci-method=communicator​, the vcis are one-to-one matched, i.e. rank 1 vci 1 only communicates to rank 2 (any ranks with the same communicator) vci 1.

--
Hui Zhou
________________________________
From: Guilherme Valarini via discuss <discuss at mpich.org>
Sent: Monday, December 6, 2021 10:10 AM
To: discuss at mpich.org <discuss at mpich.org>
Cc: Guilherme Valarini <guilherme.a.valarini at gmail.com>
Subject: [mpich-discuss] How locking on multi-VCI works

Hello everyone,

I got one question regarding the multi-VCI support and possible locking contentions of MPICH on multi-threaded environments.

I understand that there is a direct mapping between a VCI and a communicator, so global locking is avoided on a multi-threaded application. But I wanted to know: how do these VCIs work? When I have N VCIs, do I have N virtual channels per rank (thus, one global lock per VCI-rank pair) or only 2 channels at all (one lock per VCI)? I was wondering if, for example, two MPI_Sends targeting different ranks on the same comm might need to be synchronized using such a global lock.

Thanks for the help!
Guilherme Valarini
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mpich.org/pipermail/discuss/attachments/20211206/182c0865/attachment.html>