[mpich-discuss] MPICH Test Suite - limit number of processes

[Max R. Dechantsreiter]

Hi Joachim,

Thank you, I will try that overnight.  This setting might
be useful in practical situations.

Interesting that some of the tests require 33 processes:
that is high even for most modern processors, so I presume
the idea was to run the suite on multiple nodes.

Best regards,

Max
---


On Sun, Jan 15, 2023 at 09:19:58PM +0000, Jenke, Joachim wrote:
> For our unit testing (not mpich specific) I found it useful to set the following environmental variable to reduce the time wasted in busy waiting while we run our tests massively oversubscribed:
>     export MPIR_CVAR_POLLS_BEFORE_YIELD=1
> 
> - Joachim
> ________________________________
> From: Max R. Dechantsreiter via discuss <discuss at mpich.org>
> Sent: Sunday, January 15, 2023 3:09:19 PM
> To: Zhou, Hui <zhouh at anl.gov>
> Cc: Max R. Dechantsreiter <max at performancejones.com>; discuss at mpich.org <discuss at mpich.org>
> Subject: Re: [mpich-discuss] MPICH Test Suite - limit number of processes
> 
> Thank you for your suggestion.
> 
> However, I did not see how to carry it through; perhaps one
> day I will comb through the User's Guide.
> 
> Instead I ran 'make testing' to completion on four cores:
> elapsed time was 2h36m53s on Intel(R) Xeon(R) CPU E5-2620 v4
> (2.10GHz); all but two tests passed, with the two that did
> not pass simply timing out (after 180s).
> 
> Then I ran it again after modifying "runtests" thus:
> 
> sed -i 's/^$ppnMax  = -1;/$ppnMax  = 4;/' $RUNTESTS
> sed -i 's/^$np_max     = -1;/$np_max     = 4;/' $RUNTESTS
> 
> where RUNTESTS=~test/mpi/runtests.  (Alternatively these
> variables could be modified in $RUNTESTS.in before prior to
> building MPICH.)
> 
> Changing those variables limited the maximum number of
> processes to 4 as desired.  The salient issues were the
> warnings 'Number of processors must divide 10' that I
> presume could be avoided by setting $np_max=2 or 5; the
> number of tests affected was relatively small: 13 of 2524
> tests failed, with no test timing out.  Elapsed time was
> 1h04m28s - so the savings in time was significant but not
> spectacular, suggesting that relatively few tests in the
> original run oversubscribed the cores available in my VPS.
> 
> 
> On Thu, Jan 12, 2023 at 04:29:07PM +0000, Zhou, Hui wrote:
> > Unfortunately, the number of processes for each test is hardcoded in the testlist files. There is no mechanism for filtering or dynamically modifying them. You could either maintain a custom patch to modify the testlist, or have a script applying a separate xfail list. An example of such xfail list that we used in our internal testing is in test/mpi/maint/jenkins/xfail.conf. If the custom patch solution won't work in your situation, please open an github issue describing your use case, and we can discuss to add a filtering option to the runtests script.
> >
> > --
> > Hui
> > ________________________________
> > From: Max R. Dechantsreiter via discuss <discuss at mpich.org>
> > Sent: Thursday, January 12, 2023 9:58 AM
> > To: discuss at mpich.org <discuss at mpich.org>
> > Cc: Max R. Dechantsreiter <max at performancejones.com>
> > Subject: [mpich-discuss] MPICH Test Suite - limit number of processes
> >
> > I would like to limit the number of processes used by the
> > MPICH Test Suite, which I run by "make testing" after
> > building MPICH.
> >
> > I tried
> >
> > export MPITEST_PPNMAX=<n> # The maximum number of processes per node.
> >
> > but this had no apparent effect: some tests still ran
> > using more than 4 processes, when I replaced <n> by 4.
> > How could I restrict these tests to run with maximum 4
> > processes, for example?
> >
> > _______________________________________________
> > discuss mailing list     discuss at mpich.org
> > To manage subscription options or unsubscribe:
> > https://lists.mpich.org/mailman/listinfo/discuss
> _______________________________________________
> discuss mailing list     discuss at mpich.org
> To manage subscription options or unsubscribe:
> https://lists.mpich.org/mailman/listinfo/discuss