[mpich-commits] [mpich] MPICH primary repository branch, master, updated. v3.2rc1-9-gc9d0fb4
Service Account
noreply at mpich.org
Fri Oct 16 19:41:17 CDT 2015
This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "MPICH primary repository".
The branch, master has been updated
via c9d0fb481b235dc6e2a5f3974e12f233b03384ed (commit)
via 67e1318ea71b4136a9e3b0eb77d84c7fdde3a7fa (commit)
via e1ddafb8b2b7301b84e77c11918faf89a5b583d4 (commit)
via 729dd94332ba2e360bfba75ab9db2a56aae19e76 (commit)
via 7f1e00c834d6bf85195b587d64855eaca1a48a5e (commit)
via ba0fdd3595b09f0033c849eb7d0a0167f6cf9636 (commit)
via ecbe04f11e73de7505dac9b46db8849431bb5e31 (commit)
from ab86ff374c430a00ee9065e8b1b95df27087f204 (commit)
Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.
- Log -----------------------------------------------------------------
http://git.mpich.org/mpich.git/commitdiff/c9d0fb481b235dc6e2a5f3974e12f233b03384ed
commit c9d0fb481b235dc6e2a5f3974e12f233b03384ed
Author: Yanfei Guo <yguo at anl.gov>
Date: Fri Oct 16 11:06:54 2015 -0500
maint/jenkins: add timeout script for multinode jobs
We use this script to calculate the timeout for SLURM jobs based on the
configuration, netmod, etc. This help improve the backfilling on SLURM.
No reviewer.
diff --git a/maint/jenkins/multinode/timeout.conf b/maint/jenkins/multinode/timeout.conf
new file mode 100644
index 0000000..b841c05
--- /dev/null
+++ b/maint/jenkins/multinode/timeout.conf
@@ -0,0 +1,23 @@
+# Contitional Timeout settings
+#
+# Syntax (similar to a cron file):
+# [job_name] [compiler] [jenkins_configure] [netmod] [n_nodes] [ppn] [timeout in min]
+# Other conditions only allows exact matches.
+#
+# Examples:
+# mxm gnu debug * * sed -i "..."
+# This will apply the set the XFAIL when the job is "mpich-master-mxm" or
+# "mpich-review-mxm", the compiler is "gnu", and the jenkins_configure is
+# "debug".
+#
+# master-ubuntu * * * ubuntu32 sed -i "..."
+# This will apply the set the XFAIL when the job is "mpich-master-ubuntu" and
+# the running queue is "ubuntu32".
+#
+mpich-master-multinode * * mxm 2 * 60
+mpich-master-multinode * * mxm 4 * 60
+mpich-master-multinode * * mxm 8 * 70
+mpich-master-multinode * * tcp * * 120
+mpich-master-multinode * * ofi * * 150
+mpich-master-multinode * * portals4 * * 90
+* * * * * * 120
diff --git a/maint/jenkins/set-timeout.sh b/maint/jenkins/set-timeout.sh
new file mode 100755
index 0000000..1554632
--- /dev/null
+++ b/maint/jenkins/set-timeout.sh
@@ -0,0 +1,102 @@
+#!/bin/zsh
+
+jobname=""
+compiler=""
+jenkins_configure=""
+n_nodes="1"
+ppn="1"
+netmod=""
+
+TIMEOUT_CONF="maint/jenkins/multinode/timeout.conf"
+
+#####################################################################
+## Initialization
+#####################################################################
+
+while getopts ":f:j:c:o:n:p:m:" opt; do
+ case "$opt" in
+ j)
+ jobname=$OPTARG ;;
+ c)
+ compiler=$OPTARG ;;
+ o)
+ jenkins_configure=$OPTARG ;;
+ n)
+ n_nodes=$OPTARG ;;
+ p)
+ ppn=$OPTARG ;;
+ m)
+ netmod=$OPTARG ;;
+ f)
+ TIMEOUT_CONF=$OPTARG ;;
+ \?)
+ echo "Invalid option: -$OPTARG" >&2
+ exit 1
+ esac
+done
+
+#####################################################################
+## Main (
+#####################################################################
+
+if test ! -f "$TIMEOUT_CONF" ; then
+ echo "Cannot find $TIMEOUT_CONF. No TIMEOUT will be calculated"
+ exit 0
+fi
+
+TimeoutCond() {
+ local job="$1"
+ local comp="$2"
+ local option="$3"
+ local nmod="$4"
+ local _n_nodes="$5"
+ local _ppn="$6"
+
+ local state=0
+
+ if [[ ! "$job" == "*" ]]; then
+ # clean up jobname and do substring match
+ if [[ ! "${jobname%%,*}" == *$job* ]]; then state=1; fi
+ fi
+
+ if [[ ! "$comp" == "*" ]]; then
+ if [[ ! "$compiler" == "$comp" ]]; then state=1; fi
+ fi
+
+ if [[ ! "$option" == "*" ]]; then
+ if [[ ! "$jenkins_configure" == "$option" ]]; then state=1; fi
+ fi
+
+ if [[ ! "$nmod" == "*" ]]; then
+ if [[ ! "$netmod" == "$nmod" ]]; then state=1; fi
+ fi
+
+ if [[ ! "$_n_nodes" == "*" ]]; then
+ if [[ ! "$n_nodes" == "$_n_nodes" ]]; then state=1; fi
+ fi
+
+ if [[ ! "$_ppn" == "*" ]]; then
+ if [[ ! "$ppn" == "$_ppn" ]]; then state=1; fi
+ fi
+
+ echo "$state"
+}
+
+while read -r line; do
+ #clean leading whitespaces
+ line=$(echo "$line" | sed "s/^ *//g")
+ line=$(echo "$line" | sed "s/ *$//g")
+ # echo $line
+ # skip comment line
+ if test -x "$line" -o "${line:1}" = "#" ; then
+ continue
+ fi
+
+ arr=( $(echo $line) )
+ if [[ "0" == $(TimeoutCond "${arr[1]}" "${arr[2]}" "${arr[3]}" "${arr[4]}" "${arr[5]}" "${arr[6]}") ]]; then
+ echo "${arr[@]:6}"
+ exit 0
+ fi
+done < "$TIMEOUT_CONF"
+
+exit 0
http://git.mpich.org/mpich.git/commitdiff/67e1318ea71b4136a9e3b0eb77d84c7fdde3a7fa
commit 67e1318ea71b4136a9e3b0eb77d84c7fdde3a7fa
Author: Yanfei Guo <yguo at anl.gov>
Date: Fri Oct 16 11:00:58 2015 -0500
maint/jenkins: xfail test failing on Portals 4 bug
The Portals 4 reference implementation has a bug that makes recv cancel
operations unreliable in threaded cases. We modified to the threaded
alltoall test to consistently reproduce the failure. We have xfailed it
when running with Portals 4 to not dirty the Jenkins results. Now we
apply the same xfail for any job that uses portals4 netmod.
No reviewer.
diff --git a/maint/jenkins/xfail.conf b/maint/jenkins/xfail.conf
index 119f8ae..934685b 100644
--- a/maint/jenkins/xfail.conf
+++ b/maint/jenkins/xfail.conf
@@ -23,4 +23,4 @@
# For each build, all applied XFAILS will be summaried in
# ${SOURCE}/apply-xfail.sh
#
-portals4 * * * * sed -i "s+\(^alltoall .*\)+\1 xfail=ticket0+g" test/mpi/threads/pt2pt/testlist
+* * * portals4 * sed -i "s+\(^alltoall .*\)+\1 xfail=ticket0+g" test/mpi/threads/pt2pt/testlist
http://git.mpich.org/mpich.git/commitdiff/e1ddafb8b2b7301b84e77c11918faf89a5b583d4
commit e1ddafb8b2b7301b84e77c11918faf89a5b583d4
Author: Yanfei Guo <yguo at anl.gov>
Date: Wed Oct 14 14:31:06 2015 -0500
maint/jenkins: build mpich-tarball on /sandbox
Creating mpich tarball on local disk instead of NFS.
No reviewer.
diff --git a/maint/jenkins/test-worker-tarball.sh b/maint/jenkins/test-worker-tarball.sh
index 300631b..13fa912 100755
--- a/maint/jenkins/test-worker-tarball.sh
+++ b/maint/jenkins/test-worker-tarball.sh
@@ -9,11 +9,25 @@ export CXX=g++
export FC=gfortran
cd $WORKSPACE
-git clean -x -d -f
+TMP_WORKSPACE=$(mktemp -d /sandbox/jenkins.tmp.XXXXXXXX)
mkdir build
-cd build
+
+cp -a $WORKSPACE/* $TMP_WORKSPACE/
+pushd $TMP_WORKSPACE
+
+git clean -x -d -f
+
+pushd build
../maint/release.pl --branch=master --version=master --git-repo=git://git.mpich.org/mpich.git
+popd
+
+cp -a $TMP_WORKSPACE/build/mpich-master.tar.gz $WORKSPACE/build
+
+popd
+
+rm -rf $TMP_WORKSPACE
+
exit 0
http://git.mpich.org/mpich.git/commitdiff/729dd94332ba2e360bfba75ab9db2a56aae19e76
commit 729dd94332ba2e360bfba75ab9db2a56aae19e76
Author: Yanfei Guo <yguo at anl.gov>
Date: Tue Oct 6 11:03:27 2015 -0400
maint/jenkins: remove MXM_SHM_KCOPY_MODE=off for testing
This option is no longer needed as KMEM in MXM is already working on all
nodes.
No reviewer.
diff --git a/maint/jenkins/test-worker.sh b/maint/jenkins/test-worker.sh
index fdec609..8309576 100755
--- a/maint/jenkins/test-worker.sh
+++ b/maint/jenkins/test-worker.sh
@@ -503,8 +503,6 @@ case "$netmod" in
"mxm")
MXM_LOG_LEVEL=error
export MXM_LOG_LEVEL
- MXM_SHM_KCOPY_MODE=off
- export MXM_SHM_KCOPY_MODE
;;
"ofi" | "portals4")
MXM_LOG_LEVEL=error
http://git.mpich.org/mpich.git/commitdiff/7f1e00c834d6bf85195b587d64855eaca1a48a5e
commit 7f1e00c834d6bf85195b587d64855eaca1a48a5e
Author: Yanfei Guo <yguo at anl.gov>
Date: Tue Oct 6 10:23:06 2015 -0400
maint/jenkins: remove killing for hydra_pmi_proxy
The killing for remaining hydra_pmi_proxy is no longer needed as the bug
has been fixed.
No reviewer.
diff --git a/maint/jenkins/test-worker.sh b/maint/jenkins/test-worker.sh
index fdf3e9a..fdec609 100755
--- a/maint/jenkins/test-worker.sh
+++ b/maint/jenkins/test-worker.sh
@@ -530,10 +530,6 @@ case "$jenkins_configure" in
;;
esac
-if killall -9 hydra_pmi_proxy; then
- echo "leftover hydra_pmi_proxy processes killed"
-fi
-
#####################################################################
## Copy Test results and Cleanup
#####################################################################
http://git.mpich.org/mpich.git/commitdiff/ba0fdd3595b09f0033c849eb7d0a0167f6cf9636
commit ba0fdd3595b09f0033c849eb7d0a0167f6cf9636
Author: Yanfei Guo <yguo at anl.gov>
Date: Tue Sep 29 11:17:19 2015 -0500
maint/jenkins: add multi-node test scripts
Two scripts are added for multi-node tests. They are designed to be
submitted as two consecutive jobs.
mn-build.sh builds the MPI library and test binaries on local disk. The
temporary workspace on local disk is packed as a tarball and saved in
the Jenkins workspace. This script should be submitted as a 1-node SLURM
job using `srun`.
mn-run.sh recreates the temporary workspace and extracts the files from
the tarball in the previous step. Then it runs all the tests and
collects the results. This script should be submitted as a 2-node SLURM
job using `srun`.
Notes:
1. These two scripts must use the same temporary workspace location. The
location of temporary workspace is specified using "-t" option.
2. Each of these two scripts should only be ran one time. When submitted
through `srun`, SLURM will try to run these scripts on every allocated
nodes, which is incorrect. These scripts checks the SLURM_NODEID
variable and exit if it is not running on the first allocated node.
3. In order to specify interface for tcp jobs. We hack the
test/mpi/runtests script to include "-iface ib0" with mpiexec.
No reviewer.
diff --git a/maint/jenkins/multinode/mn-build.sh b/maint/jenkins/multinode/mn-build.sh
new file mode 100755
index 0000000..01cb213
--- /dev/null
+++ b/maint/jenkins/multinode/mn-build.sh
@@ -0,0 +1,506 @@
+#!/bin/zsh -xe
+
+# This script only run on the first node. The mpiexec will automatically spawned
+if test ! $SLURM_NODEID -eq 0; then
+ exit 0
+fi
+
+mount | grep autotest
+
+hostname
+
+WORKSPACE=""
+TMP_WORKSPACE=""
+compiler="gnu"
+jenkins_configure="default"
+queue="ib64"
+netmod="default"
+ofi_prov="sockets"
+N_MAKE_JOBS=8
+GIT_BRANCH=""
+BUILD_MODE="per-commit"
+
+#####################################################################
+## Initialization
+#####################################################################
+
+while getopts ":h:c:o:q:m:n:b:t:" opt; do
+ case "$opt" in
+ h)
+ WORKSPACE=$OPTARG ;;
+ c)
+ compiler=$OPTARG ;;
+ o)
+ jenkins_configure=$OPTARG ;;
+ q)
+ queue=$OPTARG ;;
+ m)
+ _netmod=${OPTARG%%,*}
+ if [[ "$_netmod" == "ofi" ]]; then
+ netmod=$_netmod
+ ofi_prov=${OPTARG/$_netmod,}
+ else
+ netmod=$_netmod
+ fi
+ ;;
+ n)
+ N_MAKE_JOBS=$OPTARG ;;
+ b)
+ GIT_BRANCH=$OPTARG ;;
+ t)
+ TMP_WORKSPACE=$OPTARG ;;
+ \?)
+ echo "Invalid option: -$OPTARG" >&2
+ exit 1
+ esac
+done
+
+cd $WORKSPACE
+
+if test "$GIT_BRANCH" = "" ; then
+ BUILD_MODE="nightly"
+fi
+
+case "$BUILD_MODE" in
+ "nightly")
+ if [[ -x mpich-master/maint/jenkins/skip_test.sh ]]; then
+ ./mpich-master/maint/jenkins/skip_test.sh -j $JOB_NAME -c $compiler -o $jenkins_configure -q $queue -m $_netmod \
+ -s mpich-master/test/mpi/summary.junit.xml
+ if [[ -f mpich-master/test/mpi/summary.junit.xml ]]; then
+ exit 0
+ fi
+ fi
+ ;;
+ "per-commit")
+ if [[ -x maint/jenkins/skip_test.sh ]]; then
+ ./maint/jenkins/skip_test.sh -j $JOB_NAME -c $compiler -o $jenkins_configure -q $queue -m $_netmod \
+ -s test/mpi/summary.junit.xml
+ if [[ -f test/mpi/summary.junit.xml ]]; then
+ exit 0
+ fi
+ fi
+ ;;
+esac
+
+
+if test -d "$TMP_WORKSPACE"; then
+ rm -rf "$TMP_WORKSPACE"
+fi
+mkdir -p "$TMP_WORKSPACE"
+SRC=$WORKSPACE
+TMP_SRC=$TMP_WORKSPACE
+
+# Preparing the source
+case "$BUILD_MODE" in
+ "nightly")
+ SRC=$WORKSPACE/mpich-master
+ TMP_SRC=$TMP_WORKSPACE/mpich-master
+ cp $WORKSPACE/mpich-master.tar.gz $TMP_WORKSPACE/
+ pushd "$TMP_WORKSPACE"
+ tar zxvf mpich-master.tar.gz
+ popd
+ ;;
+ "per-commit")
+ git clean -x -d -f
+ cp -a $WORKSPACE/* $TMP_WORKSPACE/
+ ;;
+ *)
+ echo "Invalid BUILD_MODE $BUILD_MODE. Set by mistake?"
+ exit 1
+esac
+
+
+#####################################################################
+## Functions
+#####################################################################
+
+CollectResults() {
+ # TODO: copy saved test binaries (for failed cases)
+ if [[ "$BUILD_MODE" != "per-commit" ]]; then
+ find . \
+ \( -name "filtered-make.txt" \
+ -o -name "apply-xfail.sh" \
+ -o -name "autogen.log" \
+ -o -name "config.log" \
+ -o -name "c.txt" \
+ -o -name "m.txt" \
+ -o -name "mi.txt" \
+ -o -name "summary.junit.xml" \) \
+ | while read -r line; do
+ mkdir -p "$SRC/$(dirname $line)"
+ done
+ fi
+
+ find . \
+ \( -name "filtered-make.txt" -o \
+ -name "apply-xfail.sh" -o \
+ -name "autogen.log" -o \
+ -name "config.log" -o \
+ -name "c.txt" -o \
+ -name "m.txt" -o \
+ -name "mi.txt" -o \
+ -name "summary.junit.xml" \) \
+ -exec cp {} $SRC/{} \;
+}
+
+#####################################################################
+## Logic to generate random configure options
+#####################################################################
+RandArgs() {
+ # Chosen *without* replacement. If an option is chosen twice,
+ # then there will be fewer options
+ n_choice=$1
+ array=(${(P)${2}})
+ optname=$3
+ negoptname=$4
+ chosen=()
+ args=""
+ ret_args=""
+ array_len=$#array
+ idx=0
+
+ for i in `seq $array_len`; do
+ chosen[$i]=0
+ done
+
+ for i in `seq $n_choice`; do
+ let idx=$[RANDOM % $array_len]+1
+ if [ $chosen[$idx] -eq 1 ]; then continue; fi
+ chosen[$idx]=1
+ args=("${(s/;/)array[$idx]}")
+ name=$args[1]
+ if [ $#args -eq 1 ]; then
+ # Only the name is provided. Choose one of three
+ # choices:
+ # No option (skip this one)
+ # just --$optname-$name
+ # just --$negoptname-$name
+ let idx=$[RANDOM % 3]+1
+ if [ $idx -eq 1 ]; then
+ ret_args="$ret_args --$optname-$name"
+ elif [ $idx -eq 2 ]; then
+ ret_args="$ret_args --$negoptname-$name"
+ fi
+ else
+ let idx=$[RANDOM % ($#args-1)]+2
+ # Special cases
+ if [ "$args[$idx]" = "ch3:sock" ]; then
+ ret_args="$ret_args --disable-ft-tests --disable-comm-overlap-tests"
+ elif [ "$args[$idx]" = "gforker" ]; then
+ if [ $chosen[4] -eq 1 ]; then
+ continue
+ else
+ ret_args="$ret_args --with-namepublisher=file"
+ chosen[4]=1
+ fi
+ elif [ "$name" = "namepublisher" -a "$args[$idx]" = "no" ]; then
+ if [ $chosen[3] -eq 1 ]; then
+ continue
+ fi
+ elif [ "$args[$idx]" = "ndebug" -a "$CC" = "suncc" -a "$label" = "ubuntu32" ]; then
+ # On ubuntu32, suncc has a bug whose workaround is to add -O flag (ticket #2105)
+ CFLAGS="-O1"
+ export CFLAGS
+ fi
+ ret_args="$ret_args --$optname-$name=$args[$idx]"
+ fi
+ done
+ echo $ret_args
+}
+
+RandConfig() {
+ # WARNING: If moving anything in the two following arrays, check the indices in "Special cases" above
+ enable_array=(
+ 'error-checking;no;runtime;all'
+ 'error-messages;all;generic;class;none'
+ 'timer-type;linux86_cycle;clock_gettime;gettimeofday'
+ 'timing;none;all;runtime;log;log_detailed'
+ 'g;none;all;handle;dbg;log;meminit;handlealloc;instr;mem;mutex;mutexnesting'
+ 'fast;O0;O1;O2;O3;ndebug;all;yes;none'
+ 'fortran'
+ 'cxx'
+ 'romio'
+ 'check-compiler-flags'
+ 'strict;c99;posix'
+ 'debuginfo'
+ 'weak-symbols;no;yes'
+ 'threads;single;multiple;runtime'
+ 'thread-cs;global'
+ 'refcount;lock-free;none'
+ 'mutex-timing'
+ 'handle-allocation;tls;mutex'
+ 'multi-aliases'
+ 'predefined-refcount'
+ 'alloca'
+ 'yield;sched_yield;select'
+ 'runtimevalues'
+ )
+ with_array=(
+ 'logging;none'
+ 'pmi;simple'
+ 'pm;gforker'
+ 'namepublisher;no;file'
+ 'device;ch3;ch3:sock'
+ )
+ let n_enable=$#enable_array+1
+ let n_with=$#with_array+1
+ enable_args=$(RandArgs $n_enable "enable_array" "enable" "disable")
+ with_args=$(RandArgs $n_with "with_array" "with" "without")
+ echo "$enable_args $with_args"
+}
+
+PrepareEnv() {
+ case "$queue" in
+ "ubuntu32" | "ubuntu64" )
+ source /software/common/adm/etc/softenv-aliases.sh
+ source /software/common/adm/etc/softenv-load.sh
+ soft add +intel
+ soft add +pgi
+ soft add +absoft
+ soft add +nagfor
+ soft add +solarisstudio-12.4
+ soft add +ekopath
+ ;;
+ "freebsd64" | "freebsd32")
+ PATH=/usr/local/bin:$PATH
+ LD_LIBRARY_PATH=/usr/local/lib/gcc46:$LD_LIBRARY_PATH
+ export LD_LIBRARY_PATH
+ ;;
+ "solaris")
+ PATH=/usr/gcc/4.3/bin:/usr/gnu/bin:/usr/sbin:$PATH
+ CONFIG_SHELL=/usr/gnu/bin/sh
+ export CONFIG_SHELL
+ ;;
+ esac
+ PATH=$HOME/software/autotools/bin:$PATH
+ export PATH
+ echo "$PATH"
+}
+
+SetCompiler() {
+ case "$compiler" in
+ "gnu")
+ CC=gcc
+ CXX=g++
+ F77=gfortran
+ FC=gfortran
+ ;;
+ "clang")
+ CC=clang
+ CXX=clang++
+ F77=gfortran
+ FC=gfortran
+ ;;
+ "intel")
+ CC=icc
+ CXX=icpc
+ F77=ifort
+ FC=ifort
+ ;;
+ "pgi")
+ CC=pgcc
+ CXX=pgcpp
+ F77=pgf77
+ FC=pgfortran
+ ;;
+ "absoft")
+ CC=gcc
+ CXX=g++
+ F77=af77
+ FC=af90
+ ;;
+ "nag")
+ CC=gcc
+ CXX=g++
+ F77=nagfor
+ FC=nagfor
+ FFLAGS="-mismatch"
+ FCFLAGS="-mismatch"
+ export FFLAGS
+ export FCFLAGS
+ ;;
+ "solstudio")
+ CC=suncc
+ CXX=sunCC
+ F77=sunf77
+ FC=sunf90
+ ;;
+ "sunstudio")
+ CC=cc
+ CXX=CC
+ F77=f77
+ FC=f90
+ ;;
+ "pathscale")
+ CC=pathcc
+ CXX=pathCC
+ F77=pathf95
+ FC=pathf95
+ ;;
+ *)
+ echo "Unknown compiler suite"
+ exit 1
+ esac
+
+ export CC
+ export CXX
+ export F77
+ export FC
+
+ which $CC
+ which $CXX
+ which $F77
+ which $FC
+}
+
+SetNetmod() {
+ netmod_opt="__NULL__"
+ case "$netmod" in
+ "default") # for solaris, may use with sock
+ netmod_opt=
+ ;;
+ "mxm")
+ netmod_opt="--with-device=ch3:nemesis:mxm --with-mxm=$HOME/software/mellanox/mxm --disable-spawn --disable-ft-tests"
+ ;;
+ "ofi")
+ netmod_opt="--with-device=ch3:nemesis:ofi --with-ofi=$HOME/software/libfabric/$ofi_prov --disable-spawn --disable-ft-tests LD_LIBRARY_PATH=$HOME/software/libfabric/lib"
+ ;;
+ "portals4")
+ netmod_opt="--with-device=ch3:nemesis:portals4 --with-portals4=$HOME/software/portals4 --disable-spawn --disable-ft-tests"
+ ;;
+ "sock")
+ netmod_opt="--with-device=ch3:sock --disable-ft-tests --disable-comm-overlap-tests"
+ ;;
+ "tcp")
+ netmod_opt=
+ ;;
+ *)
+ echo "Unknown netmod type"
+ exit 1
+ esac
+ export netmod_opt
+ echo "$netmod_opt"
+}
+
+SetConfigOpt() {
+ config_opt="__TO_BE_FILLED__"
+ case "$jenkins_configure" in
+ "default")
+ config_opt=
+ ;;
+ "strict")
+ config_opt="--enable-strict"
+ ;;
+ "fast")
+ config_opt="--enable-fast=all"
+ ;;
+ "nofast")
+ config_opt="--disable-fast"
+ ;;
+ "noshared")
+ config_opt="--disable-shared"
+ ;;
+ "debug")
+ config_opt="--enable-g=all"
+ ;;
+ "noweak")
+ config_opt="--disable-weak-symbols"
+ ;;
+ "strictnoweak")
+ config_opt="--enable-strict --disable-weak-symbols"
+ ;;
+ "nofortran")
+ config_opt="--disable-fortran"
+ ;;
+ "nocxx")
+ config_opt="--disable-cxx"
+ ;;
+ "multithread")
+ config_opt="--enable-threads=multiple"
+ ;;
+ "debuginfo")
+ config_opt="--enable-debuginfo"
+ ;;
+ "noerrorchecking")
+ config_opt="--disable-error-checking"
+ ;;
+ "sock") # for solaris + sock
+ config_opt="--with-device=ch3:sock --disable-ft-tests --disable-comm-overlap-tests"
+ ;;
+ "mpd")
+ config_opt="--with-pm=mpd --with-namepublisher=file"
+ ;;
+ "gforker")
+ config_opt="--with-pm=gforker --with-namepublisher=file"
+ ;;
+ "shmem")
+ config_opt=
+ ;;
+ "async")
+ config_opt=
+ ;;
+ "random")
+ config_opt=$(RandArgs)
+ ;;
+ *)
+ echo "Bad configure option: $jenkins_configure"
+ exit 1
+ esac
+
+ if test "$queue" = "osx" -a "$FC" = "ifort"; then
+ config_opt="$config_opt lv_cv_ld_force_load=no"
+ fi
+
+ # ROMIO is always disabled on multinode tests
+ config_opt="$config_opt --disable-romio"
+
+ export config_opt
+ echo "$config_opt"
+}
+
+#####################################################################
+## Main() { Setup Environment and Build
+#####################################################################
+# determine if this is a nightly job or a per-commit job
+PrepareEnv
+
+SetCompiler "$compiler"
+
+pushd "$TMP_SRC"
+
+if test "$BUILD_MODE" = "per-commit" ; then
+ ./autogen.sh 2>&1 | tee autogen.log
+fi
+
+if [[ -x maint/jenkins/set-xfail.sh ]]; then
+ ./maint/jenkins/set-xfail.sh -j $JOB_NAME -c $compiler -o $jenkins_configure -q $queue -m $_netmod
+fi
+
+./configure --prefix="$TMP_SRC/_inst" $(SetNetmod $netmod) $(SetConfigOpt $jenkins_configure) \
+ --disable-perftest \
+ 2>&1 | tee c.txt
+make -j$N_MAKE_JOBS 2>&1 | tee m.txt
+if test "${pipestatus[-2]}" != "0"; then
+ CollectResults
+ exit 1
+fi
+make -j$N_MAKE_JOBS install 2>&1 | tee mi.txt
+if test "${pipestatus[-2]}" != "0"; then
+ CollectResults
+ exit 1
+fi
+cat m.txt mi.txt | ./maint/clmake > filtered-make.txt 2>&1
+
+# Make the binaries of tests before rsync back to NFS
+pushd test/mpi
+make -j$N_MAKE_JOBS
+popd
+
+tar cf $TMP_WORKSPACE/mpich-test-pack.tar *
+cp $TMP_WORKSPACE/mpich-test-pack.tar $WORKSPACE
+rm $TMP_WORKSPACE/mpich-test-pack.tar
+popd
+
+rm -rf $TMP_WORKSPACE
+exit 0
+
diff --git a/maint/jenkins/multinode/mn-run.sh b/maint/jenkins/multinode/mn-run.sh
new file mode 100755
index 0000000..3b6a12b
--- /dev/null
+++ b/maint/jenkins/multinode/mn-run.sh
@@ -0,0 +1,240 @@
+#!/bin/zsh -xe
+
+# This script only run on the first node. The mpiexec will automatically spawned
+if test ! $SLURM_NODEID -eq 0; then
+ exit 0
+fi
+
+hostname
+
+WORKSPACE=""
+TMP_WORKSPACE=""
+compiler="gnu"
+jenkins_configure="default"
+queue="ib64"
+netmod="default"
+ofi_prov="sockets"
+N_MAKE_JOBS=8
+GIT_BRANCH=""
+BUILD_MODE="per-commit"
+
+#####################################################################
+## Initialization
+#####################################################################
+
+while getopts ":h:c:o:q:m:n:b:t:" opt; do
+ case "$opt" in
+ h)
+ WORKSPACE=$OPTARG ;;
+ c)
+ compiler=$OPTARG ;;
+ o)
+ jenkins_configure=$OPTARG ;;
+ q)
+ queue=$OPTARG ;;
+ m)
+ _netmod=${OPTARG%%,*}
+ if [[ "$_netmod" == "ofi" ]]; then
+ netmod=$_netmod
+ ofi_prov=${OPTARG/$_netmod,}
+ else
+ netmod=$_netmod
+ fi
+ ;;
+ n)
+ N_MAKE_JOBS=$OPTARG ;;
+ b)
+ GIT_BRANCH=$OPTARG ;;
+ t)
+ TMP_WORKSPACE=$OPTARG ;;
+ \?)
+ echo "Invalid option: -$OPTARG" >&2
+ exit 1
+ esac
+done
+
+cd $WORKSPACE
+
+if test "$GIT_BRANCH" = "" ; then
+ BUILD_MODE="nightly"
+fi
+
+case "$BUILD_MODE" in
+ "nightly")
+ if [[ -x mpich-master/maint/jenkins/skip_test.sh ]]; then
+ ./mpich-master/maint/jenkins/skip_test.sh -j $JOB_NAME -c $compiler -o $jenkins_configure -q $queue -m $_netmod \
+ -s mpich-master/test/mpi/summary.junit.xml
+ if [[ -f mpich-master/test/mpi/summary.junit.xml ]]; then
+ exit 0
+ fi
+ fi
+ ;;
+ "per-commit")
+ if [[ -x maint/jenkins/skip_test.sh ]]; then
+ ./maint/jenkins/skip_test.sh -j $JOB_NAME -c $compiler -o $jenkins_configure -q $queue -m $_netmod \
+ -s test/mpi/summary.junit.xml
+ if [[ -f test/mpi/summary.junit.xml ]]; then
+ exit 0
+ fi
+ fi
+ ;;
+esac
+
+
+if test -d "$TMP_WORKSPACE"; then
+ rm -rf "$TMP_WORKSPACE"
+fi
+SRC=$WORKSPACE
+TMP_SRC=$TMP_WORKSPACE
+
+# Preparing the source
+case "$BUILD_MODE" in
+ "nightly")
+ SRC=$WORKSPACE/mpich-master
+ TMP_SRC=$TMP_WORKSPACE/mpich-master
+ ;;
+ "per-commit")
+ ;;
+ *)
+ echo "Invalid BUILD_MODE $BUILD_MODE. Set by mistake?"
+ exit 1
+esac
+
+srun mkdir -p $TMP_SRC
+
+# distribute source and binary
+pushd $WORKSPACE
+srun cp $WORKSPACE/mpich-test-pack.tar $TMP_SRC
+srun tar xf $TMP_SRC/mpich-test-pack.tar -C $TMP_SRC
+popd
+
+
+#####################################################################
+## Functions
+#####################################################################
+
+CollectResults() {
+ # TODO: copy saved test binaries (for failed cases)
+ if [[ "$BUILD_MODE" != "per-commit" ]]; then
+ find . \
+ \( -name "filtered-make.txt" \
+ -o -name "apply-xfail.sh" \
+ -o -name "autogen.log" \
+ -o -name "config.log" \
+ -o -name "c.txt" \
+ -o -name "m.txt" \
+ -o -name "mi.txt" \
+ -o -name "summary.junit.xml" \) \
+ | while read -r line; do
+ mkdir -p "$SRC/$(dirname $line)"
+ done
+ fi
+
+ find . \
+ \( -name "filtered-make.txt" -o \
+ -name "apply-xfail.sh" -o \
+ -name "autogen.log" -o \
+ -name "config.log" -o \
+ -name "c.txt" -o \
+ -name "m.txt" -o \
+ -name "mi.txt" -o \
+ -name "summary.junit.xml" \) \
+ -exec cp {} $SRC/{} \;
+}
+
+PrepareEnv() {
+ case "$queue" in
+ "ubuntu32" | "ubuntu64" )
+ source /software/common/adm/etc/softenv-aliases.sh
+ source /software/common/adm/etc/softenv-load.sh
+ soft add +intel
+ soft add +pgi
+ soft add +absoft
+ soft add +nagfor
+ soft add +solarisstudio-12.4
+ soft add +ekopath
+ ;;
+ "freebsd64" | "freebsd32")
+ PATH=/usr/local/bin:$PATH
+ LD_LIBRARY_PATH=/usr/local/lib/gcc46:$LD_LIBRARY_PATH
+ export LD_LIBRARY_PATH
+ ;;
+ "solaris")
+ PATH=/usr/gcc/4.3/bin:/usr/gnu/bin:/usr/sbin:$PATH
+ CONFIG_SHELL=/usr/gnu/bin/sh
+ export CONFIG_SHELL
+ ;;
+ esac
+ PATH=$HOME/software/autotools/bin:$PATH
+ export PATH
+ echo "$PATH"
+}
+
+
+#####################################################################
+## Main() { Run tests
+## Multi-node tests are running from Jenkins workspace
+#####################################################################
+
+pushd "$TMP_SRC"
+
+# Preparation
+case "$jenkins_configure" in
+ "mpd")
+ $TMP_SRC/_inst/bin/mpd &
+ sleep 1
+ ;;
+ "async")
+ MPIR_CVAR_ASYNC_PROGRESS=1
+ export MPIR_CVAR_ASYNC_PROGRESS
+ ;;
+ "multithread")
+ MPIR_CVAR_DEFAULT_THREAD_LEVEL=MPI_THREAD_MULTIPLE
+ export MPIR_CVAR_DEFAULT_THREAD_LEVEL
+ ;;
+esac
+
+case "$netmod" in
+ "mxm")
+ MXM_LOG_LEVEL=error
+ export MXM_LOG_LEVEL
+ ;;
+ "ofi" | "portals4")
+ MXM_LOG_LEVEL=error
+ export MXM_LOG_LEVEL
+ ;;
+esac
+
+# run only the minimum level of datatype tests when it is per-commit job
+if [[ "$BUILD_MODE" = "per-commit" ]]; then
+ MPITEST_DATATYPE_TEST_LEVEL=min
+ export MPITEST_DATATYPE_TEST_LEVEL
+fi
+
+# hack for passing -iface ib0
+if test "$netmod" = "tcp"; then
+ sed -i 's+/bin/mpiexec+/bin/mpiexec -iface ib0+g' test/mpi/runtests
+fi
+
+make testing
+
+# Cleanup
+case "$jenkins_configure" in
+ "mpd")
+ $TMP_SRC/_inst/bin/mpdallexit
+ ;;
+ "async")
+ unset MPIR_CVAR_ASYNC_PROGRESS
+ ;;
+esac
+
+#####################################################################
+## Copy Test results and Cleanup
+#####################################################################
+
+CollectResults
+
+popd
+srun rm -rf $TMP_WORKSPACE
+exit 0
+
http://git.mpich.org/mpich.git/commitdiff/ecbe04f11e73de7505dac9b46db8849431bb5e31
commit ecbe04f11e73de7505dac9b46db8849431bb5e31
Author: Yanfei Guo <yguo at anl.gov>
Date: Mon Sep 28 13:22:04 2015 -0500
maint/jenkins: print hostname in test worker scripts
The hostname of the node that runs the test is printed at the beginning
of the test job.
No reviewer.
diff --git a/maint/jenkins/test-worker-abi-prolog.sh b/maint/jenkins/test-worker-abi-prolog.sh
index 52dd32f..e9e7146 100755
--- a/maint/jenkins/test-worker-abi-prolog.sh
+++ b/maint/jenkins/test-worker-abi-prolog.sh
@@ -1,5 +1,7 @@
#!/bin/zsh -xe
+hostname
+
WORKSPACE=""
compiler="gnu"
jenkins_configure="default"
diff --git a/maint/jenkins/test-worker-abi.sh b/maint/jenkins/test-worker-abi.sh
index 1722558..4538e90 100755
--- a/maint/jenkins/test-worker-abi.sh
+++ b/maint/jenkins/test-worker-abi.sh
@@ -1,5 +1,7 @@
#!/bin/zsh -x
+hostname
+
WORKSPACE=""
build_mpi="mpich-master"
compiler="gnu"
diff --git a/maint/jenkins/test-worker-armci.sh b/maint/jenkins/test-worker-armci.sh
index d11164c..a073864 100755
--- a/maint/jenkins/test-worker-armci.sh
+++ b/maint/jenkins/test-worker-armci.sh
@@ -1,5 +1,7 @@
#!/bin/zsh -xe
+hostname
+
WORKSPACE=""
GIT_BRANCH=""
BUILD_MODE="per-commit"
diff --git a/maint/jenkins/test-worker-tarball.sh b/maint/jenkins/test-worker-tarball.sh
index ee443a6..300631b 100755
--- a/maint/jenkins/test-worker-tarball.sh
+++ b/maint/jenkins/test-worker-tarball.sh
@@ -1,5 +1,7 @@
#!/bin/zsh -xe
+hostname
+
export PATH=$HOME/software/autotools/bin:$HOME/software/sowing/bin:$PATH
export DOCTEXT_PATH=$HOME/software/sowing/share/doctext
export CC=gcc
diff --git a/maint/jenkins/test-worker.sh b/maint/jenkins/test-worker.sh
index 2a0e115..fdf3e9a 100755
--- a/maint/jenkins/test-worker.sh
+++ b/maint/jenkins/test-worker.sh
@@ -1,5 +1,7 @@
#!/bin/zsh -xe
+hostname
+
WORKSPACE=""
compiler="gnu"
jenkins_configure="default"
-----------------------------------------------------------------------
Summary of changes:
.../{test-worker.sh => multinode/mn-build.sh} | 94 +++------
maint/jenkins/multinode/mn-run.sh | 240 ++++++++++++++++++++
maint/jenkins/multinode/timeout.conf | 23 ++
maint/jenkins/{set-xfail.sh => set-timeout.sh} | 50 ++--
maint/jenkins/test-worker-abi-prolog.sh | 2 +
maint/jenkins/test-worker-abi.sh | 2 +
maint/jenkins/test-worker-armci.sh | 2 +
maint/jenkins/test-worker-tarball.sh | 20 ++-
maint/jenkins/test-worker.sh | 8 +-
maint/jenkins/xfail.conf | 2 +-
10 files changed, 343 insertions(+), 100 deletions(-)
copy maint/jenkins/{test-worker.sh => multinode/mn-build.sh} (88%)
create mode 100755 maint/jenkins/multinode/mn-run.sh
create mode 100644 maint/jenkins/multinode/timeout.conf
copy maint/jenkins/{set-xfail.sh => set-timeout.sh} (66%)
hooks/post-receive
--
MPICH primary repository
More information about the commits
mailing list