[mpich-discuss] MPI_Info key host error in SLURM
Iker Martín Álvarez
martini at uji.es
Mon Mar 1 13:37:43 CST 2021
Hi Ken,
Thanks for your reply.
Here are attached two files with the arg "-v". *BatchOutput.txt *has the
output which is giving the error message when the code is executed with the
command *sbatch*, while the file *InteractiveOutput.txt* is the one
executed interactively in the node and work as expected. Both of them has
been compiled and executed with mpich 3.4.1.
Thanks, Iker
El vie, 26 feb 2021 a las 16:15, Raffenetti, Kenneth J. (<
raffenet at mcs.anl.gov>) escribió:
> Hi,
>
> Could you add "-v" to your mpiexec command and provide the output? The
> "host" info key is handled by the process manager when executing the spawn
> command.
>
> Ken
>
> On 2/24/21, 11:55 AM, "Iker Martín Álvarez via discuss" <
> discuss at mpich.org> wrote:
>
> Hi,
> I have been working around the MPI_Comm_spawn function with the key
> "host" along a value in a system with the SLURM resource manager (slurm-wlm
> 17.11.2). The function works as expected, but when I send to execute the
> code with the sbatch command, an error arises. This does not happen when I
> execute directly the code in the same machine SLURM decided to execute it
> when it was sended with sbatch. In both cases with the key "host", as when
> I do not use the key, it works just fine.
>
> The same code has been tested with MPICH 3.3.2 and 3.4.1, which gives
> different errors. Also, I tried it with other implementations (OpenMPI and
> Intel MPI), which works as expected creating the processes in the indicated
> host.
>
> I would like to create processes by MPI_Comm_spawn in an assigned
> host, so if there are other key values for the Info argument, I could try
> them, but I have not found any looking through MPICH documentation.
>
> Here is the code I have been using:
>
> int main(int argc, char ** argv) {
>
> int myId, numP;
> MPI_Info info;
> MPI_Comm comm;
>
> MPI_Init(&argc, &argv);
> MPI_Comm_rank(MPI_COMM_WORLD, &myId);
> MPI_Comm_size(MPI_COMM_WORLD, &numP);
>
> MPI_Comm comm_par;
> MPI_Comm_get_parent(&comm_par);
> if(comm_par != MPI_COMM_NULL ) {
> if(myId == ROOT) {
> printf("SONS\n"); fflush(stdout);
> }
> } else {
> if(myId == ROOT) {
> printf("FATHERS\n"); fflush(stdout);
> }
> MPI_Info_create(&info);
> MPI_Info_set(info, "host", "n00");
> MPI_Comm_spawn(argv[0], MPI_ARGV_NULL, numP, info, ROOT,
> MPI_COMM_WORLD, &comm, MPI_ERRCODES_IGNORE);
> }
> MPI_Finalize();
> return 0;
> }
>
>
> For MPICH 3.4.1 there is sometimes no error and the code stops working
> at MPI_Comm_spawn function, other times this error is shown:
> Abort(1615120) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init:
> Internal MPI error!, error stack:
> MPIR_Init_thread(152)...:
> MPID_Init(562)..........:
> MPIDU_Init_shm_init(195):
> Init_shm_barrier(94)....: Internal MPI error! barrier not initialized
>
>
> Also, the error code for MPICH 3.3.2:
>
> Assertion failed in file
> src/mpid/ch3/channels/nemesis/src/mpid_nem_init.c at line 683: our_pg_rank
> < pg->size
> Assertion failed in file
> src/mpid/ch3/channels/nemesis/src/mpid_nem_init.c at line 683: our_pg_rank
> < pg->size
> Assertion failed in file
> src/mpid/ch3/channels/nemesis/src/mpid_nem_init.c at line 683: our_pg_rank
> < pg->size
> libbacktrace: no debug info in ELF executable
> libbacktrace: no debug info in ELF executable
> libbacktrace: no debug info in ELF executable
> libbacktrace: no debug info in ELF executable
> libbacktrace: no debug info in ELF executable
> libbacktrace: no debug info in ELF executable
> libbacktrace: no debug info in ELF executable
> libbacktrace: no debug info in ELF executable
> libbacktrace: no debug info in ELF executable
> libbacktrace: no debug info in ELF executable
> libbacktrace: no debug info in ELF executable
> libbacktrace: no debug info in ELF executable
> internal ABORT - process 0
>
> Thanks, Iker
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mpich.org/pipermail/discuss/attachments/20210301/4a166ead/attachment-0001.html>
-------------- next part --------------
Test
host: n00
==================================================================================================
mpiexec options:
----------------
Base path: /home/martini/MPI/mpich-3.4.1_INST/bin/
Launcher: (null)
Debug level: 1
Enable X: -1
Global environment:
-------------------
LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:
LD_LIBRARY_PATH=/home/martini/MPI/mpich-3.4.1_INST/lib
SLURM_TASK_PID=8625
SSH_CONNECTION=150.128.98.104 37392 150.128.80.83 22
LESSCLOSE=/usr/bin/lesspipe %s %s
SLURM_PRIO_PROCESS=0
LANG=es_ES.UTF-8
SLURM_SUBMIT_DIR=/home/martini/tfm/TestSpawn/Threads/PingPong
HOSTNAME=n00
LD_RUN_PATH_modshare=/home/martini/MPI/mpich-3.4.1_INST/lib:1
OLDPWD=/home/martini/TestsSergio/slurm-job-expansion-test
ENVIRONMENT=BATCH
PATH_modshare=/usr/sbin:1:/bin:1:/usr/bin:1:/usr/games:1:/snap/bin:1:/home/martini/MPI/mpich-3.4.1_INST/bin:1:/usr/local/sbin:1:/usr/local/bin:1:/sbin:1:/usr/local/games:1
LOADEDMODULES_modshare=/home/martini/MODULES/modulefiles/mpich3.4:1
MPI_RUN=/home/martini/MPI/mpich-3.4.1_INST/bin/mpirun
SLURM_CHECKPOINT_IMAGE_DIR=/var/slurm/checkpoint
SLURM_JOB_GID=1002
SLURMD_NODENAME=n00
LD_LIBRARY_PATH_modshare=/home/martini/MPI/mpich-3.4.1_INST/lib:1
XDG_SESSION_ID=2265
MODULES_CMD=/usr/lib/x86_64-linux-gnu/modulecmd.tcl
USER=martini
ENV=/usr/share/modules/init/profile.sh
PWD=/home/martini/tfm/TestSpawn/Threads/PingPong
SLURM_JOB_NODELIST=n00
HOME=/home/martini
SLURM_CLUSTER_NAME=nasp
SSH_CLIENT=150.128.98.104 37392 22
SLURM_JOB_CPUS_PER_NODE=20
BASH_ENV=/usr/share/modules/init/bash
XDG_DATA_DIRS=/usr/local/share:/usr/share:/var/lib/snapd/desktop
SLURM_TOPOLOGY_ADDR=n00
_LMFILES__modshare=/home/martini/MODULES/modulefiles/mpich3.4:1
SLURM_WORKING_CLUSTER=nasp:nasp:6817:8192
SLURM_JOB_NAME=runBase.sh
TMPDIR=/tmp
LOADEDMODULES=/home/martini/MODULES/modulefiles/mpich3.4
SLURM_NODE_ALIASES=(null)
SLURM_TOPOLOGY_ADDR_PATTERN=node
SSH_TTY=/dev/pts/1
MAIL=/var/mail/martini
SLURM_JOB_NUM_NODES=1
MPI_HOME=/home/martini/MPI/mpich-3.4.1_INST
SHELL=/bin/bash
TERM=xterm
SLURM_JOB_UID=1002
MANPATH_modshare=:1:/home/martini/MPI/mpich-3.4.1_INST/share/man/:1
SLURM_JOB_PARTITION=P1
LD_RUN_PATH=/home/martini/MPI/mpich-3.4.1_INST/lib
SLURM_JOB_USER=martini
SHLVL=2
SLURM_SUBMIT_HOST=nasp
MANPATH=:/home/martini/MPI/mpich-3.4.1_INST/share/man/
MODULEPATH=/etc/environment-modules/modules:/usr/share/modules/versions:/usr/share/modules/$MODULE_VERSION/modulefiles:/usr/share/modules/modulefiles
SLURM_GTIDS=0
LOGNAME=martini
XDG_RUNTIME_DIR=/run/user/1002
MODULEPATH_modshare=/usr/share/modules/$MODULE_VERSION/modulefiles:1:/etc/environment-modules/modules:1:/usr/share/modules/modulefiles:1:/usr/share/modules/versions:1
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/martini/MPI/mpich-3.4.1_INST/bin
SLURM_JOB_ID=2177
_LMFILES_=/home/martini/MODULES/modulefiles/mpich3.4
MODULESHOME=/usr/share/modules
LESSOPEN=| /usr/bin/lesspipe %s
BASH_FUNC_module%%=() { unset _mlre _mlIFS _mlshdbg;
if [ "${MODULES_SILENT_SHELL_DEBUG:-0}" = '1' ]; then
case "$-" in
*v*x*)
set +vx;
_mlshdbg='vx'
;;
*v*)
set +v;
_mlshdbg='v'
;;
*x*)
set +x;
_mlshdbg='x'
;;
*)
_mlshdbg=''
;;
esac;
fi;
if [ -n "${IFS+x}" ]; then
_mlIFS=$IFS;
fi;
IFS=' ';
for _mlv in ${MODULES_RUN_QUARANTINE:-};
do
if [ "${_mlv}" = "${_mlv##*[!A-Za-z0-9_]}" -a "${_mlv}" = "${_mlv#[0-9]}" ]; then
if [ -n "`eval 'echo ${'$_mlv'+x}'`" ]; then
_mlre="${_mlre:-}${_mlv}_modquar='`eval 'echo ${'$_mlv'}'`' ";
fi;
_mlrv="MODULES_RUNENV_${_mlv}";
_mlre="${_mlre:-}${_mlv}='`eval 'echo ${'$_mlrv':-}'`' ";
fi;
done;
if [ -n "${_mlre:-}" ]; then
_mlre="eval ${_mlre}";
fi;
eval `${_mlre:-}/usr/bin/tclsh /usr/lib/x86_64-linux-gnu/modulecmd.tcl bash $*`;
_mlstatus=$?;
if [ -n "${_mlIFS+x}" ]; then
IFS=$_mlIFS;
else
unset IFS;
fi;
if [ -n "${_mlshdbg:-}" ]; then
set -$_mlshdbg;
fi;
unset _mlre _mlv _mlrv _mlIFS _mlshdbg;
return $_mlstatus
}
BASH_FUNC_switchml%%=() { typeset swfound=1;
if [ "${MODULES_USE_COMPAT_VERSION:-0}" = '1' ]; then
typeset swname='main';
if [ -e /usr/lib/x86_64-linux-gnu/modulecmd.tcl ]; then
typeset swfound=0;
unset MODULES_USE_COMPAT_VERSION;
fi;
else
typeset swname='compatibility';
if [ -e /usr/lib/x86_64-linux-gnu/modulecmd-compat ]; then
typeset swfound=0;
MODULES_USE_COMPAT_VERSION=1;
export MODULES_USE_COMPAT_VERSION;
fi;
fi;
if [ $swfound -eq 0 ]; then
echo "Switching to Modules $swname version";
source /usr/share/modules/init/bash;
else
echo "Cannot switch to Modules $swname version, command not found";
return 1;
fi
}
BASH_FUNC__moduleraw%%=() { unset _mlre _mlIFS _mlshdbg;
if [ "${MODULES_SILENT_SHELL_DEBUG:-0}" = '1' ]; then
case "$-" in
*v*x*)
set +vx;
_mlshdbg='vx'
;;
*v*)
set +v;
_mlshdbg='v'
;;
*x*)
set +x;
_mlshdbg='x'
;;
*)
_mlshdbg=''
;;
esac;
fi;
if [ -n "${IFS+x}" ]; then
_mlIFS=$IFS;
fi;
IFS=' ';
for _mlv in ${MODULES_RUN_QUARANTINE:-};
do
if [ "${_mlv}" = "${_mlv##*[!A-Za-z0-9_]}" -a "${_mlv}" = "${_mlv#[0-9]}" ]; then
if [ -n "`eval 'echo ${'$_mlv'+x}'`" ]; then
_mlre="${_mlre:-}${_mlv}_modquar='`eval 'echo ${'$_mlv'}'`' ";
fi;
_mlrv="MODULES_RUNENV_${_mlv}";
_mlre="${_mlre:-}${_mlv}='`eval 'echo ${'$_mlrv':-}'`' ";
fi;
done;
if [ -n "${_mlre:-}" ]; then
_mlre="eval ${_mlre}";
fi;
eval `${_mlre:-}/usr/bin/tclsh /usr/lib/x86_64-linux-gnu/modulecmd.tcl bash $*`;
_mlstatus=$?;
if [ -n "${_mlIFS+x}" ]; then
IFS=$_mlIFS;
else
unset IFS;
fi;
if [ -n "${_mlshdbg:-}" ]; then
set -$_mlshdbg;
fi;
unset _mlre _mlv _mlrv _mlIFS _mlshdbg;
return $_mlstatus
}
_=/home/martini/MPI/mpich-3.4.1_INST/bin/mpirun
Hydra internal environment:
---------------------------
GFORTRAN_UNBUFFERED_PRECONNECTED=y
Proxy information:
*********************
[1] proxy: n00 (20 cores)
Exec list: ./batch5.out (4 processes);
==================================================================================================
[mpiexec at n00] Timeout set to -1 (-1 means infinite)
[mpiexec at n00] Got a control port string of n00:45129
Proxy launch args: /home/martini/MPI/mpich-3.4.1_INST/bin/hydra_pmi_proxy --control-port n00:45129 --debug --rmk slurm --launcher slurm --demux poll --pgid 0 --retries 10 --usize -2 --pmi-port 0 --gpus-per-proc -2 --proxy-id
Arguments being passed to proxy 0:
--version 3.4.1 --iface-ip-env-name MPIR_CVAR_CH3_INTERFACE_HOSTNAME --hostname n00 --global-core-map 0,20,20 --pmi-id-map 0,0 --global-process-count 4 --auto-cleanup 1 --pmi-kvsname kvs_8631_0_15624359_n00 --pmi-process-mapping (vector,(0,1,20)) --global-inherited-env 67 'LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:' 'LD_LIBRARY_PATH=/home/martini/MPI/mpich-3.4.1_INST/lib' 'SLURM_TASK_PID=8625' 'SSH_CONNECTION=150.128.98.104 37392 150.128.80.83 22' 'LESSCLOSE=/usr/bin/lesspipe %s %s' 'SLURM_PRIO_PROCESS=0' 'LANG=es_ES.UTF-8' 'SLURM_SUBMIT_DIR=/home/martini/tfm/TestSpawn/Threads/PingPong' 'HOSTNAME=n00' 'LD_RUN_PATH_modshare=/home/martini/MPI/mpich-3.4.1_INST/lib:1' 'OLDPWD=/home/martini/TestsSergio/slurm-job-expansion-test' 'ENVIRONMENT=BATCH' 'PATH_modshare=/usr/sbin:1:/bin:1:/usr/bin:1:/usr/games:1:/snap/bin:1:/home/martini/MPI/mpich-3.4.1_INST/bin:1:/usr/local/sbin:1:/usr/local/bin:1:/sbin:1:/usr/local/games:1' 'LOADEDMODULES_modshare=/home/martini/MODULES/modulefiles/mpich3.4:1' 'MPI_RUN=/home/martini/MPI/mpich-3.4.1_INST/bin/mpirun' 'SLURM_CHECKPOINT_IMAGE_DIR=/var/slurm/checkpoint' 'SLURM_JOB_GID=1002' 'SLURMD_NODENAME=n00' 'LD_LIBRARY_PATH_modshare=/home/martini/MPI/mpich-3.4.1_INST/lib:1' 'XDG_SESSION_ID=2265' 'MODULES_CMD=/usr/lib/x86_64-linux-gnu/modulecmd.tcl' 'USER=martini' 'ENV=/usr/share/modules/init/profile.sh' 'PWD=/home/martini/tfm/TestSpawn/Threads/PingPong' 'SLURM_JOB_NODELIST=n00' 'HOME=/home/martini' 'SLURM_CLUSTER_NAME=nasp' 'SSH_CLIENT=150.128.98.104 37392 22' 'SLURM_JOB_CPUS_PER_NODE=20' 'BASH_ENV=/usr/share/modules/init/bash' 'XDG_DATA_DIRS=/usr/local/share:/usr/share:/var/lib/snapd/desktop' 'SLURM_TOPOLOGY_ADDR=n00' '_LMFILES__modshare=/home/martini/MODULES/modulefiles/mpich3.4:1' 'SLURM_WORKING_CLUSTER=nasp:nasp:6817:8192' 'SLURM_JOB_NAME=runBase.sh' 'TMPDIR=/tmp' 'LOADEDMODULES=/home/martini/MODULES/modulefiles/mpich3.4' 'SLURM_NODE_ALIASES=(null)' 'SLURM_TOPOLOGY_ADDR_PATTERN=node' 'SSH_TTY=/dev/pts/1' 'MAIL=/var/mail/martini' 'SLURM_JOB_NUM_NODES=1' 'MPI_HOME=/home/martini/MPI/mpich-3.4.1_INST' 'SHELL=/bin/bash' 'TERM=xterm' 'SLURM_JOB_UID=1002' 'MANPATH_modshare=:1:/home/martini/MPI/mpich-3.4.1_INST/share/man/:1' 'SLURM_JOB_PARTITION=P1' 'LD_RUN_PATH=/home/martini/MPI/mpich-3.4.1_INST/lib' 'SLURM_JOB_USER=martini' 'SHLVL=2' 'SLURM_SUBMIT_HOST=nasp' 'MANPATH=:/home/martini/MPI/mpich-3.4.1_INST/share/man/' 'MODULEPATH=/etc/environment-modules/modules:/usr/share/modules/versions:/usr/share/modules/$MODULE_VERSION/modulefiles:/usr/share/modules/modulefiles' 'SLURM_GTIDS=0' 'LOGNAME=martini' 'XDG_RUNTIME_DIR=/run/user/1002' 'MODULEPATH_modshare=/usr/share/modules/$MODULE_VERSION/modulefiles:1:/etc/environment-modules/modules:1:/usr/share/modules/modulefiles:1:/usr/share/modules/versions:1' 'PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/martini/MPI/mpich-3.4.1_INST/bin' 'SLURM_JOB_ID=2177' '_LMFILES_=/home/martini/MODULES/modulefiles/mpich3.4' 'MODULESHOME=/usr/share/modules' 'LESSOPEN=| /usr/bin/lesspipe %s' 'BASH_FUNC_module%%=() { unset _mlre _mlIFS _mlshdbg;
if [ "${MODULES_SILENT_SHELL_DEBUG:-0}" = '1' ]; then
case "$-" in
*v*x*)
set +vx;
_mlshdbg='vx'
;;
*v*)
set +v;
_mlshdbg='v'
;;
*x*)
set +x;
_mlshdbg='x'
;;
*)
_mlshdbg=''
;;
esac;
fi;
if [ -n "${IFS+x}" ]; then
_mlIFS=$IFS;
fi;
IFS=' ';
for _mlv in ${MODULES_RUN_QUARANTINE:-};
do
if [ "${_mlv}" = "${_mlv##*[!A-Za-z0-9_]}" -a "${_mlv}" = "${_mlv#[0-9]}" ]; then
if [ -n "`eval 'echo ${'$_mlv'+x}'`" ]; then
_mlre="${_mlre:-}${_mlv}_modquar='`eval 'echo ${'$_mlv'}'`' ";
fi;
_mlrv="MODULES_RUNENV_${_mlv}";
_mlre="${_mlre:-}${_mlv}='`eval 'echo ${'$_mlrv':-}'`' ";
fi;
done;
if [ -n "${_mlre:-}" ]; then
_mlre="eval ${_mlre}";
fi;
eval `${_mlre:-}/usr/bin/tclsh /usr/lib/x86_64-linux-gnu/modulecmd.tcl bash $*`;
_mlstatus=$?;
if [ -n "${_mlIFS+x}" ]; then
IFS=$_mlIFS;
else
unset IFS;
fi;
if [ -n "${_mlshdbg:-}" ]; then
set -$_mlshdbg;
fi;
unset _mlre _mlv _mlrv _mlIFS _mlshdbg;
return $_mlstatus
}' 'BASH_FUNC_switchml%%=() { typeset swfound=1;
if [ "${MODULES_USE_COMPAT_VERSION:-0}" = '1' ]; then
typeset swname='main';
if [ -e /usr/lib/x86_64-linux-gnu/modulecmd.tcl ]; then
typeset swfound=0;
unset MODULES_USE_COMPAT_VERSION;
fi;
else
typeset swname='compatibility';
if [ -e /usr/lib/x86_64-linux-gnu/modulecmd-compat ]; then
typeset swfound=0;
MODULES_USE_COMPAT_VERSION=1;
export MODULES_USE_COMPAT_VERSION;
fi;
fi;
if [ $swfound -eq 0 ]; then
echo "Switching to Modules $swname version";
source /usr/share/modules/init/bash;
else
echo "Cannot switch to Modules $swname version, command not found";
return 1;
fi
}' 'BASH_FUNC__moduleraw%%=() { unset _mlre _mlIFS _mlshdbg;
if [ "${MODULES_SILENT_SHELL_DEBUG:-0}" = '1' ]; then
case "$-" in
*v*x*)
set +vx;
_mlshdbg='vx'
;;
*v*)
set +v;
_mlshdbg='v'
;;
*x*)
set +x;
_mlshdbg='x'
;;
*)
_mlshdbg=''
;;
esac;
fi;
if [ -n "${IFS+x}" ]; then
_mlIFS=$IFS;
fi;
IFS=' ';
for _mlv in ${MODULES_RUN_QUARANTINE:-};
do
if [ "${_mlv}" = "${_mlv##*[!A-Za-z0-9_]}" -a "${_mlv}" = "${_mlv#[0-9]}" ]; then
if [ -n "`eval 'echo ${'$_mlv'+x}'`" ]; then
_mlre="${_mlre:-}${_mlv}_modquar='`eval 'echo ${'$_mlv'}'`' ";
fi;
_mlrv="MODULES_RUNENV_${_mlv}";
_mlre="${_mlre:-}${_mlv}='`eval 'echo ${'$_mlrv':-}'`' ";
fi;
done;
if [ -n "${_mlre:-}" ]; then
_mlre="eval ${_mlre}";
fi;
eval `${_mlre:-}/usr/bin/tclsh /usr/lib/x86_64-linux-gnu/modulecmd.tcl bash $*`;
_mlstatus=$?;
if [ -n "${_mlIFS+x}" ]; then
IFS=$_mlIFS;
else
unset IFS;
fi;
if [ -n "${_mlshdbg:-}" ]; then
set -$_mlshdbg;
fi;
unset _mlre _mlv _mlrv _mlIFS _mlshdbg;
return $_mlstatus
}' '_=/home/martini/MPI/mpich-3.4.1_INST/bin/mpirun' --global-user-env 0 --global-system-env 1 'GFORTRAN_UNBUFFERED_PRECONNECTED=y' --proxy-core-count 20 --exec --exec-appnum 0 --exec-proc-count 4 --exec-local-env 0 --exec-wdir /home/martini/tfm/TestSpawn/Threads/PingPong --exec-args 1 ./batch5.out
[mpiexec at n00] Launch arguments: /usr/bin/srun -N 1 -n 1 --input none /home/martini/MPI/mpich-3.4.1_INST/bin/hydra_pmi_proxy --control-port n00:45129 --debug --rmk slurm --launcher slurm --demux poll --pgid 0 --retries 10 --usize -2 --pmi-port 0 --gpus-per-proc -2 --proxy-id -1
[proxy:0:0 at n00] got pmi command (from 6): init
pmi_version=1 pmi_subversion=1
[proxy:0:0 at n00] PMI response: cmd=response_to_init pmi_version=1 pmi_subversion=1 rc=0
[mpiexec at n00] [pgid: 0] got PMI command: cmd=put -bcast-1-0=2F6465762F73686D2F6D706963685F736861725F746D7033395355686E
[mpiexec at n00] [pgid: 0] got PMI command: cmd=barrier_in
[mpiexec at n00] PMI response to fd 7 pid 4: cmd=keyval_cache -bcast-1-0=2F6465762F73686D2F6D706963685F736861725F746D7033395355686E
[mpiexec at n00] PMI response to fd 7 pid 4: cmd=barrier_out
[proxy:0:0 at n00] got pmi command (from 6): get_maxes
[proxy:0:0 at n00] PMI response: cmd=maxes kvsname_max=256 keylen_max=64 vallen_max=1024
[proxy:0:0 at n00] got pmi command (from 6): get_appnum
[proxy:0:0 at n00] PMI response: cmd=appnum appnum=0
[proxy:0:0 at n00] got pmi command (from 6): get_my_kvsname
[proxy:0:0 at n00] PMI response: cmd=my_kvsname kvsname=kvs_8631_0_15624359_n00
[proxy:0:0 at n00] got pmi command (from 6): get
kvsname=kvs_8631_0_15624359_n00 key=PMI_process_mapping
[proxy:0:0 at n00] PMI response: cmd=get_result rc=0 msg=success value=(vector,(0,1,20))
[proxy:0:0 at n00] got pmi command (from 6): barrier_in
[proxy:0:0 at n00] got pmi command (from 5): init
pmi_version=1 pmi_subversion=1
[proxy:0:0 at n00] PMI response: cmd=response_to_init pmi_version=1 pmi_subversion=1 rc=0
[proxy:0:0 at n00] got pmi command (from 5): get_maxes
[proxy:0:0 at n00] PMI response: cmd=maxes kvsname_max=256 keylen_max=64 vallen_max=1024
[proxy:0:0 at n00] got pmi command (from 5): get_appnum
[proxy:0:0 at n00] PMI response: cmd=appnum appnum=0
[proxy:0:0 at n00] got pmi command (from 5): get_my_kvsname
[proxy:0:0 at n00] PMI response: cmd=my_kvsname kvsname=kvs_8631_0_15624359_n00
[proxy:0:0 at n00] got pmi command (from 5): get
kvsname=kvs_8631_0_15624359_n00 key=PMI_process_mapping
[proxy:0:0 at n00] PMI response: cmd=get_result rc=0 msg=success value=(vector,(0,1,20))
[proxy:0:0 at n00] got pmi command (from 5): barrier_in
[proxy:0:0 at n00] got pmi command (from 9): init
pmi_version=1 pmi_subversion=1
[proxy:0:0 at n00] PMI response: cmd=response_to_init pmi_version=1 pmi_subversion=1 rc=0
[proxy:0:0 at n00] got pmi command (from 9): get_maxes
[proxy:0:0 at n00] PMI response: cmd=maxes kvsname_max=256 keylen_max=64 vallen_max=1024
[proxy:0:0 at n00] got pmi command (from 9): get_appnum
[proxy:0:0 at n00] PMI response: cmd=appnum appnum=0
[proxy:0:0 at n00] got pmi command (from 9): get_my_kvsname
[proxy:0:0 at n00] PMI response: cmd=my_kvsname kvsname=kvs_8631_0_15624359_n00
[proxy:0:0 at n00] got pmi command (from 9): get
kvsname=kvs_8631_0_15624359_n00 key=PMI_process_mapping
[proxy:0:0 at n00] PMI response: cmd=get_result rc=0 msg=success value=(vector,(0,1,20))
[proxy:0:0 at n00] got pmi command (from 9): barrier_in
[proxy:0:0 at n00] got pmi command (from 4): init
pmi_version=1 pmi_subversion=1
[proxy:0:0 at n00] PMI response: cmd=response_to_init pmi_version=1 pmi_subversion=1 rc=0
[proxy:0:0 at n00] got pmi command (from 4): get_maxes
[proxy:0:0 at n00] PMI response: cmd=maxes kvsname_max=256 keylen_max=64 vallen_max=1024
[proxy:0:0 at n00] got pmi command (from 4): get_appnum
[proxy:0:0 at n00] PMI response: cmd=appnum appnum=0
[proxy:0:0 at n00] got pmi command (from 4): get_my_kvsname
[proxy:0:0 at n00] PMI response: cmd=my_kvsname kvsname=kvs_8631_0_15624359_n00
[proxy:0:0 at n00] got pmi command (from 4): get
kvsname=kvs_8631_0_15624359_n00 key=PMI_process_mapping
[proxy:0:0 at n00] PMI response: cmd=get_result rc=0 msg=success value=(vector,(0,1,20))
[proxy:0:0 at n00] got pmi command (from 4): put
kvsname=kvs_8631_0_15624359_n00 key=-bcast-1-0 value=2F6465762F73686D2F6D706963685F736861725F746D7033395355686E
[proxy:0:0 at n00] cached command: -bcast-1-0=2F6465762F73686D2F6D706963685F736861725F746D7033395355686E
[proxy:0:0 at n00] PMI response: cmd=put_result rc=0 msg=success
[proxy:0:0 at n00] got pmi command (from 4): barrier_in
[proxy:0:0 at n00] flushing 1 put command(s) out
[proxy:0:0 at n00] forwarding command (cmd=put -bcast-1-0=2F6465762F73686D2F6D706963685F736861725F746D7033395355686E) upstream
[proxy:0:0 at n00] forwarding command (cmd=barrier_in) upstream
[proxy:0:0 at n00] PMI response: cmd=barrier_out
[proxy:0:0 at n00] PMI response: cmd=barrier_out
[proxy:0:0 at n00] PMI response: cmd=barrier_out
[proxy:0:0 at n00] PMI response: cmd=barrier_out
[proxy:0:0 at n00] got pmi command (from 5): get
kvsname=kvs_8631_0_15624359_n00 key=-bcast-1-0
[proxy:0:0 at n00] PMI response: cmd=get_result rc=0 msg=success value=2F6465762F73686D2F6D706963685F736861725F746D7033395355686E
[proxy:0:0 at n00] got pmi command (from 6): get
kvsname=kvs_8631_0_15624359_n00 key=-bcast-1-0
[proxy:0:0 at n00] PMI response: cmd=get_result rc=0 msg=success value=2F6465762F73686D2F6D706963685F736861725F746D7033395355686E
[proxy:0:0 at n00] got pmi command (from 9): get
kvsname=kvs_8631_0_15624359_n00 key=-bcast-1-0
[proxy:0:0 at n00] PMI response: cmd=get_result rc=0 msg=success value=2F6465762F73686D2F6D706963685F736861725F746D7033395355686E
[proxy:0:0 at n00] got pmi command (from 4): put
kvsname=kvs_8631_0_15624359_n00 key=-allgather-shm-1-0 value=0200A7D1C0A8020A0000000000000000
[proxy:0:0 at n00] cached command: -allgather-shm-1-0=0200A7D1C0A8020A0000000000000000
[proxy:0:0 at n00] PMI response: cmd=put_result rc=0 msg=success
[proxy:0:0 at n00] got pmi command (from 5): put
kvsname=kvs_8631_0_15624359_n00 key=-allgather-shm-1-1 value=02009D61C0A8020A0000000000000000
[proxy:0:0 at n00] cached command: -allgather-shm-1-1=02009D61C0A8020A0000000000000000
[proxy:0:0 at n00] PMI response: cmd=put_result rc=0 msg=success
[proxy:0:0 at n00] got pmi command (from 6): put
kvsname=kvs_8631_0_15624359_n00 key=-allgather-shm-1-2 value=02008285C0A8020A0000000000000000
[proxy:0:0 at n00] cached command: -allgather-shm-1-2=02008285C0A8020A0000000000000000
[proxy:0:0 at n00] PMI response: cmd=put_result rc=0 msg=success
[proxy:0:0 at n00] got pmi command (from 9): put
kvsname=kvs_8631_0_15624359_n00 key=-allgather-shm-1-3 value=02008981C0A8020A0000000000000000
[proxy:0:0 at n00] cached command: -allgather-shm-1-3=02008981C0A8020A0000000000000000
[proxy:0:0 at n00] PMI response: cmd=put_result rc=0 msg=success
[proxy:0:0 at n00] got pmi command (from 4): barrier_in
[proxy:0:0 at n00] got pmi command (from 5): barrier_in
[proxy:0:0 at n00] got pmi command (from 6): barrier_in
[mpiexec at n00] [pgid: 0] got PMI command: cmd=put -allgather-shm-1-0=0200A7D1C0A8020A0000000000000000 -allgather-shm-1-1=02009D61C0A8020A0000000000000000 -allgather-shm-1-2=02008285C0A8020A0000000000000000 -allgather-shm-1-3=02008981C0A8020A0000000000000000
[proxy:0:0 at n00] got pmi command (from 9): barrier_in
[proxy:0:0 at n00] flushing 4 put command(s) out
[proxy:0:0 at n00] forwarding command (cmd=put -allgather-shm-1-0=0200A7D1C0A8020A0000000000000000 -allgather-shm-1-1=02009D61C0A8020A0000000000000000 -allgather-shm-1-2=02008285C0A8020A0000000000000000 -allgather-shm-1-3=02008981C0A8020A0000000000000000) upstream
[proxy:0:0 at n00] forwarding command (cmd=barrier_in) upstream
[mpiexec at n00] [pgid: 0] got PMI command: cmd=barrier_in
[mpiexec at n00] PMI response to fd 7 pid 9: cmd=keyval_cache -allgather-shm-1-0=0200A7D1C0A8020A0000000000000000 -allgather-shm-1-1=02009D61C0A8020A0000000000000000 -allgather-shm-1-2=02008285C0A8020A0000000000000000 -allgather-shm-1-3=02008981C0A8020A0000000000000000
[mpiexec at n00] PMI response to fd 7 pid 9: cmd=barrier_out
[proxy:0:0 at n00] PMI response: cmd=barrier_out
[proxy:0:0 at n00] PMI response: cmd=barrier_out
[proxy:0:0 at n00] PMI response: cmd=barrier_out
[proxy:0:0 at n00] PMI response: cmd=barrier_out
[proxy:0:0 at n00] got pmi command (from 4): get
kvsname=kvs_8631_0_15624359_n00 key=-allgather-shm-1-0
[proxy:0:0 at n00] PMI response: cmd=get_result rc=0 msg=success value=0200A7D1C0A8020A0000000000000000
[proxy:0:0 at n00] got pmi command (from 5): get
kvsname=kvs_8631_0_15624359_n00 key=-allgather-shm-1-1
[proxy:0:0 at n00] PMI response: cmd=get_result rc=0 msg=success value=02009D61C0A8020A0000000000000000
[proxy:0:0 at n00] got pmi command (from 6): get
kvsname=kvs_8631_0_15624359_n00 key=-allgather-shm-1-2
[proxy:0:0 at n00] PMI response: cmd=get_result rc=0 msg=success value=02008285C0A8020A0000000000000000
[proxy:0:0 at n00] got pmi command (from 9): get
kvsname=kvs_8631_0_15624359_n00 key=-allgather-shm-1-3
[proxy:0:0 at n00] PMI response: cmd=get_result rc=0 msg=success value=02008981C0A8020A0000000000000000
PADRES
Name n00
Name n00
Name n00
Name n00
[mpiexec at n00] [pgid: 0] got PMI command: mcmd=spawn
nprocs=4
execname=./batch5.out
totspawns=1
spawnssofar=1
argcnt=0
preput_num=1
preput_key_0=PARENT_ROOT_PORT_NAME
preput_val_0=tag#0$connentry#0200A7D1C0A8020A0000000000000000$
info_num=1
info_key_0=host
info_val_0=n00
endcmd
[mpiexec at n00] Got a control port string of n00:44475
Proxy launch args: /home/martini/MPI/mpich-3.4.1_INST/bin/hydra_pmi_proxy --control-port n00:44475 --debug --rmk slurm --launcher slurm --demux poll --pgid 1 --retries 10 --usize -2 --pmi-port 0 --gpus-per-proc -2 --proxy-id
Arguments being passed to proxy 0:
--version 3.4.1 --iface-ip-env-name MPIR_CVAR_CH3_INTERFACE_HOSTNAME --hostname n00 --global-core-map 0,1,20 --pmi-id-map 0,0 --global-process-count 4 --auto-cleanup 1 --pmi-kvsname kvs_8631_1_1985138167_n00 --pmi-spawner-kvsname kvs_8631_0_15624359_n00 --pmi-process-mapping (vector,(0,1,1)) --global-inherited-env 67 'LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:' 'LD_LIBRARY_PATH=/home/martini/MPI/mpich-3.4.1_INST/lib' 'SLURM_TASK_PID=8625' 'SSH_CONNECTION=150.128.98.104 37392 150.128.80.83 22' 'LESSCLOSE=/usr/bin/lesspipe %s %s' 'SLURM_PRIO_PROCESS=0' 'LANG=es_ES.UTF-8' 'SLURM_SUBMIT_DIR=/home/martini/tfm/TestSpawn/Threads/PingPong' 'HOSTNAME=n00' 'LD_RUN_PATH_modshare=/home/martini/MPI/mpich-3.4.1_INST/lib:1' 'OLDPWD=/home/martini/TestsSergio/slurm-job-expansion-test' 'ENVIRONMENT=BATCH' 'PATH_modshare=/usr/sbin:1:/bin:1:/usr/bin:1:/usr/games:1:/snap/bin:1:/home/martini/MPI/mpich-3.4.1_INST/bin:1:/usr/local/sbin:1:/usr/local/bin:1:/sbin:1:/usr/local/games:1' 'LOADEDMODULES_modshare=/home/martini/MODULES/modulefiles/mpich3.4:1' 'MPI_RUN=/home/martini/MPI/mpich-3.4.1_INST/bin/mpirun' 'SLURM_CHECKPOINT_IMAGE_DIR=/var/slurm/checkpoint' 'SLURM_JOB_GID=1002' 'SLURMD_NODENAME=n00' 'LD_LIBRARY_PATH_modshare=/home/martini/MPI/mpich-3.4.1_INST/lib:1' 'XDG_SESSION_ID=2265' 'MODULES_CMD=/usr/lib/x86_64-linux-gnu/modulecmd.tcl' 'USER=martini' 'ENV=/usr/share/modules/init/profile.sh' 'PWD=/home/martini/tfm/TestSpawn/Threads/PingPong' 'SLURM_JOB_NODELIST=n00' 'HOME=/home/martini' 'SLURM_CLUSTER_NAME=nasp' 'SSH_CLIENT=150.128.98.104 37392 22' 'SLURM_JOB_CPUS_PER_NODE=20' 'BASH_ENV=/usr/share/modules/init/bash' 'XDG_DATA_DIRS=/usr/local/share:/usr/share:/var/lib/snapd/desktop' 'SLURM_TOPOLOGY_ADDR=n00' '_LMFILES__modshare=/home/martini/MODULES/modulefiles/mpich3.4:1' 'SLURM_WORKING_CLUSTER=nasp:nasp:6817:8192' 'SLURM_JOB_NAME=runBase.sh' 'TMPDIR=/tmp' 'LOADEDMODULES=/home/martini/MODULES/modulefiles/mpich3.4' 'SLURM_NODE_ALIASES=(null)' 'SLURM_TOPOLOGY_ADDR_PATTERN=node' 'SSH_TTY=/dev/pts/1' 'MAIL=/var/mail/martini' 'SLURM_JOB_NUM_NODES=1' 'MPI_HOME=/home/martini/MPI/mpich-3.4.1_INST' 'SHELL=/bin/bash' 'TERM=xterm' 'SLURM_JOB_UID=1002' 'MANPATH_modshare=:1:/home/martini/MPI/mpich-3.4.1_INST/share/man/:1' 'SLURM_JOB_PARTITION=P1' 'LD_RUN_PATH=/home/martini/MPI/mpich-3.4.1_INST/lib' 'SLURM_JOB_USER=martini' 'SHLVL=2' 'SLURM_SUBMIT_HOST=nasp' 'MANPATH=:/home/martini/MPI/mpich-3.4.1_INST/share/man/' 'MODULEPATH=/etc/environment-modules/modules:/usr/share/modules/versions:/usr/share/modules/$MODULE_VERSION/modulefiles:/usr/share/modules/modulefiles' 'SLURM_GTIDS=0' 'LOGNAME=martini' 'XDG_RUNTIME_DIR=/run/user/1002' 'MODULEPATH_modshare=/usr/share/modules/$MODULE_VERSION/modulefiles:1:/etc/environment-modules/modules:1:/usr/share/modules/modulefiles:1:/usr/share/modules/versions:1' 'PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/martini/MPI/mpich-3.4.1_INST/bin' 'SLURM_JOB_ID=2177' '_LMFILES_=/home/martini/MODULES/modulefiles/mpich3.4' 'MODULESHOME=/usr/share/modules' 'LESSOPEN=| /usr/bin/lesspipe %s' 'BASH_FUNC_module%%=() { unset _mlre _mlIFS _mlshdbg;
if [ "${MODULES_SILENT_SHELL_DEBUG:-0}" = '1' ]; then
case "$-" in
*v*x*)
set +vx;
_mlshdbg='vx'
;;
*v*)
set +v;
_mlshdbg='v'
;;
*x*)
set +x;
_mlshdbg='x'
;;
*)
_mlshdbg=''
;;
esac;
fi;
if [ -n "${IFS+x}" ]; then
_mlIFS=$IFS;
fi;
IFS=' ';
for _mlv in ${MODULES_RUN_QUARANTINE:-};
do
if [ "${_mlv}" = "${_mlv##*[!A-Za-z0-9_]}" -a "${_mlv}" = "${_mlv#[0-9]}" ]; then
if [ -n "`eval 'echo ${'$_mlv'+x}'`" ]; then
_mlre="${_mlre:-}${_mlv}_modquar='`eval 'echo ${'$_mlv'}'`' ";
fi;
_mlrv="MODULES_RUNENV_${_mlv}";
_mlre="${_mlre:-}${_mlv}='`eval 'echo ${'$_mlrv':-}'`' ";
fi;
done;
if [ -n "${_mlre:-}" ]; then
_mlre="eval ${_mlre}";
fi;
eval `${_mlre:-}/usr/bin/tclsh /usr/lib/x86_64-linux-gnu/modulecmd.tcl bash $*`;
_mlstatus=$?;
if [ -n "${_mlIFS+x}" ]; then
IFS=$_mlIFS;
else
unset IFS;
fi;
if [ -n "${_mlshdbg:-}" ]; then
set -$_mlshdbg;
fi;
unset _mlre _mlv _mlrv _mlIFS _mlshdbg;
return $_mlstatus
}' 'BASH_FUNC_switchml%%=() { typeset swfound=1;
if [ "${MODULES_USE_COMPAT_VERSION:-0}" = '1' ]; then
typeset swname='main';
if [ -e /usr/lib/x86_64-linux-gnu/modulecmd.tcl ]; then
typeset swfound=0;
unset MODULES_USE_COMPAT_VERSION;
fi;
else
typeset swname='compatibility';
if [ -e /usr/lib/x86_64-linux-gnu/modulecmd-compat ]; then
typeset swfound=0;
MODULES_USE_COMPAT_VERSION=1;
export MODULES_USE_COMPAT_VERSION;
fi;
fi;
if [ $swfound -eq 0 ]; then
echo "Switching to Modules $swname version";
source /usr/share/modules/init/bash;
else
echo "Cannot switch to Modules $swname version, command not found";
return 1;
fi
}' 'BASH_FUNC__moduleraw%%=() { unset _mlre _mlIFS _mlshdbg;
if [ "${MODULES_SILENT_SHELL_DEBUG:-0}" = '1' ]; then
case "$-" in
*v*x*)
set +vx;
_mlshdbg='vx'
;;
*v*)
set +v;
_mlshdbg='v'
;;
*x*)
set +x;
_mlshdbg='x'
;;
*)
_mlshdbg=''
;;
esac;
fi;
if [ -n "${IFS+x}" ]; then
_mlIFS=$IFS;
fi;
IFS=' ';
for _mlv in ${MODULES_RUN_QUARANTINE:-};
do
if [ "${_mlv}" = "${_mlv##*[!A-Za-z0-9_]}" -a "${_mlv}" = "${_mlv#[0-9]}" ]; then
if [ -n "`eval 'echo ${'$_mlv'+x}'`" ]; then
_mlre="${_mlre:-}${_mlv}_modquar='`eval 'echo ${'$_mlv'}'`' ";
fi;
_mlrv="MODULES_RUNENV_${_mlv}";
_mlre="${_mlre:-}${_mlv}='`eval 'echo ${'$_mlrv':-}'`' ";
fi;
done;
if [ -n "${_mlre:-}" ]; then
_mlre="eval ${_mlre}";
fi;
eval `${_mlre:-}/usr/bin/tclsh /usr/lib/x86_64-linux-gnu/modulecmd.tcl bash $*`;
_mlstatus=$?;
if [ -n "${_mlIFS+x}" ]; then
IFS=$_mlIFS;
else
unset IFS;
fi;
if [ -n "${_mlshdbg:-}" ]; then
set -$_mlshdbg;
fi;
unset _mlre _mlv _mlrv _mlIFS _mlshdbg;
return $_mlstatus
}' '_=/home/martini/MPI/mpich-3.4.1_INST/bin/mpirun' --global-user-env 0 --global-system-env 1 'GFORTRAN_UNBUFFERED_PRECONNECTED=y' --proxy-core-count 1 --exec --exec-appnum 0 --exec-proc-count 4 --exec-local-env 1 'PMI_SPAWNED=1' --exec-wdir /home/martini/tfm/TestSpawn/Threads/PingPong --exec-args 1 ./batch5.out
[mpiexec at n00] Launch arguments: /usr/bin/srun --nodelist n00 -N 1 -n 1 --input none /home/martini/MPI/mpich-3.4.1_INST/bin/hydra_pmi_proxy --control-port n00:44475 --debug --rmk slurm --launcher slurm --demux poll --pgid 1 --retries 10 --usize -2 --pmi-port 0 --gpus-per-proc -2 --proxy-id -1
[mpiexec at n00] PMI response to fd 7 pid 4: cmd=spawn_result rc=0
[proxy:0:0 at n00] got pmi command (from 4): spawn
nprocs=4 execname=./batch5.out totspawns=1 spawnssofar=1 argcnt=0 preput_num=1 preput_key_0=PARENT_ROOT_PORT_NAME preput_val_0=tag#0$connentry#0200A7D1C0A8020A0000000000000000$ info_num=1 info_key_0=host info_val_0=n00 endcmd
[proxy:0:0 at n00] we don't understand this command spawn; forwarding upstream
[proxy:0:0 at n00] we don't understand the response spawn_result; forwarding downstream
[proxy:1:0 at n00] got pmi command (from 4): init
pmi_version=1 pmi_subversion=1
[proxy:1:0 at n00] PMI response: cmd=response_to_init pmi_version=1 pmi_subversion=1 rc=0
[mpiexec at n00] [pgid: 1] got PMI command: cmd=put -bcast-1-0=2F6465762F73686D2F6D706963685F736861725F746D70396F597A524C
[mpiexec at n00] [pgid: 1] got PMI command: cmd=barrier_in
[mpiexec at n00] PMI response to fd 11 pid 5: cmd=keyval_cache PARENT_ROOT_PORT_NAME=tag#0$connentry#0200A7D1C0A8020A0000000000000000$ -bcast-1-0=2F6465762F73686D2F6D706963685F736861725F746D70396F597A524C
[mpiexec at n00] PMI response to fd 11 pid 5: cmd=barrier_out
[mpiexec at n00] [pgid: 1] got PMI command: cmd=abort exitcode=1615120
-------------- next part --------------
host: n06
==================================================================================================
mpiexec options:
----------------
Base path: /home/martini/MPI/mpich-3.4.1_INST/bin/
Launcher: (null)
Debug level: 1
Enable X: -1
Global environment:
-------------------
LD_LIBRARY_PATH=/home/martini/MPI/mpich-3.4.1_INST/lib
LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:
SSH_CONNECTION=192.168.1.1 54318 192.168.1.16 22
LESSCLOSE=/usr/bin/lesspipe %s %s
LANG=es_ES.UTF-8
LD_RUN_PATH_modshare=/home/martini/MPI/mpich-3.4.1_INST/lib:1
OLDPWD=/home/martini
PATH_modshare=/usr/sbin:1:/bin:1:/usr/bin:1:/usr/games:1:/snap/bin:1:/home/martini/MPI/mpich-3.4.1_INST/bin:1:/usr/local/sbin:1:/usr/local/bin:1:/sbin:1:/usr/local/games:1
LOADEDMODULES_modshare=/home/martini/MODULES/modulefiles/mpich3.4:1
MPI_RUN=/home/martini/MPI/mpich-3.4.1_INST/bin/mpirun
LD_LIBRARY_PATH_modshare=/home/martini/MPI/mpich-3.4.1_INST/lib:1
XDG_SESSION_ID=1858
MODULES_CMD=/usr/lib/x86_64-linux-gnu/modulecmd.tcl
USER=martini
ENV=/usr/share/modules/init/profile.sh
PWD=/home/martini/tfm/TestSpawn/Threads/PingPong
HOME=/home/martini
SSH_CLIENT=192.168.1.1 54318 22
BASH_ENV=/usr/share/modules/init/bash
XDG_DATA_DIRS=/usr/local/share:/usr/share:/var/lib/snapd/desktop
_LMFILES__modshare=/home/martini/MODULES/modulefiles/mpich3.4:1
LOADEDMODULES=/home/martini/MODULES/modulefiles/mpich3.4
SSH_TTY=/dev/pts/0
MAIL=/var/mail/martini
MPI_HOME=/home/martini/MPI/mpich-3.4.1_INST
TERM=xterm
SHELL=/bin/bash
MANPATH_modshare=:1:/home/martini/MPI/mpich-3.4.1_INST/share/man/:1
LD_RUN_PATH=/home/martini/MPI/mpich-3.4.1_INST/lib
SHLVL=1
MANPATH=:/home/martini/MPI/mpich-3.4.1_INST/share/man/
MODULEPATH=/etc/environment-modules/modules:/usr/share/modules/versions:/usr/share/modules/$MODULE_VERSION/modulefiles:/usr/share/modules/modulefiles
LOGNAME=martini
XDG_RUNTIME_DIR=/run/user/1002
MODULEPATH_modshare=/usr/share/modules/$MODULE_VERSION/modulefiles:1:/etc/environment-modules/modules:1:/usr/share/modules/modulefiles:1:/usr/share/modules/versions:1
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/martini/MPI/mpich-3.4.1_INST/bin
_LMFILES_=/home/martini/MODULES/modulefiles/mpich3.4
MODULESHOME=/usr/share/modules
LESSOPEN=| /usr/bin/lesspipe %s
BASH_FUNC_module%%=() { _moduleraw "$*" 2>&1
}
BASH_FUNC_switchml%%=() { typeset swfound=1;
if [ "${MODULES_USE_COMPAT_VERSION:-0}" = '1' ]; then
typeset swname='main';
if [ -e /usr/lib/x86_64-linux-gnu/modulecmd.tcl ]; then
typeset swfound=0;
unset MODULES_USE_COMPAT_VERSION;
fi;
else
typeset swname='compatibility';
if [ -e /usr/lib/x86_64-linux-gnu/modulecmd-compat ]; then
typeset swfound=0;
MODULES_USE_COMPAT_VERSION=1;
export MODULES_USE_COMPAT_VERSION;
fi;
fi;
if [ $swfound -eq 0 ]; then
echo "Switching to Modules $swname version";
source /usr/share/modules/init/bash;
else
echo "Cannot switch to Modules $swname version, command not found";
return 1;
fi
}
BASH_FUNC__moduleraw%%=() { unset _mlre _mlIFS _mlshdbg;
if [ "${MODULES_SILENT_SHELL_DEBUG:-0}" = '1' ]; then
case "$-" in
*v*x*)
set +vx;
_mlshdbg='vx'
;;
*v*)
set +v;
_mlshdbg='v'
;;
*x*)
set +x;
_mlshdbg='x'
;;
*)
_mlshdbg=''
;;
esac;
fi;
if [ -n "${IFS+x}" ]; then
_mlIFS=$IFS;
fi;
IFS=' ';
for _mlv in ${MODULES_RUN_QUARANTINE:-};
do
if [ "${_mlv}" = "${_mlv##*[!A-Za-z0-9_]}" -a "${_mlv}" = "${_mlv#[0-9]}" ]; then
if [ -n "`eval 'echo ${'$_mlv'+x}'`" ]; then
_mlre="${_mlre:-}${_mlv}_modquar='`eval 'echo ${'$_mlv'}'`' ";
fi;
_mlrv="MODULES_RUNENV_${_mlv}";
_mlre="${_mlre:-}${_mlv}='`eval 'echo ${'$_mlrv':-}'`' ";
fi;
done;
if [ -n "${_mlre:-}" ]; then
_mlre="eval ${_mlre}";
fi;
eval `${_mlre:-}/usr/bin/tclsh /usr/lib/x86_64-linux-gnu/modulecmd.tcl bash $*`;
_mlstatus=$?;
if [ -n "${_mlIFS+x}" ]; then
IFS=$_mlIFS;
else
unset IFS;
fi;
if [ -n "${_mlshdbg:-}" ]; then
set -$_mlshdbg;
fi;
unset _mlre _mlv _mlrv _mlIFS _mlshdbg;
return $_mlstatus
}
_=/home/martini/MPI/mpich-3.4.1_INST/bin/mpirun
Hydra internal environment:
---------------------------
GFORTRAN_UNBUFFERED_PRECONNECTED=y
Proxy information:
*********************
[1] proxy: n06 (1 cores)
Exec list: ./batch5.out (4 processes);
==================================================================================================
[mpiexec at n06] Timeout set to -1 (-1 means infinite)
[mpiexec at n06] Got a control port string of n06:37519
Proxy launch args: /home/martini/MPI/mpich-3.4.1_INST/bin/hydra_pmi_proxy --control-port n06:37519 --debug --rmk user --launcher ssh --demux poll --pgid 0 --retries 10 --usize -2 --pmi-port 0 --gpus-per-proc -2 --proxy-id
Arguments being passed to proxy 0:
--version 3.4.1 --iface-ip-env-name MPIR_CVAR_CH3_INTERFACE_HOSTNAME --hostname n06 --global-core-map 0,1,1 --pmi-id-map 0,0 --global-process-count 4 --auto-cleanup 1 --pmi-kvsname kvs_25318_0_67517667_n06 --pmi-process-mapping (vector,(0,1,1)) --global-inherited-env 43 'LD_LIBRARY_PATH=/home/martini/MPI/mpich-3.4.1_INST/lib' 'LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:' 'SSH_CONNECTION=192.168.1.1 54318 192.168.1.16 22' 'LESSCLOSE=/usr/bin/lesspipe %s %s' 'LANG=es_ES.UTF-8' 'LD_RUN_PATH_modshare=/home/martini/MPI/mpich-3.4.1_INST/lib:1' 'OLDPWD=/home/martini' 'PATH_modshare=/usr/sbin:1:/bin:1:/usr/bin:1:/usr/games:1:/snap/bin:1:/home/martini/MPI/mpich-3.4.1_INST/bin:1:/usr/local/sbin:1:/usr/local/bin:1:/sbin:1:/usr/local/games:1' 'LOADEDMODULES_modshare=/home/martini/MODULES/modulefiles/mpich3.4:1' 'MPI_RUN=/home/martini/MPI/mpich-3.4.1_INST/bin/mpirun' 'LD_LIBRARY_PATH_modshare=/home/martini/MPI/mpich-3.4.1_INST/lib:1' 'XDG_SESSION_ID=1858' 'MODULES_CMD=/usr/lib/x86_64-linux-gnu/modulecmd.tcl' 'USER=martini' 'ENV=/usr/share/modules/init/profile.sh' 'PWD=/home/martini/tfm/TestSpawn/Threads/PingPong' 'HOME=/home/martini' 'SSH_CLIENT=192.168.1.1 54318 22' 'BASH_ENV=/usr/share/modules/init/bash' 'XDG_DATA_DIRS=/usr/local/share:/usr/share:/var/lib/snapd/desktop' '_LMFILES__modshare=/home/martini/MODULES/modulefiles/mpich3.4:1' 'LOADEDMODULES=/home/martini/MODULES/modulefiles/mpich3.4' 'SSH_TTY=/dev/pts/0' 'MAIL=/var/mail/martini' 'MPI_HOME=/home/martini/MPI/mpich-3.4.1_INST' 'TERM=xterm' 'SHELL=/bin/bash' 'MANPATH_modshare=:1:/home/martini/MPI/mpich-3.4.1_INST/share/man/:1' 'LD_RUN_PATH=/home/martini/MPI/mpich-3.4.1_INST/lib' 'SHLVL=1' 'MANPATH=:/home/martini/MPI/mpich-3.4.1_INST/share/man/' 'MODULEPATH=/etc/environment-modules/modules:/usr/share/modules/versions:/usr/share/modules/$MODULE_VERSION/modulefiles:/usr/share/modules/modulefiles' 'LOGNAME=martini' 'XDG_RUNTIME_DIR=/run/user/1002' 'MODULEPATH_modshare=/usr/share/modules/$MODULE_VERSION/modulefiles:1:/etc/environment-modules/modules:1:/usr/share/modules/modulefiles:1:/usr/share/modules/versions:1' 'PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/martini/MPI/mpich-3.4.1_INST/bin' '_LMFILES_=/home/martini/MODULES/modulefiles/mpich3.4' 'MODULESHOME=/usr/share/modules' 'LESSOPEN=| /usr/bin/lesspipe %s' 'BASH_FUNC_module%%=() { _moduleraw "$*" 2>&1
}' 'BASH_FUNC_switchml%%=() { typeset swfound=1;
if [ "${MODULES_USE_COMPAT_VERSION:-0}" = '1' ]; then
typeset swname='main';
if [ -e /usr/lib/x86_64-linux-gnu/modulecmd.tcl ]; then
typeset swfound=0;
unset MODULES_USE_COMPAT_VERSION;
fi;
else
typeset swname='compatibility';
if [ -e /usr/lib/x86_64-linux-gnu/modulecmd-compat ]; then
typeset swfound=0;
MODULES_USE_COMPAT_VERSION=1;
export MODULES_USE_COMPAT_VERSION;
fi;
fi;
if [ $swfound -eq 0 ]; then
echo "Switching to Modules $swname version";
source /usr/share/modules/init/bash;
else
echo "Cannot switch to Modules $swname version, command not found";
return 1;
fi
}' 'BASH_FUNC__moduleraw%%=() { unset _mlre _mlIFS _mlshdbg;
if [ "${MODULES_SILENT_SHELL_DEBUG:-0}" = '1' ]; then
case "$-" in
*v*x*)
set +vx;
_mlshdbg='vx'
;;
*v*)
set +v;
_mlshdbg='v'
;;
*x*)
set +x;
_mlshdbg='x'
;;
*)
_mlshdbg=''
;;
esac;
fi;
if [ -n "${IFS+x}" ]; then
_mlIFS=$IFS;
fi;
IFS=' ';
for _mlv in ${MODULES_RUN_QUARANTINE:-};
do
if [ "${_mlv}" = "${_mlv##*[!A-Za-z0-9_]}" -a "${_mlv}" = "${_mlv#[0-9]}" ]; then
if [ -n "`eval 'echo ${'$_mlv'+x}'`" ]; then
_mlre="${_mlre:-}${_mlv}_modquar='`eval 'echo ${'$_mlv'}'`' ";
fi;
_mlrv="MODULES_RUNENV_${_mlv}";
_mlre="${_mlre:-}${_mlv}='`eval 'echo ${'$_mlrv':-}'`' ";
fi;
done;
if [ -n "${_mlre:-}" ]; then
_mlre="eval ${_mlre}";
fi;
eval `${_mlre:-}/usr/bin/tclsh /usr/lib/x86_64-linux-gnu/modulecmd.tcl bash $*`;
_mlstatus=$?;
if [ -n "${_mlIFS+x}" ]; then
IFS=$_mlIFS;
else
unset IFS;
fi;
if [ -n "${_mlshdbg:-}" ]; then
set -$_mlshdbg;
fi;
unset _mlre _mlv _mlrv _mlIFS _mlshdbg;
return $_mlstatus
}' '_=/home/martini/MPI/mpich-3.4.1_INST/bin/mpirun' --global-user-env 0 --global-system-env 1 'GFORTRAN_UNBUFFERED_PRECONNECTED=y' --proxy-core-count 1 --exec --exec-appnum 0 --exec-proc-count 4 --exec-local-env 0 --exec-wdir /home/martini/tfm/TestSpawn/Threads/PingPong --exec-args 1 ./batch5.out
[mpiexec at n06] Launch arguments: /home/martini/MPI/mpich-3.4.1_INST/bin/hydra_pmi_proxy --control-port n06:37519 --debug --rmk user --launcher ssh --demux poll --pgid 0 --retries 10 --usize -2 --pmi-port 0 --gpus-per-proc -2 --proxy-id 0
[proxy:0:0 at n06] got pmi command (from 8): init
pmi_version=1 pmi_subversion=1
[proxy:0:0 at n06] PMI response: cmd=response_to_init pmi_version=1 pmi_subversion=1 rc=0
[proxy:0:0 at n06] got pmi command (from 8): get_maxes
[proxy:0:0 at n06] PMI response: cmd=maxes kvsname_max=256 keylen_max=64 vallen_max=1024
[proxy:0:0 at n06] got pmi command (from 8): get_appnum
[proxy:0:0 at n06] PMI response: cmd=appnum appnum=0
[proxy:0:0 at n06] got pmi command (from 8): get_my_kvsname
[proxy:0:0 at n06] PMI response: cmd=my_kvsname kvsname=kvs_25318_0_67517667_n06
[proxy:0:0 at n06] got pmi command (from 8): get
kvsname=kvs_25318_0_67517667_n06 key=PMI_process_mapping
[proxy:0:0 at n06] PMI response: cmd=get_result rc=0 msg=success value=(vector,(0,1,1))
[proxy:0:0 at n06] got pmi command (from 8): barrier_in
[proxy:0:0 at n06] got pmi command (from 13): init
pmi_version=1 pmi_subversion=1
[proxy:0:0 at n06] PMI response: cmd=response_to_init pmi_version=1 pmi_subversion=1 rc=0
[proxy:0:0 at n06] got pmi command (from 13): get_maxes
[proxy:0:0 at n06] PMI response: cmd=maxes kvsname_max=256 keylen_max=64 vallen_max=1024
[proxy:0:0 at n06] got pmi command (from 13): get_appnum
[proxy:0:0 at n06] PMI response: cmd=appnum appnum=0
[proxy:0:0 at n06] got pmi command (from 13): get_my_kvsname
[proxy:0:0 at n06] PMI response: cmd=my_kvsname kvsname=kvs_25318_0_67517667_n06
[proxy:0:0 at n06] got pmi command (from 13): get
kvsname=kvs_25318_0_67517667_n06 key=PMI_process_mapping
[proxy:0:0 at n06] PMI response: cmd=get_result rc=0 msg=success value=(vector,(0,1,1))
[proxy:0:0 at n06] got pmi command (from 13): barrier_in
[proxy:0:0 at n06] got pmi command (from 0): init
pmi_version=1 pmi_subversion=1
[proxy:0:0 at n06] PMI response: cmd=response_to_init pmi_version=1 pmi_subversion=1 rc=0
[proxy:0:0 at n06] got pmi command (from 0): get_maxes
[proxy:0:0 at n06] PMI response: cmd=maxes kvsname_max=256 keylen_max=64 vallen_max=1024
[proxy:0:0 at n06] got pmi command (from 0): get_appnum
[proxy:0:0 at n06] PMI response: cmd=appnum appnum=0
[proxy:0:0 at n06] got pmi command (from 0): get_my_kvsname
[proxy:0:0 at n06] PMI response: cmd=my_kvsname kvsname=kvs_25318_0_67517667_n06
[proxy:0:0 at n06] got pmi command (from 0): get
kvsname=kvs_25318_0_67517667_n06 key=PMI_process_mapping
[proxy:0:0 at n06] PMI response: cmd=get_result rc=0 msg=success value=(vector,(0,1,1))
[proxy:0:0 at n06] got pmi command (from 0): put
kvsname=kvs_25318_0_67517667_n06 key=-bcast-1-0 value=2F6465762F73686D2F6D706963685F736861725F746D7075664F353959
[proxy:0:0 at n06] cached command: -bcast-1-0=2F6465762F73686D2F6D706963685F736861725F746D7075664F353959
[proxy:0:0 at n06] PMI response: cmd=put_result rc=0 msg=success
[proxy:0:0 at n06] got pmi command (from 6): init
pmi_version=1 pmi_subversion=1
[proxy:0:0 at n06] PMI response: cmd=response_to_init pmi_version=1 pmi_subversion=1 rc=0
[proxy:0:0 at n06] got pmi command (from 0): barrier_in
[proxy:0:0 at n06] got pmi command (from 6): get_maxes
[proxy:0:0 at n06] PMI response: cmd=maxes kvsname_max=256 keylen_max=64 vallen_max=1024
[proxy:0:0 at n06] got pmi command (from 6): get_appnum
[proxy:0:0 at n06] PMI response: cmd=appnum appnum=0
[proxy:0:0 at n06] got pmi command (from 6): get_my_kvsname
[proxy:0:0 at n06] PMI response: cmd=my_kvsname kvsname=kvs_25318_0_67517667_n06
[proxy:0:0 at n06] got pmi command (from 6): get
kvsname=kvs_25318_0_67517667_n06 key=PMI_process_mapping
[proxy:0:0 at n06] PMI response: cmd=get_result rc=0 msg=success value=(vector,(0,1,1))
[proxy:0:0 at n06] got pmi command (from 6): barrier_in
[proxy:0:0 at n06] flushing 1 put command(s) out
[mpiexec at n06] [pgid: 0] got PMI command: cmd=put -bcast-1-0=2F6465762F73686D2F6D706963685F736861725F746D7075664F353959
[proxy:0:0 at n06] forwarding command (cmd=put -bcast-1-0=2F6465762F73686D2F6D706963685F736861725F746D7075664F353959) upstream
[proxy:0:0 at n06] forwarding command (cmd=barrier_in) upstream
[mpiexec at n06] [pgid: 0] got PMI command: cmd=barrier_in
[mpiexec at n06] PMI response to fd 6 pid 6: cmd=keyval_cache -bcast-1-0=2F6465762F73686D2F6D706963685F736861725F746D7075664F353959
[mpiexec at n06] PMI response to fd 6 pid 6: cmd=barrier_out
[proxy:0:0 at n06] PMI response: cmd=barrier_out
[proxy:0:0 at n06] PMI response: cmd=barrier_out
[proxy:0:0 at n06] PMI response: cmd=barrier_out
[proxy:0:0 at n06] PMI response: cmd=barrier_out
[proxy:0:0 at n06] got pmi command (from 6): get
kvsname=kvs_25318_0_67517667_n06 key=-bcast-1-0
[proxy:0:0 at n06] PMI response: cmd=get_result rc=0 msg=success value=2F6465762F73686D2F6D706963685F736861725F746D7075664F353959
[proxy:0:0 at n06] got pmi command (from 8): get
kvsname=kvs_25318_0_67517667_n06 key=-bcast-1-0
[proxy:0:0 at n06] PMI response: cmd=get_result rc=0 msg=success value=2F6465762F73686D2F6D706963685F736861725F746D7075664F353959
[proxy:0:0 at n06] got pmi command (from 13): get
kvsname=kvs_25318_0_67517667_n06 key=-bcast-1-0
[proxy:0:0 at n06] PMI response: cmd=get_result rc=0 msg=success value=2F6465762F73686D2F6D706963685F736861725F746D7075664F353959
[proxy:0:0 at n06] got pmi command (from 0): put
kvsname=kvs_25318_0_67517667_n06 key=-allgather-shm-1-0 value=020087B3C0A802100000000000000000
[proxy:0:0 at n06] cached command: -allgather-shm-1-0=020087B3C0A802100000000000000000
[proxy:0:0 at n06] PMI response: cmd=put_result rc=0 msg=success
[proxy:0:0 at n06] got pmi command (from 6): put
kvsname=kvs_25318_0_67517667_n06 key=-allgather-shm-1-1 value=02009D0FC0A802100000000000000000
[proxy:0:0 at n06] cached command: -allgather-shm-1-1=02009D0FC0A802100000000000000000
[proxy:0:0 at n06] PMI response: cmd=put_result rc=0 msg=success
[proxy:0:0 at n06] got pmi command (from 8): put
kvsname=kvs_25318_0_67517667_n06 key=-allgather-shm-1-2 value=0200ABFFC0A802100000000000000000
[proxy:0:0 at n06] cached command: -allgather-shm-1-2=0200ABFFC0A802100000000000000000
[proxy:0:0 at n06] PMI response: cmd=put_result rc=0 msg=success
[proxy:0:0 at n06] got pmi command (from 13): put
kvsname=kvs_25318_0_67517667_n06 key=-allgather-shm-1-3 value=0200A64FC0A802100000000000000000
[proxy:0:0 at n06] cached command: -allgather-shm-1-3=0200A64FC0A802100000000000000000
[proxy:0:0 at n06] PMI response: cmd=put_result rc=0 msg=success
[proxy:0:0 at n06] got pmi command (from 0): barrier_in
[proxy:0:0 at n06] got pmi command (from 6): barrier_in
[proxy:0:0 at n06] got pmi command (from 8): barrier_in
[proxy:0:0 at n06] got pmi command (from 13): barrier_in
[proxy:0:0 at n06] flushing 4 put command(s) out
[proxy:0:0 at n06] forwarding command (cmd=put -allgather-shm-1-0=020087B3C0A802100000000000000000 -allgather-shm-1-1=02009D0FC0A802100000000000000000 -allgather-shm-1-2=0200ABFFC0A802100000000000000000 -allgather-shm-1-3=0200A64FC0A802100000000000000000) upstream
[mpiexec at n06] [pgid: 0] got PMI command: cmd=put -allgather-shm-1-0=020087B3C0A802100000000000000000 -allgather-shm-1-1=02009D0FC0A802100000000000000000 -allgather-shm-1-2=0200ABFFC0A802100000000000000000 -allgather-shm-1-3=0200A64FC0A802100000000000000000
[proxy:0:0 at n06] forwarding command (cmd=barrier_in) upstream
[mpiexec at n06] [pgid: 0] got PMI command: cmd=barrier_in
[mpiexec at n06] PMI response to fd 6 pid 13: cmd=keyval_cache -allgather-shm-1-0=020087B3C0A802100000000000000000 -allgather-shm-1-1=02009D0FC0A802100000000000000000 -allgather-shm-1-2=0200ABFFC0A802100000000000000000 -allgather-shm-1-3=0200A64FC0A802100000000000000000
[mpiexec at n06] PMI response to fd 6 pid 13: cmd=barrier_out
[proxy:0:0 at n06] PMI response: cmd=barrier_out
[proxy:0:0 at n06] PMI response: cmd=barrier_out
[proxy:0:0 at n06] PMI response: cmd=barrier_out
[proxy:0:0 at n06] PMI response: cmd=barrier_out
[proxy:0:0 at n06] got pmi command (from 0): get
kvsname=kvs_25318_0_67517667_n06 key=-allgather-shm-1-0
[proxy:0:0 at n06] PMI response: cmd=get_result rc=0 msg=success value=020087B3C0A802100000000000000000
[proxy:0:0 at n06] got pmi command (from 6): get
kvsname=kvs_25318_0_67517667_n06 key=-allgather-shm-1-1
[proxy:0:0 at n06] PMI response: cmd=get_result rc=0 msg=success value=02009D0FC0A802100000000000000000
[proxy:0:0 at n06] got pmi command (from 8): get
kvsname=kvs_25318_0_67517667_n06 key=-allgather-shm-1-2
[proxy:0:0 at n06] PMI response: cmd=get_result rc=0 msg=success value=0200ABFFC0A802100000000000000000
[proxy:0:0 at n06] got pmi command (from 13): get
kvsname=kvs_25318_0_67517667_n06 key=-allgather-shm-1-3
[proxy:0:0 at n06] PMI response: cmd=get_result rc=0 msg=success value=0200A64FC0A802100000000000000000
PADRES
Name n06
Name n06
Name n06
Name n06
[proxy:0:0 at n06] got pmi command (from 0): spawn
nprocs=4 execname=./batch5.out totspawns=1 spawnssofar=1 argcnt=0 preput_num=1 preput_key_0=PARENT_ROOT_PORT_NAME preput_val_0=tag#0$connentry#020087B3C0A802100000000000000000$ info_num=1 info_key_0=host info_val_0=n00 endcmd
[proxy:0:0 at n06] we don't understand this command spawn; forwarding upstream
[mpiexec at n06] [pgid: 0] got PMI command: mcmd=spawn
nprocs=4
execname=./batch5.out
totspawns=1
spawnssofar=1
argcnt=0
preput_num=1
preput_key_0=PARENT_ROOT_PORT_NAME
preput_val_0=tag#0$connentry#020087B3C0A802100000000000000000$
info_num=1
info_key_0=host
info_val_0=n00
endcmd
[mpiexec at n06] Got a control port string of n06:45957
Proxy launch args: /home/martini/MPI/mpich-3.4.1_INST/bin/hydra_pmi_proxy --control-port n06:45957 --debug --rmk user --launcher ssh --demux poll --pgid 1 --retries 10 --usize -2 --pmi-port 0 --gpus-per-proc -2 --proxy-id
Arguments being passed to proxy 0:
--version 3.4.1 --iface-ip-env-name MPIR_CVAR_CH3_INTERFACE_HOSTNAME --hostname n00 --global-core-map 0,1,1 --pmi-id-map 0,0 --global-process-count 4 --auto-cleanup 1 --pmi-kvsname kvs_25318_1_247201886_n06 --pmi-spawner-kvsname kvs_25318_0_67517667_n06 --pmi-process-mapping (vector,(0,1,1)) --global-inherited-env 43 'LD_LIBRARY_PATH=/home/martini/MPI/mpich-3.4.1_INST/lib' 'LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:' 'SSH_CONNECTION=192.168.1.1 54318 192.168.1.16 22' 'LESSCLOSE=/usr/bin/lesspipe %s %s' 'LANG=es_ES.UTF-8' 'LD_RUN_PATH_modshare=/home/martini/MPI/mpich-3.4.1_INST/lib:1' 'OLDPWD=/home/martini' 'PATH_modshare=/usr/sbin:1:/bin:1:/usr/bin:1:/usr/games:1:/snap/bin:1:/home/martini/MPI/mpich-3.4.1_INST/bin:1:/usr/local/sbin:1:/usr/local/bin:1:/sbin:1:/usr/local/games:1' 'LOADEDMODULES_modshare=/home/martini/MODULES/modulefiles/mpich3.4:1' 'MPI_RUN=/home/martini/MPI/mpich-3.4.1_INST/bin/mpirun' 'LD_LIBRARY_PATH_modshare=/home/martini/MPI/mpich-3.4.1_INST/lib:1' 'XDG_SESSION_ID=1858' 'MODULES_CMD=/usr/lib/x86_64-linux-gnu/modulecmd.tcl' 'USER=martini' 'ENV=/usr/share/modules/init/profile.sh' 'PWD=/home/martini/tfm/TestSpawn/Threads/PingPong' 'HOME=/home/martini' 'SSH_CLIENT=192.168.1.1 54318 22' 'BASH_ENV=/usr/share/modules/init/bash' 'XDG_DATA_DIRS=/usr/local/share:/usr/share:/var/lib/snapd/desktop' '_LMFILES__modshare=/home/martini/MODULES/modulefiles/mpich3.4:1' 'LOADEDMODULES=/home/martini/MODULES/modulefiles/mpich3.4' 'SSH_TTY=/dev/pts/0' 'MAIL=/var/mail/martini' 'MPI_HOME=/home/martini/MPI/mpich-3.4.1_INST' 'TERM=xterm' 'SHELL=/bin/bash' 'MANPATH_modshare=:1:/home/martini/MPI/mpich-3.4.1_INST/share/man/:1' 'LD_RUN_PATH=/home/martini/MPI/mpich-3.4.1_INST/lib' 'SHLVL=1' 'MANPATH=:/home/martini/MPI/mpich-3.4.1_INST/share/man/' 'MODULEPATH=/etc/environment-modules/modules:/usr/share/modules/versions:/usr/share/modules/$MODULE_VERSION/modulefiles:/usr/share/modules/modulefiles' 'LOGNAME=martini' 'XDG_RUNTIME_DIR=/run/user/1002' 'MODULEPATH_modshare=/usr/share/modules/$MODULE_VERSION/modulefiles:1:/etc/environment-modules/modules:1:/usr/share/modules/modulefiles:1:/usr/share/modules/versions:1' 'PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/martini/MPI/mpich-3.4.1_INST/bin' '_LMFILES_=/home/martini/MODULES/modulefiles/mpich3.4' 'MODULESHOME=/usr/share/modules' 'LESSOPEN=| /usr/bin/lesspipe %s' 'BASH_FUNC_module%%=() { _moduleraw "$*" 2>&1
}' 'BASH_FUNC_switchml%%=() { typeset swfound=1;
if [ "${MODULES_USE_COMPAT_VERSION:-0}" = '1' ]; then
typeset swname='main';
if [ -e /usr/lib/x86_64-linux-gnu/modulecmd.tcl ]; then
typeset swfound=0;
unset MODULES_USE_COMPAT_VERSION;
fi;
else
typeset swname='compatibility';
if [ -e /usr/lib/x86_64-linux-gnu/modulecmd-compat ]; then
typeset swfound=0;
MODULES_USE_COMPAT_VERSION=1;
export MODULES_USE_COMPAT_VERSION;
fi;
fi;
if [ $swfound -eq 0 ]; then
echo "Switching to Modules $swname version";
source /usr/share/modules/init/bash;
else
echo "Cannot switch to Modules $swname version, command not found";
return 1;
fi
}' 'BASH_FUNC__moduleraw%%=() { unset _mlre _mlIFS _mlshdbg;
if [ "${MODULES_SILENT_SHELL_DEBUG:-0}" = '1' ]; then
case "$-" in
*v*x*)
set +vx;
_mlshdbg='vx'
;;
*v*)
set +v;
_mlshdbg='v'
;;
*x*)
set +x;
_mlshdbg='x'
;;
*)
_mlshdbg=''
;;
esac;
fi;
if [ -n "${IFS+x}" ]; then
_mlIFS=$IFS;
fi;
IFS=' ';
for _mlv in ${MODULES_RUN_QUARANTINE:-};
do
if [ "${_mlv}" = "${_mlv##*[!A-Za-z0-9_]}" -a "${_mlv}" = "${_mlv#[0-9]}" ]; then
if [ -n "`eval 'echo ${'$_mlv'+x}'`" ]; then
_mlre="${_mlre:-}${_mlv}_modquar='`eval 'echo ${'$_mlv'}'`' ";
fi;
_mlrv="MODULES_RUNENV_${_mlv}";
_mlre="${_mlre:-}${_mlv}='`eval 'echo ${'$_mlrv':-}'`' ";
fi;
done;
if [ -n "${_mlre:-}" ]; then
_mlre="eval ${_mlre}";
fi;
eval `${_mlre:-}/usr/bin/tclsh /usr/lib/x86_64-linux-gnu/modulecmd.tcl bash $*`;
_mlstatus=$?;
if [ -n "${_mlIFS+x}" ]; then
IFS=$_mlIFS;
else
unset IFS;
fi;
if [ -n "${_mlshdbg:-}" ]; then
set -$_mlshdbg;
fi;
unset _mlre _mlv _mlrv _mlIFS _mlshdbg;
return $_mlstatus
}' '_=/home/martini/MPI/mpich-3.4.1_INST/bin/mpirun' --global-user-env 0 --global-system-env 1 'GFORTRAN_UNBUFFERED_PRECONNECTED=y' --proxy-core-count 1 --exec --exec-appnum 0 --exec-proc-count 4 --exec-local-env 1 'PMI_SPAWNED=1' --exec-wdir /home/martini/tfm/TestSpawn/Threads/PingPong --exec-args 1 ./batch5.out
[mpiexec at n06] Launch arguments: /usr/bin/ssh -x n00 "/home/martini/MPI/mpich-3.4.1_INST/bin/hydra_pmi_proxy" --control-port n06:45957 --debug --rmk user --launcher ssh --demux poll --pgid 1 --retries 10 --usize -2 --pmi-port 0 --gpus-per-proc -2 --proxy-id 0
[mpiexec at n06] PMI response to fd 6 pid 0: cmd=spawn_result rc=0
[proxy:0:0 at n06] we don't understand the response spawn_result; forwarding downstream
[proxy:1:0 at n00] got pmi command (from 9): init
pmi_version=1 pmi_subversion=1
[proxy:1:0 at n00] PMI response: cmd=response_to_init pmi_version=1 pmi_subversion=1 rc=0
[proxy:1:0 at n00] got pmi command (from 9): get_maxes
[proxy:1:0 at n00] PMI response: cmd=maxes kvsname_max=256 keylen_max=64 vallen_max=1024
[proxy:1:0 at n00] got pmi command (from 9): get_appnum
[proxy:1:0 at n00] PMI response: cmd=appnum appnum=0
[proxy:1:0 at n00] got pmi command (from 9): get_my_kvsname
[proxy:1:0 at n00] PMI response: cmd=my_kvsname kvsname=kvs_25318_1_247201886_n06
[proxy:1:0 at n00] got pmi command (from 9): get
kvsname=kvs_25318_1_247201886_n06 key=PMI_process_mapping
[proxy:1:0 at n00] PMI response: cmd=get_result rc=0 msg=success value=(vector,(0,1,1))
[proxy:1:0 at n00] got pmi command (from 9): barrier_in
[proxy:1:0 at n00] got pmi command (from 4): init
pmi_version=1 pmi_subversion=1
[proxy:1:0 at n00] PMI response: cmd=response_to_init pmi_version=1 pmi_subversion=1 rc=0
[proxy:1:0 at n00] got pmi command (from 4): get_maxes
[proxy:1:0 at n00] PMI response: cmd=maxes kvsname_max=256 keylen_max=64 vallen_max=1024
[proxy:1:0 at n00] got pmi command (from 4): get_appnum
[proxy:1:0 at n00] PMI response: cmd=appnum appnum=0
[proxy:1:0 at n00] got pmi command (from 4): get_my_kvsname
[proxy:1:0 at n00] PMI response: cmd=my_kvsname kvsname=kvs_25318_1_247201886_n06
[proxy:1:0 at n00] got pmi command (from 4): get
kvsname=kvs_25318_1_247201886_n06 key=PMI_process_mapping
[proxy:1:0 at n00] PMI response: cmd=get_result rc=0 msg=success value=(vector,(0,1,1))
[proxy:1:0 at n00] got pmi command (from 4): put
kvsname=kvs_25318_1_247201886_n06 key=-bcast-1-0 value=2F6465762F73686D2F6D706963685F736861725F746D70743631524334
[proxy:1:0 at n00] cached command: -bcast-1-0=2F6465762F73686D2F6D706963685F736861725F746D70743631524334
[proxy:1:0 at n00] PMI response: cmd=put_result rc=0 msg=success
[proxy:1:0 at n00] got pmi command (from 4): barrier_in
[proxy:1:0 at n00] got pmi command (from 5): init
pmi_version=1 pmi_subversion=1
[proxy:1:0 at n00] PMI response: cmd=response_to_init pmi_version=1 pmi_subversion=1 rc=0
[proxy:1:0 at n00] got pmi command (from 5): get_maxes
[proxy:1:0 at n00] PMI response: cmd=maxes kvsname_max=256 keylen_max=64 vallen_max=1024
[proxy:1:0 at n00] got pmi command (from 5): get_appnum
[proxy:1:0 at n00] PMI response: cmd=appnum appnum=0
[proxy:1:0 at n00] got pmi command (from 5): get_my_kvsname
[proxy:1:0 at n00] PMI response: cmd=my_kvsname kvsname=kvs_25318_1_247201886_n06
[proxy:1:0 at n00] got pmi command (from 5): get
kvsname=kvs_25318_1_247201886_n06 key=PMI_process_mapping
[proxy:1:0 at n00] PMI response: cmd=get_result rc=0 msg=success value=(vector,(0,1,1))
[proxy:1:0 at n00] got pmi command (from 5): barrier_in
[proxy:1:0 at n00] got pmi command (from 6): init
pmi_version=1 pmi_subversion=1
[proxy:1:0 at n00] PMI response: cmd=response_to_init pmi_version=1 pmi_subversion=1 rc=0
[proxy:1:0 at n00] got pmi command (from 6): get_maxes
[proxy:1:0 at n00] PMI response: cmd=maxes kvsname_max=256 keylen_max=64 vallen_max=1024
[proxy:1:0 at n00] got pmi command (from 6): get_appnum
[proxy:1:0 at n00] PMI response: cmd=appnum appnum=0
[proxy:1:0 at n00] got pmi command (from 6): get_my_kvsname
[proxy:1:0 at n00] PMI response: cmd=my_kvsname kvsname=kvs_25318_1_247201886_n06
[proxy:1:0 at n00] got pmi command (from 6): get
kvsname=kvs_25318_1_247201886_n06 key=PMI_process_mapping
[proxy:1:0 at n00] PMI response: cmd=get_result rc=0 msg=success value=(vector,(0,1,1))
[mpiexec at n06] [pgid: 1] got PMI command: cmd=put -bcast-1-0=2F6465762F73686D2F6D706963685F736861725F746D70743631524334
[proxy:1:0 at n00] got pmi command (from 6): barrier_in
[proxy:1:0 at n00] flushing 1 put command(s) out
[proxy:1:0 at n00] forwarding command (cmd=put -bcast-1-0=2F6465762F73686D2F6D706963685F736861725F746D70743631524334) upstream
[proxy:1:0 at n00] forwarding command (cmd=barrier_in) upstream
[mpiexec at n06] [pgid: 1] got PMI command: cmd=barrier_in
[mpiexec at n06] PMI response to fd 9 pid 6: cmd=keyval_cache PARENT_ROOT_PORT_NAME=tag#0$connentry#020087B3C0A802100000000000000000$ -bcast-1-0=2F6465762F73686D2F6D706963685F736861725F746D70743631524334
[mpiexec at n06] PMI response to fd 9 pid 6: cmd=barrier_out
[proxy:1:0 at n00] PMI response: cmd=barrier_out
[proxy:1:0 at n00] PMI response: cmd=barrier_out
[proxy:1:0 at n00] PMI response: cmd=barrier_out
[proxy:1:0 at n00] PMI response: cmd=barrier_out
[proxy:1:0 at n00] got pmi command (from 5): get
kvsname=kvs_25318_1_247201886_n06 key=-bcast-1-0
[proxy:1:0 at n00] PMI response: cmd=get_result rc=0 msg=success value=2F6465762F73686D2F6D706963685F736861725F746D70743631524334
[proxy:1:0 at n00] got pmi command (from 6): get
kvsname=kvs_25318_1_247201886_n06 key=-bcast-1-0
[proxy:1:0 at n00] PMI response: cmd=get_result rc=0 msg=success value=2F6465762F73686D2F6D706963685F736861725F746D70743631524334
[proxy:1:0 at n00] got pmi command (from 9): get
kvsname=kvs_25318_1_247201886_n06 key=-bcast-1-0
[proxy:1:0 at n00] PMI response: cmd=get_result rc=0 msg=success value=2F6465762F73686D2F6D706963685F736861725F746D70743631524334
[proxy:1:0 at n00] got pmi command (from 4): put
kvsname=kvs_25318_1_247201886_n06 key=-allgather-shm-1-0 value=02008269C0A8020A0000000000000000
[proxy:1:0 at n00] cached command: -allgather-shm-1-0=02008269C0A8020A0000000000000000
[proxy:1:0 at n00] PMI response: cmd=put_result rc=0 msg=success
[proxy:1:0 at n00] got pmi command (from 5): put
kvsname=kvs_25318_1_247201886_n06 key=-allgather-shm-1-1 value=02009BB1C0A8020A0000000000000000
[proxy:1:0 at n00] cached command: -allgather-shm-1-1=02009BB1C0A8020A0000000000000000
[proxy:1:0 at n00] PMI response: cmd=put_result rc=0 msg=success
[proxy:1:0 at n00] got pmi command (from 6): put
kvsname=kvs_25318_1_247201886_n06 key=-allgather-shm-1-2 value=0200848BC0A8020A0000000000000000
[proxy:1:0 at n00] cached command: -allgather-shm-1-2=0200848BC0A8020A0000000000000000
[proxy:1:0 at n00] PMI response: cmd=put_result rc=0 msg=success
[proxy:1:0 at n00] got pmi command (from 9): put
kvsname=kvs_25318_1_247201886_n06 key=-allgather-shm-1-3 value=02008E1FC0A8020A0000000000000000
[proxy:1:0 at n00] cached command: -allgather-shm-1-3=02008E1FC0A8020A0000000000000000
[proxy:1:0 at n00] PMI response: cmd=put_result rc=0 msg=success
[proxy:1:0 at n00] got pmi command (from 4): barrier_in
[proxy:1:0 at n00] got pmi command (from 5): barrier_in
[proxy:1:0 at n00] got pmi command (from 6): barrier_in
[mpiexec at n06] [pgid: 1] got PMI command: cmd=put -allgather-shm-1-0=02008269C0A8020A0000000000000000 -allgather-shm-1-1=02009BB1C0A8020A0000000000000000 -allgather-shm-1-2=0200848BC0A8020A0000000000000000 -allgather-shm-1-3=02008E1FC0A8020A0000000000000000
[proxy:1:0 at n00] got pmi command (from 9): barrier_in
[proxy:1:0 at n00] flushing 4 put command(s) out
[proxy:1:0 at n00] forwarding command (cmd=put -allgather-shm-1-0=02008269C0A8020A0000000000000000 -allgather-shm-1-1=02009BB1C0A8020A0000000000000000 -allgather-shm-1-2=0200848BC0A8020A0000000000000000 -allgather-shm-1-3=02008E1FC0A8020A0000000000000000) upstream
[proxy:1:0 at n00] forwarding command (cmd=barrier_in) upstream
[mpiexec at n06] [pgid: 1] got PMI command: cmd=barrier_in
[mpiexec at n06] PMI response to fd 9 pid 9: cmd=keyval_cache -allgather-shm-1-0=02008269C0A8020A0000000000000000 -allgather-shm-1-1=02009BB1C0A8020A0000000000000000 -allgather-shm-1-2=0200848BC0A8020A0000000000000000 -allgather-shm-1-3=02008E1FC0A8020A0000000000000000
[mpiexec at n06] PMI response to fd 9 pid 9: cmd=barrier_out
[proxy:1:0 at n00] PMI response: cmd=barrier_out
[proxy:1:0 at n00] PMI response: cmd=barrier_out
[proxy:1:0 at n00] PMI response: cmd=barrier_out
[proxy:1:0 at n00] PMI response: cmd=barrier_out
[proxy:1:0 at n00] got pmi command (from 4): get
kvsname=kvs_25318_1_247201886_n06 key=-allgather-shm-1-0
[proxy:1:0 at n00] PMI response: cmd=get_result rc=0 msg=success value=02008269C0A8020A0000000000000000
[proxy:1:0 at n00] got pmi command (from 5): get
kvsname=kvs_25318_1_247201886_n06 key=-allgather-shm-1-1
[proxy:1:0 at n00] PMI response: cmd=get_result rc=0 msg=success value=02009BB1C0A8020A0000000000000000
[proxy:1:0 at n00] got pmi command (from 6): get
kvsname=kvs_25318_1_247201886_n06 key=-allgather-shm-1-2
[proxy:1:0 at n00] PMI response: cmd=get_result rc=0 msg=success value=0200848BC0A8020A0000000000000000
[proxy:1:0 at n00] got pmi command (from 9): get
kvsname=kvs_25318_1_247201886_n06 key=-allgather-shm-1-3
[proxy:1:0 at n00] PMI response: cmd=get_result rc=0 msg=success value=02008E1FC0A8020A0000000000000000
[proxy:1:0 at n00] got pmi command (from 4): get
kvsname=kvs_25318_1_247201886_n06 key=PARENT_ROOT_PORT_NAME
[proxy:1:0 at n00] PMI response: cmd=get_result rc=0 msg=success value=tag#0$connentry#020087B3C0A802100000000000000000$
[proxy:1:0 at n00] got pmi command (from 6): get
kvsname=kvs_25318_1_247201886_n06 key=PARENT_ROOT_PORT_NAME
[proxy:1:0 at n00] PMI response: cmd=get_result rc=0 msg=success value=tag#0$connentry#020087B3C0A802100000000000000000$
[proxy:1:0 at n00] got pmi command (from 9): get
kvsname=kvs_25318_1_247201886_n06 key=PARENT_ROOT_PORT_NAME
[proxy:1:0 at n00] PMI response: cmd=get_result rc=0 msg=success value=tag#0$connentry#020087B3C0A802100000000000000000$
[proxy:1:0 at n00] got pmi command (from 5): get
kvsname=kvs_25318_1_247201886_n06 key=PARENT_ROOT_PORT_NAME
[proxy:1:0 at n00] PMI response: cmd=get_result rc=0 msg=success value=tag#0$connentry#020087B3C0A802100000000000000000$
HIJOS
A 0 con Grp=1
A 2 con Grp=1
A 1 con Grp=1
A 3 con Grp=1
A 0 con Grp=0
A 1 con Grp=0
A 2 con Grp=0
A 3 con Grp=0
[proxy:0:0 at n06] got pmi command (from 0): finalize
[proxy:0:0 at n06] PMI response: cmd=finalize_ack
[proxy:0:0 at n06] got pmi command (from 6): finalize
[proxy:0:0 at n06] PMI response: cmd=finalize_ack
[proxy:0:0 at n06] got pmi command (from 8): finalize
[proxy:0:0 at n06] PMI response: cmd=finalize_ack
[proxy:1:0 at n00] got pmi command (from 4): finalize
[proxy:1:0 at n00] PMI response: cmd=finalize_ack
[proxy:0:0 at n06] got pmi command (from 13): finalize
[proxy:0:0 at n06] PMI response: cmd=finalize_ack
[proxy:1:0 at n00] got pmi command (from 5): finalize
[proxy:1:0 at n00] PMI response: cmd=finalize_ack
B 0 con Grp=0
[proxy:1:0 at n00] got pmi command (from 6): finalize
[proxy:1:0 at n00] PMI response: cmd=finalize_ack
B 3 con Grp=0
[proxy:1:0 at n00] got pmi command (from 9): finalize
[proxy:1:0 at n00] PMI response: cmd=finalize_ack
B 1 con Grp=0
B 0 con Grp=1
B 1 con Grp=1
B 2 con Grp=0
B 2 con Grp=1
B 3 con Grp=1
More information about the discuss
mailing list