[mpich-discuss] MPICH 3.2 on BlueGene/Q

[Rob Latham]

On 01/08/2016 04:35 AM, Dominic Chien wrote:
> Hi,
>
> I have been trying to install MPICH 3 on a BlueGene/Q system to evaluate the MPI-3 functionalities that are missed in the IBM supported MPI, and I also want to use it to build ARMCI-MPI3, GA and NWChem on this system.
>
> Based on the instruction prepared by Argonne National Lab (https://wiki.mpich.org/mpich/index.php/BGQ),  I managed to build the MPICH 3.1rc4 on this system. Although I am still evaluating the stability and correctness of this version on BGQ, it seem promising so far.
>
> However, I got problem to build MPICH 3.2 with the same instruction, can anyone shade me some light to let me get it compile?

Yes, the MPICH developers made some changes to the device interface and 
the PAMI device (which blue gene uses) did not get those updates.

What I've done is forked MPICH from the last working version (somewhere 
around last July) and then cherry-picked a dozen or so "important" 
patches (where 'important' meant mostly I/O related, since that's my 
primary area of focus)

https://xgitlab.cels.anl.gov/robl/MPICH-BlueGene

(I just now noticed that our README doesn't look so good.  I'll clean it 
up but wanted you to get an answer as soon as possible)

If there are 3.2 features you'd like to cherry pick into this fork, and 
they don't touch the device interface, or depend on patches that do, 
then we can definitely do that.

If you are working on Argonne's Blue Gene, I built and installed this 
fork to /soft/libraries/unsupported/mpich

Let me know if you run into any problems.

==rob

> Regards,
> Dominic
>
> Here is the environment variables
> ====================================================================================
> export CC=/opt/ibmcmp/vac/bg/12.1/bin/bgxlc_r
> export CXX=/opt/ibmcmp/vacpp/bg/12.1/bin/bgxlC_r
> export F77=/opt/ibmcmp/xlf/bg/14.1/bin/bgxlf_r
> export FC=/opt/ibmcmp/xlf/bg/14.1/bin/bgxlf90_r
> export AR=/bgsys/drivers/V1R2M0/ppc64/gnu-linux/powerpc64-bgq-linux/bin/ar
> export LD=/bgsys/drivers/V1R2M0/ppc64/gnu-linux/powerpc64-bgq-linux/bin/ld
> export RANLIB=/bgsys/drivers/V1R2M0/ppc64/gnu-linux/powerpc64-bgq-linux/bin/ranlib
> export MPICHLIB_CXXFLAGS="-qhot -qinline=800 -qflag=i:i -qsaveopt -qsuppress=1506-236"
> export MPICHLIB_CFLAGS=${MPICHLIB_CXXFLAGS}
> export MPICHLIB_FFLAGS=${MPICHLIB_CXXFLAGS}
> export MPICHLIB_F90FLAGS=${MPICHLIB_CXXFLAGS}
> ====================================================================================
>
> Here is the configure
> ====================================================================================
> ../configure --host=powerpc64-bgq-linux --with-device=pamid --with-file-system=gpfs:BGQ --with-bgq-install-dir=/bgsys/drivers/V1R2M0/ppc64 --with-pami=/bgsys/drivers/V1R2M0/ppc64/comm/sys --with-pami-include=/bgsys/drivers/V1R2M0/ppc64/comm/sys/include --with-pami-lib=/bgsys/drivers/V1R2M0/ppc64/comm/sys/lib --disable-wrapper-rpath --enable-fast=nochkmsg,notiming,O3 --with-assert-level=0 --disable-error-messages --disable-debuginfo --enable-thread-cs=per-object --with-atomic-primitives --enable-handle-allocation=tls --enable-refcount=lock-free --disable-predefined-refcount --with-cross-file=src/mpid/pamid/cross/bgq8 --prefix=/scratch/home/chiensh/apps/mpich/3.2.rc2
> ====================================================================================
>
> Here is the make log
> ====================================================================================
> ...
>   CC       src/mpi/attr/lib_libmpi_la-attr_delete.lo
> "/scratch/home/chiensh/mpich/mpich-3.2rc2/src/mpid/pamid/include/mpidi_thread.h", line 64.9: 1506-358 (I) "MPIU_THREAD_CS_ENTER" is defined on line 80 of ../src/util/thread/mpiu_thread_multiple.h.
> "/scratch/home/chiensh/mpich/mpich-3.2rc2/src/mpid/pamid/include/mpidi_thread.h", line 65.9: 1506-358 (I) "MPIU_THREAD_CS_EXIT" is defined on line 81 of ../src/util/thread/mpiu_thread_multiple.h.
> "/scratch/home/chiensh/mpich/mpich-3.2rc2/src/mpid/pamid/include/mpidi_thread.h", line 66.9: 1506-358 (I) "MPIU_THREAD_CS_YIELD" is defined on line 82 of ../src/util/thread/mpiu_thread_multiple.h.
> "../src/include/mpiimpl.h", line 1184.5: 1506-046 (S) Syntax error.
> "../src/include/mpiimpl.h", line 1481.5: 1506-046 (S) Syntax error.
> "../src/include/mpiimpl.h", line 1636.5: 1506-046 (S) Syntax error.
> "../src/include/mpiimpl.h", line 2785.5: 1506-343 (S) Redeclaration of MPID_Send differs from previous declaration on line 2760 of "../src/include/mpiimpl.h".
> "../src/include/mpiimpl.h", line 2785.5: 1506-377 (I) The type "int" of parameter 2 differs from the previous type "long".
> "../src/include/mpiimpl.h", line 2884.5: 1506-343 (S) Redeclaration of MPID_Isend differs from previous declaration on line 2865 of "../src/include/mpiimpl.h".
> "../src/include/mpiimpl.h", line 2884.5: 1506-377 (I) The type "int" of parameter 2 differs from the previous type "long".
> "../src/include/mpitimpl.h", line 245.5: 1506-046 (S) Syntax error.
> "../src/include/mpitimpl.h", line 900.40: 1506-022 (S) "total" is not a member of "struct {...}".
> "/scratch/home/chiensh/mpich/mpich-3.2rc2/src/mpid/pamid/include/../src/mpid_request.h", line 156.21: 1506-022 (S) "cc" is not a member of "struct MPID_Request".
> "/scratch/home/chiensh/mpich/mpich-3.2rc2/src/mpid/pamid/include/../src/mpid_request.h", line 157.8: 1506-022 (S) "cc_ptr" is not a member of "struct MPID_Request".
> "/scratch/home/chiensh/mpich/mpich-3.2rc2/src/mpid/pamid/include/../src/mpid_request.h", line 157.23: 1506-022 (S) "cc" is not a member of "struct MPID_Request".
> "/scratch/home/chiensh/mpich/mpich-3.2rc2/src/mpid/pamid/include/../src/mpid_request.h", line 283.62: 1506-099 (S) Unexpected argument.
> "/scratch/home/chiensh/mpich/mpich-3.2rc2/src/mpid/pamid/include/../src/mpid_request.h", line 326.23: 1506-022 (S) "cc_ptr" is not a member of "struct MPID_Request".
> "/scratch/home/chiensh/mpich/mpich-3.2rc2/src/mpid/pamid/include/../src/mpid_request.h", line 341.23: 1506-022 (S) "cc_ptr" is not a member of "struct MPID_Request".
> "/scratch/home/chiensh/mpich/mpich-3.2rc2/src/mpid/pamid/include/mpidpost.h", line 56.44: 1506-022 (S) "vcr" is not a member of "struct MPID_Comm".
> "/scratch/home/chiensh/mpich/mpich-3.2rc2/src/mpid/pamid/include/mpidpost.h", line 83.43: 1506-022 (S) "vcrt" is not a member of "struct MPID_Comm".
> "/scratch/home/chiensh/mpich/mpich-3.2rc2/src/mpid/pamid/include/mpidpost.h", line 84.37: 1506-022 (S) "vcrt" is not a member of "struct MPID_Comm".
> "/scratch/home/chiensh/mpich/mpich-3.2rc2/src/mpid/pamid/include/mpidpost.h", line 84.57: 1506-022 (S) "vcr" is not a member of "struct MPID_Comm".
> "/scratch/home/chiensh/mpich/mpich-3.2rc2/src/mpid/pamid/include/mpidpost.h", line 91.42: 1506-022 (S) "vcr" is not a member of "struct MPID_Comm".
> "/scratch/home/chiensh/mpich/mpich-3.2rc2/src/mpid/pamid/include/mpidpost.h", line 92.41: 1506-022 (S) "vcr" is not a member of "struct MPID_Comm".
> "../src/mpi/attr/attr_delete.c", line 65.26: 1506-045 (S) Undeclared identifier GLOBAL.
> make[2]: *** [src/mpi/attr/lib_libmpi_la-attr_delete.lo] Error 1
> make[2]: Leaving directory `/scratch/home/chiensh/mpich/mpich-3.2rc2/build'
> make[1]: *** [all-recursive] Error 1
> make[1]: Leaving directory `/scratch/home/chiensh/mpich/mpich-3.2rc2/build'
> make: *** [all] Error 2
> ====================================================================================
> _______________________________________________
> discuss mailing list     discuss at mpich.org
> To manage subscription options or unsubscribe:
> https://lists.mpich.org/mailman/listinfo/discuss
>
_______________________________________________
discuss mailing list     discuss at mpich.org
To manage subscription options or unsubscribe:
https://lists.mpich.org/mailman/listinfo/discuss