[mpich-discuss] Building MPE for use with OpenMPI
William Gropp
wgropp at illinois.edu
Wed Dec 2 16:49:10 CST 2015
I’ve fixed this by testing for the bug in using const in the removed routines; in the process of testing with Open MPI, I found a few other bugs that I also fixed. I’ve pushed the fixes to mpe2.git, and created a new tarball, which is installed at the ftp site (with link to mpe2.tar.gz).
Bill
William Gropp
Director, Parallel Computing Institute
Thomas M. Siebel Chair in Computer Science
Chief Scientist, NCSA
University of Illinois Urbana-Champaign
On Dec 2, 2015, at 9:17 AM, Jeff Hammond <jeff.science at gmail.com> wrote:
> But you are right that a configure test could be useful to make MPE work with MPICH releases that have the unnecessary const attributes in MPI_Type_hindexed, MPI_Type_struct, MPI_Address, PMPI_Type_hindexed, PMPI_Type_struct, PMPI_Address.
>
> Best,
>
> Jeff
>
> On Wed, Dec 2, 2015 at 7:16 AM, Jeff Hammond <jeff.science at gmail.com> wrote:
> Hi Bill,
>
> My original idea was wrong. Since those deprecated MPI-2 functions should never have const in the declaration, the right solution is to remove MPE_CONST from them. The macro logic is fine as-is as long as an MPI implementation only adds const to MPI-3 functions.
>
> The attached patch implements the aforementioned change. Of course, it must be coupled to my patch to MPICH for the function signatures to align.
>
> I don't have write privileges on the MPE repo, not that it is critical.
>
> Best,
>
> Jeff
>
> On Wed, Dec 2, 2015 at 7:08 AM, William Gropp <wgropp at illinois.edu> wrote:
> It sounds like this one needs a special case for the current MPICH bug. That isn’t hard and I can do that, Jeff, if you don’t want to. Let me know. It will need a configure test for the suspect cases.
>
> Bill
>
> William Gropp
> Director, Parallel Computing Institute
> Thomas M. Siebel Chair in Computer Science
> Chief Scientist, NCSA
> University of Illinois Urbana-Champaign
>
>
>
>
>
> On Dec 1, 2015, at 10:42 PM, Jeff Hammond <jeff.science at gmail.com> wrote:
>
>> Ok, I didn't understand your issue before. I thought your issue was with MPICH not agreeing with OpenMPI. That needed to be fixed anyways, so it's good that your query prompted it, but I'm sorry it wasn't what you needed.
>>
>> MPE already has MPE_CONST, but it will be "const" whenever MPI_VERSION>=3, which is what late-model MPICH will give you. The solution is probably to the logic below to allow for manual override, rather than OR-ing on MPI_VERSION, which cannot be overridden.
>>
>> #if defined(HAVE_CONST_IN_MPI_DECL) || MPI_VERSION >= 3
>> #define MPE_CONST const
>> #else
>> #define MPE_CONST
>> #endif
>>
>> I'll try to work up a patch later.
>>
>> As for configure, the documentation is rightly written for the user who downloads a release tarball, not the git repo. So yes, you should generate it with autogen.sh.
>>
>> Jeff
>>
>> On Tue, Dec 1, 2015 at 7:04 PM, Schilling, Herbert W. (GRC-VE00) <hschilling at nasa.gov> wrote:
>> >
>> > Hi Jeff,
>> >
>> > Thanks. Unfortunately, that is basically the same source code from
>> >
>> > ftp://ftp.mcs.anl.gov/pub/mpi/mpe/mpe2.tar.gz
>> >
>> > where I had the issues with the conflicting declarations as Pavan talked about.
>> >
>> > I could remove the deprecated routines as most of the problems are caused by them, but it looks like some non-deprecated routines also have the const issue, e.g.
>> >
>> > visual_mess.c:560: error: conflicting types for 'MPI_Issend'
>> > /u/hschilli/local/include/mpi.h:1543: error: previous declaration of 'MPI_Issend' was here
>> >
>> >
>> > BTW, that repo for MPE doesn't appear to have a configure file. I know I could create it with autogen.sh but the INSTALL file doesn't say that. It just says run configure. Might be confusing for some people, I think!
>> >
>> > Thanks for everyone's help
>> >
>> >
>> >
>> >
>> >
>> > On Dec 1, 2015, at 2:03 PM, Jeff Hammond <jeff.science at gmail.com>
>> > wrote:
>> >
>> > MPE lives here now: http://git.mpich.org/mpe.git/
>> >
>> > Jeff
>> >
>> > On Tue, Dec 1, 2015 at 10:15 AM, Schilling, Herbert W. (GRC-VE00) <hschilling at nasa.gov> wrote:
>> >>
>> >> Hi Jeff,
>> >>
>> >> Thanks. Pardon my ignorance but I I wanted to just use MPE with an existing OpenMPI installation and it's not clear to me ( even after searching and reading ), how to do that from the full MPICH code base at that repo. I can't really even find MPE in that code at all. What am I missing?
>> >>
>> >> Thanks
>> >>
>> >> On Dec 1, 2015, at 10:03 AM, Jeff Hammond <jeff.science at gmail.com>
>> >> wrote:
>> >>
>> >> Herbert:
>> >>
>> >> Please try https://github.com/jeffhammond/mpich/tree/hammond-mpi2-deprecated-functions-noconst and let me know if you see any problems.
>> >>
>> >> Best,
>> >>
>> >> Jeff
>> >>
>> >> On Tue, Dec 1, 2015 at 5:46 AM, Schilling, Herbert W. (GRC-VE00) <hschilling at nasa.gov> wrote:
>> >>>
>> >>> Hi Jeff and Pavan,
>> >>>
>> >>> Thanks for all the help on this. Jeff, when you have the fix up there, I can give it a shot!
>> >>>
>> >>>
>> >>> On Dec 1, 2015, at 1:15 AM, Jeff Hammond <jeff.science at gmail.com>
>> >>> wrote:
>> >>>
>> >>> Facepalm. Indeed, changing the header but not the implementation was pretty dumb on my part. I'll fix that tomorrow.
>> >>>
>> >>> I have no opinion on the ABI situation, particularly since I know there are other folks that Intel that do.
>> >>>
>> >>> Jeff
>> >>>
>> >>> On Mon, Nov 30, 2015 at 9:00 PM, Balaji, Pavan <balaji at anl.gov> wrote:
>> >>>>
>> >>>>
>> >>>> Thanks, Jeff. The actual function definitions should be modified as well, which your patch seems to miss.
>> >>>>
>> >>>> But I think the potentially time consuming part is to get buy-in from all the ABI consortium folks for this change. It doesn't exactly break ABI for applications, but will be an annoying change for tools. Ken is looking into this.
>> >>>>
>> >>>> -- Pavan
>> >>>>
>> >>>> > On Nov 30, 2015, at 10:54 PM, Jeff Hammond <jeff.science at gmail.com> wrote:
>> >>>> >
>> >>>> > Pavan:
>> >>>> >
>> >>>> > See hammond-mpi2-deprecated-functions-noconst branch on mpich-review. I found three functions where this change should be made. I do not know if that is the full list, so someone should verify.
>> >>>> >
>> >>>> > Best,
>> >>>> >
>> >>>> > Jeff
>> >>>> >
>> >>>> > On Mon, Nov 30, 2015 at 8:08 PM, Balaji, Pavan <balaji at anl.gov> wrote:
>> >>>> >
>> >>>> > MPI_Type_struct is not a part of the MPI standard anymore and is replaced by MPI_Type_create_struct. Both MPICH and Open MPI decided to retain it, but followed different directions for it. MPICH decided to follow the same convention as MPI_Type_create_struct with the const additions. Open MPI decided to keep the old prototypes as is.
>> >>>> >
>> >>>> > Unfortunately, MPE got stuck in between the two. The best approach would be to simply remove those functions from MPE and this problem just goes away. Would you be able to do that and contribute a patch?
>> >>>> >
>> >>>> > More generally, I think MPICH was overzealous in trying to constify prototypes for the deleted functions. It should have just left them as they are, like what Open MPI did. We'll figure out what to do about that internally, but that's not going to be a quick change because of ABI and other such stuff.
>> >>>> >
>> >>>> > Thanks,
>> >>>> >
>> >>>> > -- Pavan
>> >>>> >
>> >>>> > > On Nov 30, 2015, at 8:14 PM, Schilling, Herbert W. (GRC-VE00) <hschilling at nasa.gov> wrote:
>> >>>> > >
>> >>>> > > Hi,
>> >>>> > >
>> >>>> > > I am trying to build MPE for use with OpenMPI. I built from source Version 1.10.1 of OpenMPI.
>> >>>> > >
>> >>>> > > I am using this version of MPE ftp://ftp.mcs.anl.gov/pub/mpi/mpe/mpe2.tar.gz
>> >>>> > >
>> >>>> > > Should this work fine? I followed the instructions to build like this:
>> >>>> > >
>> >>>> > > ${MPE_SRC_DIR}/configure MPI_CC=${OPENMPI_INSTALL_DIR}/bin/mpicc \
>> >>>> > > CC=<C_COMPILER_used_to_build_OpenMPI> \
>> >>>> > > MPI_F77=${OPENMPI_INSTALL_DIR}/bin/mpif77 \
>> >>>> > > F77=<FORTRAN_COMPILER_used_to_build_OpenMPI> \
>> >>>> > > --with-java2=/usr/local/j2sdk-1.4.2
>> >>>> > > --prefix=<MPE_INSTALL_DIR>
>> >>>> > > make
>> >>>> > > make install
>> >>>> > >
>> >>>> > >
>> >>>> > >
>> >>>> > >
>> >>>> > > I keep running into problems with conflicting declarations
>> >>>> > >
>> >>>> > >
>> >>>> > > log_mpi_core.c:5611: error: conflicting types for 'MPI_Type_struct'
>> >>>> > > /u/hschilli/local/include/mpi.h:1796: error: previous declaration of 'MPI_Type_struct' was here
>> >>>> > >
>> >>>> > >
>> >>>> > > I think the issue is that MPE's declaration uses some const qualifiers while OpenMPI does not.
>> >>>> > >
>> >>>> > >
>> >>>> > >
>> >>>> > > I tried hacking my way through this with the macro
>> >>>> > >
>> >>>> > > HAVE_CONST_IN_MPI_DECL
>> >>>> > > and MPE_CONST but no luck so far.
>> >>>> > >
>> >>>> > >
>> >>>> > >
>> >>>> > > What am I doing wrong?
>> >>>> > >
>> >>>> > >
>> >>>> > > Thanks in advance.
>> >>>> > >
>> >>>> > > ---------
>> >>>> > > Herb Schilling
>> >>>> > > Scientific Applications and Visualization Team
>> >>>> > > http://ocio.grc.nasa.gov/gvis
>> >>>> > > Office of the Chief Information Officer
>> >>>> > > NASA Glenn Research Center
>> >>>> > > hschilling at nasa.gov
>> >>>> > >
>> >>>> > >
>> >>>> > >
>> >>>> > >
>> >>>> > > _______________________________________________
>> >>>> > > discuss mailing list discuss at mpich.org
>> >>>> > > To manage subscription options or unsubscribe:
>> >>>> > > https://lists.mpich.org/mailman/listinfo/discuss
>> >>>> >
>> >>>> > _______________________________________________
>> >>>> > discuss mailing list discuss at mpich.org
>> >>>> > To manage subscription options or unsubscribe:
>> >>>> > https://lists.mpich.org/mailman/listinfo/discuss
>> >>>> >
>> >>>> >
>> >>>> >
>> >>>> > --
>> >>>> > Jeff Hammond
>> >>>> > jeff.science at gmail.com
>> >>>> > http://jeffhammond.github.io/
>> >>>> > _______________________________________________
>> >>>> > discuss mailing list discuss at mpich.org
>> >>>> > To manage subscription options or unsubscribe:
>> >>>> > https://lists.mpich.org/mailman/listinfo/discuss
>> >>>>
>> >>>> _______________________________________________
>> >>>> discuss mailing list discuss at mpich.org
>> >>>> To manage subscription options or unsubscribe:
>> >>>> https://lists.mpich.org/mailman/listinfo/discuss
>> >>>
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> Jeff Hammond
>> >>> jeff.science at gmail.com
>> >>> http://jeffhammond.github.io/
>> >>> _______________________________________________
>> >>> discuss mailing list discuss at mpich.org
>> >>> To manage subscription options or unsubscribe:
>> >>> https://lists.mpich.org/mailman/listinfo/discuss
>> >>>
>> >>>
>> >>> ---------
>> >>> Herb Schilling
>> >>> Scientific Applications and Visualization Team
>> >>> http://ocio.grc.nasa.gov/gvis
>> >>> Office of the Chief Information Officer
>> >>> NASA Glenn Research Center
>> >>> hschilling at nasa.gov
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>> _______________________________________________
>> >>> discuss mailing list discuss at mpich.org
>> >>> To manage subscription options or unsubscribe:
>> >>> https://lists.mpich.org/mailman/listinfo/discuss
>> >>
>> >>
>> >>
>> >>
>> >> --
>> >> Jeff Hammond
>> >> jeff.science at gmail.com
>> >> http://jeffhammond.github.io/
>> >> _______________________________________________
>> >> discuss mailing list discuss at mpich.org
>> >> To manage subscription options or unsubscribe:
>> >> https://lists.mpich.org/mailman/listinfo/discuss
>> >>
>> >>
>> >> ---------
>> >> Herb Schilling
>> >> Scientific Applications and Visualization Team
>> >> http://ocio.grc.nasa.gov/gvis
>> >> Office of the Chief Information Officer
>> >> NASA Glenn Research Center
>> >> hschilling at nasa.gov
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> _______________________________________________
>> >> discuss mailing list discuss at mpich.org
>> >> To manage subscription options or unsubscribe:
>> >> https://lists.mpich.org/mailman/listinfo/discuss
>> >
>> >
>> >
>> >
>> > --
>> > Jeff Hammond
>> > jeff.science at gmail.com
>> > http://jeffhammond.github.io/
>> > _______________________________________________
>> > discuss mailing list discuss at mpich.org
>> > To manage subscription options or unsubscribe:
>> > https://lists.mpich.org/mailman/listinfo/discuss
>> >
>> >
>> > ---------
>> > Herb Schilling
>> > Scientific Applications and Visualization Team
>> > http://ocio.grc.nasa.gov/gvis
>> > Office of the Chief Information Officer
>> > NASA Glenn Research Center
>> > hschilling at nasa.gov
>> >
>> >
>> >
>> >
>> >
>> > _______________________________________________
>> > discuss mailing list discuss at mpich.org
>> > To manage subscription options or unsubscribe:
>> > https://lists.mpich.org/mailman/listinfo/discuss
>>
>>
>>
>>
>> --
>> Jeff Hammond
>> jeff.science at gmail.com
>> http://jeffhammond.github.io/
>> _______________________________________________
>> discuss mailing list discuss at mpich.org
>> To manage subscription options or unsubscribe:
>> https://lists.mpich.org/mailman/listinfo/discuss
>
>
> _______________________________________________
> discuss mailing list discuss at mpich.org
> To manage subscription options or unsubscribe:
> https://lists.mpich.org/mailman/listinfo/discuss
>
>
>
> --
> Jeff Hammond
> jeff.science at gmail.com
> http://jeffhammond.github.io/
>
>
>
> --
> Jeff Hammond
> jeff.science at gmail.com
> http://jeffhammond.github.io/
> _______________________________________________
> discuss mailing list discuss at mpich.org
> To manage subscription options or unsubscribe:
> https://lists.mpich.org/mailman/listinfo/discuss
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mpich.org/pipermail/discuss/attachments/20151202/a9b9cc50/attachment.html>
-------------- next part --------------
_______________________________________________
discuss mailing list discuss at mpich.org
To manage subscription options or unsubscribe:
https://lists.mpich.org/mailman/listinfo/discuss
More information about the discuss
mailing list