[mpich-discuss] RMA calls with MPI Datatypes using structs

[Palmer, Bruce J]

Attached is my test code. I've been trying to emulate the gather-scatter semantics in Global Arrays using MPI RMA calls. The top of the file has a bunch of parameters that can be used to configure different protocols for implementing GA-style puts and gets. The failure I'm seeing seems to be independent of the choice of protocol. The parameter USE_STRUCTS can be used to switch between structs and subarrays. Let me know if you have more questions.

Bruce



-----Original Message-----
From: Rob Latham [mailto:robl at mcs.anl.gov] 
Sent: Wednesday, May 25, 2016 9:07 AM
To: discuss at mpich.org
Subject: Re: [mpich-discuss] RMA calls with MPI Datatypes using structs



On 05/25/2016 10:40 AM, Palmer, Bruce J wrote:
> Hi,
>
> I've been trying to implement code using MPI_Get and MPI_Put that move 
> around data using MPI_Datatypes. If I create the data type using 
> MPI_Type_create_subarray, everything works fine. If I create the data 
> type using MPI_Type_create_struct then I get a failure if the amount 
> of data I'm moving around becomes large.  At the moment the crossover 
> point is moving a total of a million doubles on 8 cores (125000 
> doubles per core), which doesn't seem to be excessively large. The 
> error I'm seeing on a Linux cluster using and Infiniband interconnect 
> is
>
> Assertion failed in file
> ./src/mpid/ch3/channels/nemesis/include/mpid_nem_inline.h at line 406
>
> : MPID_NEM_MPICH_DATA_LEN - buf_offset >= ext_hdr_sz
>
> internal ABORT - process 1
>
> Note that the same code runs fine if the data type is created using 
> the subarray call. It looks like some internal limit is being 
> exceeded, but if so, I think the threshold may be set a bit on the low 
> side. Does anyone know more about this?

Doesn't look like something we've seen before.   Can you send us the 
working subarray and non-working struct datatype?
_______________________________________________
discuss mailing list     discuss at mpich.org
To manage subscription options or unsubscribe:
https://lists.mpich.org/mailman/listinfo/discuss
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: mpi_gatscat_test.c
URL: <http://lists.mpich.org/pipermail/discuss/attachments/20160525/99a9736a/attachment.c>
-------------- next part --------------
_______________________________________________
discuss mailing list     discuss at mpich.org
To manage subscription options or unsubscribe:
https://lists.mpich.org/mailman/listinfo/discuss