[mpich-commits] [mpich] MPICH primary repository branch, master, updated. v3.2a2-118-gafb720d
Service Account
noreply at mpich.org
Fri Jan 23 15:17:32 CST 2015
This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "MPICH primary repository".
The branch, master has been updated
via afb720d01390fe76299f9a9a90a949316af70ec1 (commit)
via 4513e539639620d35614a3277474deba1314ebd1 (commit)
via 67b0e1e8ac5443cb8edd3622ef8f99503a0c3ac6 (commit)
from 93e816cc1fbd88109c55f9acf9b5ed6efc2a12d2 (commit)
Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.
- Log -----------------------------------------------------------------
http://git.mpich.org/mpich.git/commitdiff/afb720d01390fe76299f9a9a90a949316af70ec1
commit afb720d01390fe76299f9a9a90a949316af70ec1
Author: Rob Latham <robl at mcs.anl.gov>
Date: Tue Jan 20 10:53:55 2015 -0600
fix issue with small i/o and big datatypes
HDF5 folks reported a bug with ROMIO and one of their slightly-strange (but
100% legal) datatypes. git-bisect points to the "promote size of length"
change. Seems that MPICH does not like struct datatypes with zero-count
elements? Further investigation requred. This change (construct a simpler
datatype in more cases) is sufficient to help HDF5 move forward.
See #2221
Signed-off-by: Ken Raffenetti <raffenet at mcs.anl.gov>
diff --git a/src/mpi/romio/adio/common/utils.c b/src/mpi/romio/adio/common/utils.c
index b539187..eef48cb 100644
--- a/src/mpi/romio/adio/common/utils.c
+++ b/src/mpi/romio/adio/common/utils.c
@@ -63,17 +63,40 @@ int ADIOI_Type_create_hindexed_x(int count,
int i, ret;
MPI_Datatype *types;
int *blocklens;
+ int is_big=0;
types = ADIOI_Malloc(count*sizeof(MPI_Datatype));
blocklens = ADIOI_Malloc(count*sizeof(int));
+ /* squashing two loops into one.
+ * - Look in the array_of_blocklengths for any large values
+ * - convert MPI_Count items (if they are not too big) into int-sized items
+ * after this loop we will know if we can use MPI_type_hindexed or if we
+ * need a more complicated BigMPI-style struct-of-chunks.
+ *
+ * Why not use the struct-of-chunks in all cases? HDF5 reported a bug,
+ * which I have not yet precicesly nailed down, but appears to have
+ * something to do with struct-of-chunks when the chunks are small */
+
for(i=0; i<count; i++) {
- blocklens[i] = 1;
- type_create_contiguous_x(array_of_blocklengths[i], oldtype, &(types[i]));
+ if (array_of_blocklengths[i] > INT_MAX) {
+ blocklens[i] = 1;
+ is_big=1;
+ type_create_contiguous_x(array_of_blocklengths[i], oldtype, &(types[i]));
+ } else {
+ /* OK to cast: checked for "bigness" above */
+ blocklens[i] = (int)array_of_blocklengths[i];
+ MPI_Type_contiguous(blocklens[i], oldtype, &(types[i]));
+ }
}
- ret = MPI_Type_create_struct(count, blocklens, array_of_displacements,
- types, newtype);
+ if (is_big) {
+ ret = MPI_Type_create_struct(count, blocklens, array_of_displacements,
+ types, newtype);
+ } else {
+ ret = MPI_Type_hindexed(count, blocklens,
+ array_of_displacements, oldtype, newtype);
+ }
for (i=0; i< count; i++)
MPI_Type_free(&(types[i]));
ADIOI_Free(types);
http://git.mpich.org/mpich.git/commitdiff/4513e539639620d35614a3277474deba1314ebd1
commit 4513e539639620d35614a3277474deba1314ebd1
Author: Rob Latham <robl at mcs.anl.gov>
Date: Mon Jan 19 15:36:08 2015 -0600
clean up 64-to-32 bit casts
many many places where a 64 bit value is stored in a 32 bit value
Signed-off-by: Ken Raffenetti <raffenet at mcs.anl.gov>
diff --git a/src/mpi/romio/adio/common/ad_seek.c b/src/mpi/romio/adio/common/ad_seek.c
index d8bbe85..ceaf6f2 100644
--- a/src/mpi/romio/adio/common/ad_seek.c
+++ b/src/mpi/romio/adio/common/ad_seek.c
@@ -26,8 +26,8 @@ ADIO_Offset ADIOI_GEN_SeekIndividual(ADIO_File fd, ADIO_Offset offset,
ADIO_Offset n_etypes_in_filetype, n_filetypes, etype_in_filetype;
ADIO_Offset abs_off_in_filetype=0;
ADIO_Offset size_in_filetype, sum;
- MPI_Count filetype_size;
- int etype_size, filetype_is_contig;
+ MPI_Count filetype_size, etype_size;
+ int filetype_is_contig;
MPI_Aint filetype_extent;
ADIOI_UNREFERENCED_ARG(whence);
@@ -35,7 +35,7 @@ ADIO_Offset ADIOI_GEN_SeekIndividual(ADIO_File fd, ADIO_Offset offset,
ADIOI_Datatype_iscontig(fd->filetype, &filetype_is_contig);
etype_size = fd->etype_size;
- if (filetype_is_contig) off = fd->disp + (ADIO_Offset)etype_size * offset;
+ if (filetype_is_contig) off = fd->disp + etype_size * offset;
else {
flat_file = ADIOI_Flatlist;
while (flat_file->type != fd->filetype) flat_file = flat_file->next;
diff --git a/src/mpi/romio/adio/common/utils.c b/src/mpi/romio/adio/common/utils.c
index 9814220..b539187 100644
--- a/src/mpi/romio/adio/common/utils.c
+++ b/src/mpi/romio/adio/common/utils.c
@@ -25,8 +25,12 @@ static int type_create_contiguous_x(MPI_Count count,
int blocklens[2];
MPI_Datatype types[2];
- MPI_Count c = count/INT_MAX;
- MPI_Count r = count%INT_MAX;
+ /* truly stupendously large counts will overflow an integer with this math,
+ * but that is a problem for a few decades from now. Sorry, few decades
+ * from now! */
+ ADIOI_Assert(count/INT_MAX == (int)(count/INT_MAX));
+ int c = (int)(count/INT_MAX); /* OK to cast until 'count' is 256 bits */
+ int r = count%INT_MAX;
MPI_Type_vector(c, INT_MAX, INT_MAX, oldtype, &chunks);
MPI_Type_contiguous(r, oldtype, &remainder);
http://git.mpich.org/mpich.git/commitdiff/67b0e1e8ac5443cb8edd3622ef8f99503a0c3ac6
commit 67b0e1e8ac5443cb8edd3622ef8f99503a0c3ac6
Author: Rob Latham <robl at mcs.anl.gov>
Date: Mon Jan 19 14:12:03 2015 -0600
Tweak MPIDU_Datatype_debug output
- bump up subtypes from 3 to 6. The limit is arbitrary. I am trying to
figure out a type with 4 sub-types.
- split up indexed/hindexed lists onto separate lines. MPICH debug
output format adds its own newlines, but we have to clean out MPICH's
extra debug output anyway: joining a few lines isn't that much more
work.
- output a name of the digraph that graphviz can actually parse.
Signed-off-by: Ken Raffenetti <raffenet at mcs.anl.gov>
diff --git a/src/mpid/common/datatype/mpid_type_debug.c b/src/mpid/common/datatype/mpid_type_debug.c
index 2489dbd..25a81ed 100644
--- a/src/mpid/common/datatype/mpid_type_debug.c
+++ b/src/mpid/common/datatype/mpid_type_debug.c
@@ -26,6 +26,9 @@ void MPIDI_Dataloop_dot_printf(MPID_Dataloop *loop_p, int depth, int header);
void MPIDI_Datatype_contents_printf(MPI_Datatype type, int depth, int acount);
static char *MPIDI_Datatype_depth_spacing(int depth) ATTRIBUTE((unused));
+#define NR_TYPE_CUTOFF 6 /* Number of types to display before truncating
+ output. 6 picked as arbitrary cutoff */
+
/* note: this isn't really "error handling" per se, but leave these comments
* because Bill uses them for coverage analysis.
*/
@@ -65,7 +68,8 @@ void MPIDI_Dataloop_dot_printf(MPID_Dataloop *loop_p,
if (header) {
MPIU_DBG_OUT_FMT(DATATYPE,(MPIU_DBG_FDEST,
- "digraph %p { {", loop_p));
+ /* graphviz does not like the 0xNNN format */
+ "digraph %lld { {", (long long int)loop_p));
}
switch (loop_p->kind & DLOOP_KIND_MASK) {
@@ -94,28 +98,27 @@ void MPIDI_Dataloop_dot_printf(MPID_Dataloop *loop_p,
(int) loop_p->loop_params.i_t.count,
(int) loop_p->loop_params.i_t.total_blocks));
- /* 3 picked as arbitrary cutoff */
- for (i=0; i < 3 && i < loop_p->loop_params.i_t.count; i++) {
+ for (i=0; i < NR_TYPE_CUTOFF && i < loop_p->loop_params.i_t.count; i++) {
if (i + 1 < loop_p->loop_params.i_t.count) {
/* more regions after this one */
MPIU_DBG_OUT_FMT(DATATYPE,(MPIU_DBG_FDEST,
- "(" MPI_AINT_FMT_DEC_SPEC ", %d), ",
+ "\\n(" MPI_AINT_FMT_DEC_SPEC ", %d), ",
(MPI_Aint) loop_p->loop_params.i_t.offset_array[i],
(int) loop_p->loop_params.i_t.blocksize_array[i]));
}
else {
MPIU_DBG_OUT_FMT(DATATYPE,(MPIU_DBG_FDEST,
- "(" MPI_AINT_FMT_DEC_SPEC ", %d); ",
+ "\\n(" MPI_AINT_FMT_DEC_SPEC ", %d); ",
(MPI_Aint) loop_p->loop_params.i_t.offset_array[i],
(int) loop_p->loop_params.i_t.blocksize_array[i]));
}
}
if (i < loop_p->loop_params.i_t.count) {
- MPIU_DBG_OUT(DATATYPE,"...; ");
+ MPIU_DBG_OUT(DATATYPE,"\\n...; ");
}
MPIU_DBG_OUT_FMT(DATATYPE,(MPIU_DBG_FDEST,
- "el_sz = " MPI_AINT_FMT_DEC_SPEC "; el_ext = " MPI_AINT_FMT_DEC_SPEC " }\"];\n",
+ "\\nel_sz = " MPI_AINT_FMT_DEC_SPEC "; el_ext = " MPI_AINT_FMT_DEC_SPEC " }\"];\n",
(MPI_Aint) loop_p->el_size,
(MPI_Aint) loop_p->el_extent));
break;
@@ -126,12 +129,11 @@ void MPIDI_Dataloop_dot_printf(MPID_Dataloop *loop_p,
(int) loop_p->loop_params.bi_t.count,
(int) loop_p->loop_params.bi_t.blocksize));
- /* 3 picked as arbitrary cutoff */
- for (i=0; i < 3 && i < loop_p->loop_params.bi_t.count; i++) {
+ for (i=0; i < NR_TYPE_CUTOFF && i < loop_p->loop_params.bi_t.count; i++) {
if (i + 1 < loop_p->loop_params.bi_t.count) {
/* more regions after this one */
MPIU_DBG_OUT_FMT(DATATYPE,(MPIU_DBG_FDEST,
- MPI_AINT_FMT_DEC_SPEC ", ",
+ MPI_AINT_FMT_DEC_SPEC ",\\n ",
(MPI_Aint) loop_p->loop_params.bi_t.offset_array[i]));
}
else {
@@ -145,7 +147,7 @@ void MPIDI_Dataloop_dot_printf(MPID_Dataloop *loop_p,
}
MPIU_DBG_OUT_FMT(DATATYPE,(MPIU_DBG_FDEST,
- "el_sz = " MPI_AINT_FMT_DEC_SPEC "; el_ext = " MPI_AINT_FMT_DEC_SPEC " }\"];",
+ "\\nel_sz = " MPI_AINT_FMT_DEC_SPEC "; el_ext = " MPI_AINT_FMT_DEC_SPEC " }\"];",
(MPI_Aint) loop_p->el_size,
(MPI_Aint) loop_p->el_extent));
break;
@@ -154,7 +156,7 @@ void MPIDI_Dataloop_dot_printf(MPID_Dataloop *loop_p,
" dl%d [shape = record, label = \"struct | {ct = %d; blks = ",
depth,
(int) loop_p->loop_params.s_t.count));
- for (i=0; i < 3 && i < loop_p->loop_params.s_t.count; i++) {
+ for (i=0; i < NR_TYPE_CUTOFF && i < loop_p->loop_params.s_t.count; i++) {
if (i + 1 < loop_p->loop_params.s_t.count) {
MPIU_DBG_OUT_FMT(DATATYPE,(MPIU_DBG_FDEST,"%d, ",
(int) loop_p->loop_params.s_t.blocksize_array[i]));
@@ -171,7 +173,7 @@ void MPIDI_Dataloop_dot_printf(MPID_Dataloop *loop_p,
MPIU_DBG_OUT(DATATYPE,"disps = ");
}
- for (i=0; i < 3 && i < loop_p->loop_params.s_t.count; i++) {
+ for (i=0; i < NR_TYPE_CUTOFF && i < loop_p->loop_params.s_t.count; i++) {
if (i + 1 < loop_p->loop_params.s_t.count) {
MPIU_DBG_OUT_FMT(DATATYPE,(MPIU_DBG_FDEST,MPI_AINT_FMT_DEC_SPEC ", ",
(MPI_Aint) loop_p->loop_params.s_t.offset_array[i]));
-----------------------------------------------------------------------
Summary of changes:
src/mpi/romio/adio/common/ad_seek.c | 6 ++--
src/mpi/romio/adio/common/utils.c | 39 +++++++++++++++++++++++----
src/mpid/common/datatype/mpid_type_debug.c | 28 ++++++++++---------
3 files changed, 51 insertions(+), 22 deletions(-)
hooks/post-receive
--
MPICH primary repository
More information about the commits
mailing list