[mpich-discuss] MPI-IO bug

[Wei-keng Liao]

They are reasonable doubts.
I tested this patch against test/mpi/io/resized.c and src/mpi/romio/test/hindexed.c

Could you run tests (both MPI and ROMIO) and see if the patch failed for any tests?
At least we will have an idea if it breaks anything first.

Wei-keng

On May 1, 2014, at 10:53 AM, Rob Latham wrote:

> 
> 
> On 04/09/2014 02:52 PM, Wei-keng Liao wrote:
> 
>> 
>> The patch below can fix this problem. Hope it does not break other tests.
> 
> I'm uncertain about this patch...
> 
>> Index: adio/common/ad_read_coll.c
> 
> it should have been in the write path, correct
> ?
>> @@ -368,13 +368,16 @@
>>  #endif
>>          if (file_ptr_type == ADIO_INDIVIDUAL) {
>>             /* Wei-keng reworked type processing to be a bit more
>> efficient */
>> +            for (i=0; i<flat_file->count; i++) /* skip blocklens[] == 0 */
>> +                if (flat_file->blocklens[i] > 0) break;
>> +
> 
> the flat_file->blocklens[] array might contain a UB and LB marker.  The marker will be stored in the flattened representation with a zero-length element at a particular offset.  So if you resize a type, your flattened representation would be like this:
> (offset: 50, length: 0),
> (offset: 100, length: 1024),
> (offset: 200, length: 0)
> 
> furthermore, ADIOI_Optimize_flatten will coalesce multiple zero-length elements into one, but will leave the first and last elements alone.
> 
>>              offset       = fd->fp_ind - disp;
>> -            n_filetypes  = (offset - flat_file->indices[0]) /
>> filetype_extent;
>> +            n_filetypes  = (offset - flat_file->indices[i]) /
>> filetype_extent;
> 
> doesn't this mess up tiling of the file view?
> 
> ==rob
> 
>>              offset      -= (ADIO_Offset)n_filetypes * filetype_extent;
>>              /* now offset is local to this extent */
>> 
>>              /* find the block where offset is located, skip
>> blocklens[i]==0 */
>> -            for (i=0; i<flat_file->count; i++) {
>> +            for (; i<flat_file->count; i++) {
>>                  ADIO_Offset dist;
>>                  if (flat_file->blocklens[i] == 0) continue;
>>                  dist = flat_file->indices[i] + flat_file->blocklens[i]
>> - offset;
>> 
>> Wei-keng
>> 
>> 
>> 
>> _______________________________________________
>> discuss mailing list     discuss at mpich.org
>> To manage subscription options or unsubscribe:
>> https://lists.mpich.org/mailman/listinfo/discuss
> 
> -- 
> Rob Latham
> Mathematics and Computer Science Division
> Argonne National Lab, IL USA
> _______________________________________________
> discuss mailing list     discuss at mpich.org
> To manage subscription options or unsubscribe:
> https://lists.mpich.org/mailman/listinfo/discuss