[mpich-discuss] MPI fail on 20 processes, but not on 10.

[Anatoly G]

Hi.
I have created an application. This application fails on MPI error.
Assertion failed in file src/mpid/ch3/channels/nemesis/src/ch3_progress.c
at line 640: pkt->type >= 0 && pkt->type < MPIDI_NEM_PKT_END
 internal ABORT - process 0

Scenario:
Master receives messages from slaves.
Each slave sends data using MPI_Send.
Master receives using MPI_Irecv and MPI_Recv.

There are another errors in out*.log files.
Application doesn't *fail* with 10 processes, but fails with 20.

execute command:
mpiexec.hydra -genvall -f MpiConfigMachines1.txt -launcher=rsh -n 20
/home/anatol-g/Grape/release_under_constr_MPI_tests_quantum/bin/linux64/rhe6/g++4.4.6/debug/mpi_rcv_any_multithread
100000 1000000 -1 -1 1 out

Please help,

Regards,
Anatoly.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mpich.org/pipermail/discuss/attachments/20140106/0fb6e673/attachment.html>
-------------- next part --------------
1.1.1.1:1
1.1.1.2:1000
-------------- next part --------------
A non-text attachment was scrubbed...
Name: out_r19.log
Type: application/octet-stream
Size: 4027 bytes
Desc: not available
URL: <http://lists.mpich.org/pipermail/discuss/attachments/20140106/0fb6e673/attachment.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: mpi_rcv_any_multithread.cpp
Type: text/x-c++src
Size: 10070 bytes
Desc: not available
URL: <http://lists.mpich.org/pipermail/discuss/attachments/20140106/0fb6e673/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: out_r0.log
Type: application/octet-stream
Size: 4865 bytes
Desc: not available
URL: <http://lists.mpich.org/pipermail/discuss/attachments/20140106/0fb6e673/attachment-0001.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: out_r1.log
Type: application/octet-stream
Size: 3906 bytes
Desc: not available
URL: <http://lists.mpich.org/pipermail/discuss/attachments/20140106/0fb6e673/attachment-0002.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: out_r2.log
Type: application/octet-stream
Size: 4024 bytes
Desc: not available
URL: <http://lists.mpich.org/pipermail/discuss/attachments/20140106/0fb6e673/attachment-0003.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: out_r3.log
Type: application/octet-stream
Size: 4025 bytes
Desc: not available
URL: <http://lists.mpich.org/pipermail/discuss/attachments/20140106/0fb6e673/attachment-0004.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: out_r4.log
Type: application/octet-stream
Size: 4024 bytes
Desc: not available
URL: <http://lists.mpich.org/pipermail/discuss/attachments/20140106/0fb6e673/attachment-0005.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: out_r5.log
Type: application/octet-stream
Size: 3906 bytes
Desc: not available
URL: <http://lists.mpich.org/pipermail/discuss/attachments/20140106/0fb6e673/attachment-0006.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: out_r6.log
Type: application/octet-stream
Size: 4024 bytes
Desc: not available
URL: <http://lists.mpich.org/pipermail/discuss/attachments/20140106/0fb6e673/attachment-0007.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: out_r7.log
Type: application/octet-stream
Size: 3907 bytes
Desc: not available
URL: <http://lists.mpich.org/pipermail/discuss/attachments/20140106/0fb6e673/attachment-0008.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: out_r8.log
Type: application/octet-stream
Size: 4024 bytes
Desc: not available
URL: <http://lists.mpich.org/pipermail/discuss/attachments/20140106/0fb6e673/attachment-0009.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: out_r9.log
Type: application/octet-stream
Size: 4024 bytes
Desc: not available
URL: <http://lists.mpich.org/pipermail/discuss/attachments/20140106/0fb6e673/attachment-0010.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: out_r10.log
Type: application/octet-stream
Size: 3909 bytes
Desc: not available
URL: <http://lists.mpich.org/pipermail/discuss/attachments/20140106/0fb6e673/attachment-0011.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: out_r11.log
Type: application/octet-stream
Size: 4024 bytes
Desc: not available
URL: <http://lists.mpich.org/pipermail/discuss/attachments/20140106/0fb6e673/attachment-0012.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: out_r12.log
Type: application/octet-stream
Size: 4027 bytes
Desc: not available
URL: <http://lists.mpich.org/pipermail/discuss/attachments/20140106/0fb6e673/attachment-0013.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: out_r13.log
Type: application/octet-stream
Size: 4027 bytes
Desc: not available
URL: <http://lists.mpich.org/pipermail/discuss/attachments/20140106/0fb6e673/attachment-0014.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: out_r14.log
Type: application/octet-stream
Size: 3908 bytes
Desc: not available
URL: <http://lists.mpich.org/pipermail/discuss/attachments/20140106/0fb6e673/attachment-0015.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: out_r15.log
Type: application/octet-stream
Size: 4027 bytes
Desc: not available
URL: <http://lists.mpich.org/pipermail/discuss/attachments/20140106/0fb6e673/attachment-0016.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: out_r16.log
Type: application/octet-stream
Size: 4027 bytes
Desc: not available
URL: <http://lists.mpich.org/pipermail/discuss/attachments/20140106/0fb6e673/attachment-0017.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: out_r17.log
Type: application/octet-stream
Size: 4026 bytes
Desc: not available
URL: <http://lists.mpich.org/pipermail/discuss/attachments/20140106/0fb6e673/attachment-0018.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: out_r18.log
Type: application/octet-stream
Size: 3908 bytes
Desc: not available
URL: <http://lists.mpich.org/pipermail/discuss/attachments/20140106/0fb6e673/attachment-0019.obj>