rank = 2 hostname = woogie Iter = 4233 / 100000, Current rate: 3.21 Mbit/sec (for 20.6secs), Average rate: 3.21 Mbit/sec (for 20.6secs) Iter = 8337 / 100000, Current rate: 0.85 Mbit/sec (for 75.6secs), Average rate: 1.35 Mbit/sec (for 96.3secs) Iter = 9423 / 100000, Current rate: 0.30 Mbit/sec (for 56.3secs), Average rate: 0.96 Mbit/sec (for 152.6secs) Iter = 9912 / 100000, Current rate: 0.35 Mbit/sec (for 21.7secs), Average rate: 0.89 Mbit/sec (for 174.3secs) Iter = 10412 / 100000, Current rate: 0.36 Mbit/sec (for 21.8secs), Average rate: 0.83 Mbit/sec (for 196.0secs) Iter = 10872 / 100000, Current rate: 0.36 Mbit/sec (for 20.2secs), Average rate: 0.79 Mbit/sec (for 216.2secs) Iter = 11328 / 100000, Current rate: 0.34 Mbit/sec (for 21.1secs), Average rate: 0.75 Mbit/sec (for 237.4secs) Iter = 11760 / 100000, Current rate: 0.31 Mbit/sec (for 22.1secs), Average rate: 0.71 Mbit/sec (for 259.5secs) Iter = 12159 / 100000, Current rate: 0.27 Mbit/sec (for 23.0secs), Average rate: 0.67 Mbit/sec (for 282.5secs) Iter = 12534 / 100000, Current rate: 0.29 Mbit/sec (for 20.4secs), Average rate: 0.65 Mbit/sec (for 302.9secs) Iter = 12908 / 100000, Current rate: 0.28 Mbit/sec (for 21.1secs), Average rate: 0.62 Mbit/sec (for 324.0secs) Iter = 13283 / 100000, Current rate: 0.27 Mbit/sec (for 21.7secs), Average rate: 0.60 Mbit/sec (for 345.7secs) Iter = 13657 / 100000, Current rate: 0.26 Mbit/sec (for 22.4secs), Average rate: 0.58 Mbit/sec (for 368.0secs) Iter = 14032 / 100000, Current rate: 0.26 Mbit/sec (for 23.0secs), Average rate: 0.56 Mbit/sec (for 391.0secs) Iter = 14406 / 100000, Current rate: 0.25 Mbit/sec (for 23.6secs), Average rate: 0.54 Mbit/sec (for 414.6secs) Iter = 14723 / 100000, Current rate: 0.25 Mbit/sec (for 20.1secs), Average rate: 0.53 Mbit/sec (for 434.8secs) Iter = 15038 / 100000, Current rate: 0.24 Mbit/sec (for 20.6secs), Average rate: 0.52 Mbit/sec (for 455.3secs) Iter = 15345 / 100000, Current rate: 0.23 Mbit/sec (for 21.0secs), Average rate: 0.50 Mbit/sec (for 476.3secs) Iter = 15654 / 100000, Current rate: 0.23 Mbit/sec (for 21.4secs), Average rate: 0.49 Mbit/sec (for 497.7secs) Iter = 15967 / 100000, Current rate: 0.22 Mbit/sec (for 21.8secs), Average rate: 0.48 Mbit/sec (for 519.6secs) Iter = 16279 / 100000, Current rate: 0.22 Mbit/sec (for 22.3secs), Average rate: 0.47 Mbit/sec (for 541.9secs) Iter = 16591 / 100000, Current rate: 0.21 Mbit/sec (for 22.7secs), Average rate: 0.46 Mbit/sec (for 564.6secs) Iter = 16903 / 100000, Current rate: 0.21 Mbit/sec (for 23.1secs), Average rate: 0.45 Mbit/sec (for 587.7secs) Iter = 17215 / 100000, Current rate: 0.21 Mbit/sec (for 23.5secs), Average rate: 0.44 Mbit/sec (for 611.2secs) Iter = 17527 / 100000, Current rate: 0.20 Mbit/sec (for 23.9secs), Average rate: 0.43 Mbit/sec (for 635.1secs) Iter = 17839 / 100000, Current rate: 0.20 Mbit/sec (for 24.3secs), Average rate: 0.42 Mbit/sec (for 659.4secs) Iter = 18096 / 100000, Current rate: 0.20 Mbit/sec (for 20.1secs), Average rate: 0.42 Mbit/sec (for 679.4secs) Iter = 18353 / 100000, Current rate: 0.18 Mbit/sec (for 21.7secs), Average rate: 0.41 Mbit/sec (for 701.1secs) Iter = 18590 / 100000, Current rate: 0.16 Mbit/sec (for 23.2secs), Average rate: 0.40 Mbit/sec (for 724.4secs) Iter = 18838 / 100000, Current rate: 0.16 Mbit/sec (for 24.5secs), Average rate: 0.39 Mbit/sec (for 748.8secs) ========================================== MPI_Send Recognized error rank - 2 iter = 18900, fail: ------------------------------------------ ErrorCode: 0x409AE0F ErrorString: Other MPI error, error stack: MPI_Send(173)..............: MPI_Send(buf=0x1a7c8d0, count=2048, MPI_CHAR, dest=0, tag=100, MPI_COMM_WORLD) failed MPID_nem_tcp_connpoll(1827): Communication error with rank 0: Connection reset by peer ==========================================