Skip to content

IPM profile for ds002.140779.0
ds002.140779.0
Powered by IPM
command: ../bin/datastar/paratec.mpi
codename: unknown
state: running
username: nwright
group: USE300
host: ds181 (0020A7DA4C00_AIX)
mpi_tasks: 256 on 32 hosts
start: 05/09/06/12:40:47
wallclock: 1675.73 sec
stop: 05/09/06/13:08:42
%comm: 34.4
total memory: 196.252 gbytes
total gflop/sec: 426.469885
switch(send): 1183.344 gbytes
switch(recv): 1183.325 gbytes

Computation

Event Count Pop
PM_CYC 616094811998179 *
PM_FPU0_FIN 184875250496423 *
PM_FPU1_FIN 184170476323071 *
PM_FPU_FDIV 19795481759 *
PM_FPU_FMA 353784983382445 *
PM_FPU_STF 8182327333904 *
PM_INST_CMPL 724051320374607 *
PM_LSU_LDF 140757363607578 *

Communication

% of MPI Time
HPM Counter Statistics
Event Ntasks Avg Min(rank) Max(rank)
PM_CYC * 2406620359367.89 2369932952820 (234) 2477056945362 (0)
PM_FPU0_FIN * 722168947251.65 711701343278 (234) 729275195945 (16)
PM_FPU1_FIN * 719415923137.00 708949824578 (233) 725172871722 (16)
PM_FPU_FDIV * 77326100.62 76101536 (252) 96419131 (0)
PM_FPU_FMA * 1381972591337.68 1362200126648 (233) 1390603974934 (16)
PM_FPU_STF * 31962216148.06 30783105448 (255) 36416364473 (0)
PM_INST_CMPL * 2828325470213.31 2736904479134 (86) 2916070987580 (232)
PM_LSU_LDF * 549833451592.10 541381802243 (233) 557189465800 (16)
Communication Event Statistics (100.00% detail, 1.5047e-02 error)
  Buffer Size Ncalls Total Time Min Time Max Time %MPI %Wall
MPI_Allreduce 8 108688 14132.575 5.245e-06 1.196e+02 9.57 3.29
MPI_Wait 192 1522041920 13727.429 -2.224e+01 5.449e+00 9.30 3.20
MPI_Wait 176 1513436736 13499.897 -2.264e+01 5.449e+00 9.14 3.15
MPI_Wait 160 1317130976 11689.039 -2.239e+01 5.968e+00 7.92 2.72
MPI_Wait 208 1156724968 10803.281 -2.239e+01 5.887e+00 7.32 2.52
MPI_Wait 144 1021058864 9123.948 -2.286e+01 5.449e+00 6.18 2.13
MPI_Allreduce 120472576 5120 7837.358 5.888e-03 2.254e+00 5.31 1.83
MPI_Wait 224 731844008 6902.661 -2.264e+01 5.250e+00 4.67 1.61
MPI_Wait 128 612447080 5527.415 -2.224e+01 5.723e+00 3.74 1.29
MPI_Irecv 176400 4391189760 5177.798 -2.286e+01 5.408e-02 3.51 1.21
MPI_Allreduce 1762560 175616 5023.080 2.150e-03 8.776e-02 3.40 1.17
MPI_Isend 176 756718848 3303.258 -2.264e+01 4.265e-02 2.24 0.77
MPI_Allreduce 30118144 5632 3261.730 2.847e-01 1.509e+00 2.21 0.76
MPI_Isend 192 761021440 3205.534 -2.264e+01 4.769e-02 2.17 0.75
MPI_Wait 240 338560208 3144.052 -2.222e+01 5.169e+00 2.13 0.73
MPI_Isend 160 658633196 2806.463 -2.264e+01 4.275e-02 1.90 0.65
MPI_Isend 208 578362964 2685.250 -2.061e+01 4.409e-02 1.82 0.63
MPI_Wait 112 279937392 2422.674 -2.264e+01 5.745e+00 1.64 0.56
MPI_Bcast 4 4619264 2412.351 -1.282e+01 8.488e-02 1.63 0.56
MPI_Isend 144 510529912 2127.153 -2.286e+01 4.531e-02 1.44 0.50
MPI_Isend 224 365922484 1632.266 -2.286e+01 4.272e-02 1.11 0.38
MPI_Isend 128 306224020 1344.424 -2.095e+01 4.241e-02 0.91 0.31
MPI_Allreduce 4 34560 1309.404 4.244e-05 1.111e+01 0.89 0.31
MPI_Allreduce 16 36880 1256.126 2.074e-05 1.301e+00 0.85 0.29
MPI_Wait 256 109850552 1028.538 -2.264e+01 1.968e+00 0.70 0.24
MPI_Wait 96 114690968 836.931 -2.099e+01 2.358e+00 0.57 0.20
MPI_Isend 240 169415040 733.990 -2.061e+01 7.213e-03 0.50 0.17
MPI_Isend 112 139969176 643.506 -1.239e+00 4.566e-02 0.44 0.15
MPI_Gather 4 524288 533.582 5.841e-05 1.843e-02 0.36 0.12
MPI_Bcast 118336 691200 502.275 5.770e-05 3.057e-02 0.34 0.12
MPI_Bcast 30118144 3072 424.069 1.319e-01 1.715e-01 0.29 0.10
MPI_Bcast 1492736 27648 376.727 4.683e-04 9.245e-02 0.26 0.09
MPI_Bcast 8 90320 308.096 2.623e-06 7.507e-01 0.21 0.07
MPI_Isend 96 57345964 271.562 -5.719e+00 7.181e-03 0.18 0.06
MPI_Allreduce 10976 5376 270.234 3.626e-04 2.810e-01 0.18 0.06
MPI_Isend 256 55048637 265.629 4.768e-07 2.622e-02 0.18 0.06
MPI_Wait 80 31193792 248.823 4.768e-07 2.260e+00 0.17 0.06
MPI_Bcast 1317120 18944 248.036 4.988e-04 9.076e-02 0.17 0.06
MPI_Bcast 1580544 15360 201.713 1.351e-03 6.994e-02 0.14 0.05
MPI_Wait 272 15462440 157.362 4.768e-07 2.330e+00 0.11 0.04
MPI_Bcast 1404928 10752 150.315 1.081e-03 8.040e-02 0.10 0.04
MPI_Bcast 1179648 25600 143.413 1.871e-03 1.309e-02 0.10 0.03
MPI_Bcast 688 218192 141.543 -7.196e+00 3.496e-01 0.10 0.03
MPI_Bcast 1048576 25600 110.324 2.095e-03 1.169e-02 0.07 0.03
MPI_Bcast 1843968 6656 103.534 2.365e-03 4.835e-02 0.07 0.02
MPI_Bcast 1668352 6656 98.208 1.599e-03 6.841e-02 0.07 0.02
MPI_Bcast 1756160 6144 91.224 1.841e-03 4.984e-02 0.06 0.02
MPI_Gather 12288 131072 82.514 4.005e-05 1.842e-02 0.06 0.02
MPI_Bcast 68 31744 79.424 9.537e-06 6.020e-02 0.05 0.02
MPI_Bcast 1229312 6656 75.495 1.379e-03 6.550e-02 0.05 0.02
MPI_Isend 80 15597376 75.139 1.907e-06 7.273e-03 0.05 0.02
MPI_Bcast 2048 106240 74.709 7.391e-06 3.580e-02 0.05 0.02
MPI_Bcast 1792 123136 71.540 1.001e-05 8.434e-03 0.05 0.02
MPI_Bcast 1024 205312 69.608 4.768e-06 9.667e-03 0.05 0.02
MPI_Bcast 1931776 5120 68.884 2.123e-03 5.510e-02 0.05 0.02
MPI_Bcast 1536 123136 67.619 8.106e-06 8.493e-03 0.05 0.02
MPI_Barrier 0 9216 64.850 2.909e-05 7.144e-02 0.04 0.02
MPI_Bcast 1280 123392 63.281 7.629e-06 5.853e-02 0.04 0.01
MPI_Bcast 112832 92160 60.630 1.571e-04 8.808e-03 0.04 0.01
MPI_Wait 64 7260624 59.213 4.768e-07 2.010e+00 0.04 0.01
MPI_Recv 66560 336374 58.748 4.649e-05 7.592e-03 0.04 0.01
MPI_Waitall 45760 1352792 56.908 2.146e-06 8.490e-03 0.04 0.01
MPI_Recv 49920 403641 55.302 4.053e-05 7.366e-03 0.04 0.01
MPI_Bcast 512 127680 53.808 4.053e-06 7.495e-03 0.04 0.01
MPI_Recv 62400 403446 53.088 4.554e-05 4.388e-02 0.04 0.01
MPI_Waitall 45360 932960 52.572 1.907e-06 4.650e-02 0.04 0.01
MPI_Recv 54080 403407 51.720 3.958e-05 4.445e-02 0.04 0.01
MPI_Recv 45760 403368 51.091 4.411e-05 5.303e-02 0.03 0.01
MPI_Bcast 256 127488 50.009 4.768e-06 7.254e-03 0.03 0.01
MPI_Recv 33280 470830 49.128 3.266e-05 4.420e-02 0.03 0.01
MPI_Bcast 60 19456 48.723 9.537e-06 5.363e-02 0.03 0.01
MPI_Bcast 1141504 4608 46.896 8.783e-04 4.462e-02 0.03 0.01
MPI_Recv 58240 336335 46.066 4.744e-05 7.473e-03 0.03 0.01
MPI_Recv 41600 403563 45.876 4.768e-07 4.341e-02 0.03 0.01
MPI_Bcast 2107392 3584 45.456 2.294e-03 4.437e-02 0.03 0.01
Load balance by task: HPM counters
by MPI rank, by MPI time
Load balance by task: memory, flops, timings
by MPI rank, by MPI time
Communication balance by task (sorted by MPI time)
by MPI rank , time detail by MPI time , time detail by rank , call list
Message Buffer Size Distributions: time
cumulative values, values
Message Buffer Size Distributions: Ncalls
cumulative values, values
Communication Topology : point to point data flow
data sent , data recv , time spent , map_data file map_adjacency file
Switch Traffic (volume by node)
Memory usage by node

Did You Get
What You
Wanted?
Yes No
Comments