julian.kunkel
/
ddn-ime-evaluation
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Almost all NN:16 benchmarks

This commit is contained in:
Eugen Betke 2018-10-23 23:15:57 +02:00
parent 93c6510ffa
commit 1aed4480e9
20 changed files with 12075 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

110
output/COUNT:1#NN:16#PPN:1#API:MPIIO#T:102400.txt Normal file
View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12,isc17-c13,isc17-c14,isc17-c15,isc17-c18,isc17-c22,isc17-c01,isc17-c02,isc17-c03 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 161061273600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:16#PPN:1#API:MPIIO#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Tue Oct 23 20:35:53 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 161061273600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 20:35:53 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 16 (1 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 150 GiB
aggregate filesize = 2400 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 2576968499200.
WARNING: Using actual aggregate bytes moved = 18205388800.
write 282.84 157286400 100.00 0.012199 61.37 0.000510 61.38 0
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 2576970342400.
WARNING: Using actual aggregate bytes moved = 18470400000.
write 286.69 157286400 100.00 0.001943 61.44 0.000509 61.44 1
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 2576947712000.
WARNING: Using actual aggregate bytes moved = 18311065600.
write 284.49 157286400 100.00 0.002002 61.38 0.000617 61.38 2
Max Write: 286.69 MiB/sec (300.62 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 286.69 282.84 284.67 1.58 61.40290 0 16 1 3 0 0 1 0 0 1 161061273600 102400 18205388800 MPIIO 0
Finished: Tue Oct 23 20:38:58 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12,isc17-c13,isc17-c14,isc17-c15,isc17-c18,isc17-c22,isc17-c01,isc17-c02,isc17-c03 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ tee -a ./output/COUNT:1#NN:16#PPN:1#API:MPIIO#T:102400.txt
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12,isc17-c13,isc17-c14,isc17-c15,isc17-c18,isc17-c22,isc17-c01,isc17-c02,isc17-c03 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 161061273600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Tue Oct 23 20:39:00 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 161061273600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 20:39:00 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 16 (1 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 150 GiB
aggregate filesize = 2400 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 9434828800.
read 146.58 157286400 100.00 0.013505 61.37 0.000718 61.38 0
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 9369497600.
read 145.59 157286400 100.00 0.001391 61.37 0.000729 61.38 1
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 9533849600.
read 148.16 157286400 100.00 0.001105 61.37 0.000804 61.37 2
Max Read: 148.16 MiB/sec (155.35 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 148.16 145.59 146.77 1.06 61.37606 0 16 1 3 0 0 1 0 0 1 161061273600 102400 9434828800 MPIIO 0
Finished: Tue Oct 23 20:42:04 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 32 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12571040 0xbfd1a0 0
4 12512776 0xbeee08 0
7 12461410 0xbe2562 0
1 12642687 0xc0e97f 0
3 12490986 0xbe98ea 0
5 12383049 0xbcf349 0
6 12596308 0xc03454 0
0 12424700 0xbd95fc 0

110
output/COUNT:1#NN:16#PPN:1#API:MPIIO#T:16384.txt Normal file
View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12,isc17-c13,isc17-c14,isc17-c15,isc17-c18,isc17-c22,isc17-c01,isc17-c02,isc17-c03 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 161061273600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:16#PPN:1#API:MPIIO#T:16384.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Tue Oct 23 21:59:32 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 161061273600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 21:59:32 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 16 (1 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 150 GiB
aggregate filesize = 2400 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 2576980017152.
WARNING: Using actual aggregate bytes moved = 3194421248.
write 44.22 157286400 16.00 0.014469 68.88 0.000886 68.89 0
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 2576977313792.
WARNING: Using actual aggregate bytes moved = 3202416640.
write 44.29 157286400 16.00 0.002253 68.96 0.000873 68.96 1
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 2576941858816.
WARNING: Using actual aggregate bytes moved = 3198746624.
write 44.28 157286400 16.00 0.002199 68.88 0.000872 68.89 2
Max Write: 44.29 MiB/sec (46.44 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 44.29 44.22 44.26 0.03 68.91342 0 16 1 3 0 0 1 0 0 1 161061273600 16384 3194421248 MPIIO 0
Finished: Tue Oct 23 22:02:59 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12,isc17-c13,isc17-c14,isc17-c15,isc17-c18,isc17-c22,isc17-c01,isc17-c02,isc17-c03 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ tee -a ./output/COUNT:1#NN:16#PPN:1#API:MPIIO#T:16384.txt
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12,isc17-c13,isc17-c14,isc17-c15,isc17-c18,isc17-c22,isc17-c01,isc17-c02,isc17-c03 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 161061273600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Tue Oct 23 22:03:01 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 161061273600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 22:03:01 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 16 (1 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 150 GiB
aggregate filesize = 2400 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 1819131904.
read 25.22 157286400 16.00 0.011761 68.76 0.001092 68.78 0
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 1824178176.
read 25.30 157286400 16.00 0.002407 68.75 0.001058 68.75 1
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 1819475968.
read 25.24 157286400 16.00 0.001513 68.75 0.001096 68.75 2
Max Read: 25.30 MiB/sec (26.53 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 25.30 25.22 25.26 0.03 68.76020 0 16 1 3 0 0 1 0 0 1 161061273600 16384 1819131904 MPIIO 0
Finished: Tue Oct 23 22:06:28 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 32 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12491994 0xbe9cda 0
5 12384054 0xbcf736 0
6 12597312 0xc03840 0
0 12425719 0xbd99f7 0
2 12572046 0xbfd58e 0
4 12513785 0xbef1f9 0
7 12462419 0xbe2953 0
1 12643696 0xc0ed70 0

44
output/COUNT:1#NN:16#PPN:1#API:POSIX#T:102400.txt Normal file
View File

@ -0,0 +1,44 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12,isc17-c13,isc17-c14,isc17-c15,isc17-c18,isc17-c22,isc17-c01,isc17-c02,isc17-c03 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 161061273600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:16#PPN:1#API:POSIX#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Tue Oct 23 22:45:12 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 161061273600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 22:45:12 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 16 (1 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 150 GiB
aggregate filesize = 2400 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 2576955801600.
WARNING: Using actual aggregate bytes moved = 103605350400.
write 875.39 157286400 100.00 0.000717 112.87 0.000516 112.87 0
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 2576978739200.
WARNING: Using actual aggregate bytes moved = 97390796800.
write 773.11 157286400 100.00 0.001549 120.14 0.000814 120.14 1
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 2576707174400.
WARNING: Using actual aggregate bytes moved = 98684723200.
write 763.36 157286400 100.00 0.001832 123.29 0.001142 123.29 2
Max Write: 875.39 MiB/sec (917.92 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 875.39 763.36 803.95 50.67 118.76515 0 16 1 3 1 0 1 0 0 1 161061273600 102400 103605350400 POSIX 0
Finished: Tue Oct 23 22:51:10 2018

44
output/COUNT:1#NN:16#PPN:1#API:POSIX#T:16384.txt Normal file
View File

@ -0,0 +1,44 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12,isc17-c13,isc17-c14,isc17-c15,isc17-c18,isc17-c22,isc17-c01,isc17-c02,isc17-c03 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 161061273600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:16#PPN:1#API:POSIX#T:16384.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Tue Oct 23 22:51:16 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 161061273600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 22:51:16 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 16 (1 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 150 GiB
aggregate filesize = 2400 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 2576966270976.
WARNING: Using actual aggregate bytes moved = 48308715520.
write 251.70 157286400 16.00 0.001133 183.04 0.000773 183.04 0
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 2576976969728.
WARNING: Using actual aggregate bytes moved = 40420524032.
write 238.78 157286400 16.00 0.012327 161.42 0.001165 161.44 1
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 2576938106880.
WARNING: Using actual aggregate bytes moved = 38403276800.
write 213.59 157286400 16.00 0.002032 171.47 0.001609 171.47 2
Max Write: 251.70 MiB/sec (263.93 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 251.70 213.59 234.69 15.82 171.98076 0 16 1 3 1 0 1 0 0 1 161061273600 16384 48308715520 POSIX 0
Finished: Tue Oct 23 22:59:54 2018

110
output/COUNT:1#NN:16#PPN:2#API:MPIIO#T:102400.txt Normal file
View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12,isc17-c13,isc17-c14,isc17-c15,isc17-c18,isc17-c22,isc17-c01,isc17-c02,isc17-c03 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 80530636800 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:16#PPN:2#API:MPIIO#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Tue Oct 23 21:07:08 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 80530636800 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 21:07:08 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 32 (2 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 75 GiB
aggregate filesize = 2400 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 2576966758400.
WARNING: Using actual aggregate bytes moved = 20476416000.
write 317.96 78643200 100.00 0.025281 61.39 0.000683 61.42 0
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 2576980070400.
WARNING: Using actual aggregate bytes moved = 20587827200.
write 319.40 78643200 100.00 0.003839 61.47 0.000607 61.47 1
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 2576957132800.
WARNING: Using actual aggregate bytes moved = 20513894400.
write 318.44 78643200 100.00 0.004246 61.43 0.000782 61.43 2
Max Write: 319.40 MiB/sec (334.92 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 319.40 317.96 318.60 0.60 61.44096 0 32 2 3 0 0 1 0 0 1 80530636800 102400 20476416000 MPIIO 0
Finished: Tue Oct 23 21:10:13 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12,isc17-c13,isc17-c14,isc17-c15,isc17-c18,isc17-c22,isc17-c01,isc17-c02,isc17-c03 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ tee -a ./output/COUNT:1#NN:16#PPN:2#API:MPIIO#T:102400.txt
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12,isc17-c13,isc17-c14,isc17-c15,isc17-c18,isc17-c22,isc17-c01,isc17-c02,isc17-c03 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 80530636800 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Tue Oct 23 21:10:19 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 80530636800 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 21:10:19 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 32 (2 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 75 GiB
aggregate filesize = 2400 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 15758233600.
read 244.91 78643200 100.00 0.014319 61.35 0.001103 61.36 0
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 15850905600.
read 246.35 78643200 100.00 0.001698 61.36 0.000836 61.36 1
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 15859200000.
read 246.51 78643200 100.00 0.001924 61.35 0.000782 61.35 2
Max Read: 246.51 MiB/sec (258.49 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 246.51 244.91 245.92 0.72 61.35943 0 32 2 3 0 0 1 0 0 1 80530636800 102400 15758233600 MPIIO 0
Finished: Tue Oct 23 21:13:23 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 32 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12425090 0xbd9782 0
2 12571426 0xbfd322 0
4 12513163 0xbeef8b 0
7 12461796 0xbe26e4 0
1 12643073 0xc0eb01 0
3 12491371 0xbe9a6b 0
5 12383428 0xbcf4c4 0
6 12596689 0xc035d1 0

110
output/COUNT:1#NN:16#PPN:2#API:MPIIO#T:16384.txt Normal file
View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12,isc17-c13,isc17-c14,isc17-c15,isc17-c18,isc17-c22,isc17-c01,isc17-c02,isc17-c03 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 80530636800 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:16#PPN:2#API:MPIIO#T:16384.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Tue Oct 23 22:36:20 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 80530636800 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 22:36:20 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 32 (2 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 75 GiB
aggregate filesize = 2400 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 2576973004800.
WARNING: Using actual aggregate bytes moved = 3703734272.
write 51.56 78643200 16.00 0.016513 68.48 0.006620 68.51 0
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 2576976494592.
WARNING: Using actual aggregate bytes moved = 3673653248.
write 51.12 78643200 16.00 0.001890 68.53 0.000832 68.53 1
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 2576975577088.
WARNING: Using actual aggregate bytes moved = 3665985536.
write 51.04 78643200 16.00 0.002547 68.50 0.000973 68.50 2
Max Write: 51.56 MiB/sec (54.06 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 51.56 51.04 51.24 0.23 68.51450 0 32 2 3 0 0 1 0 0 1 80530636800 16384 3703734272 MPIIO 0
Finished: Tue Oct 23 22:39:47 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12,isc17-c13,isc17-c14,isc17-c15,isc17-c18,isc17-c22,isc17-c01,isc17-c02,isc17-c03 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ tee -a ./output/COUNT:1#NN:16#PPN:2#API:MPIIO#T:16384.txt
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12,isc17-c13,isc17-c14,isc17-c15,isc17-c18,isc17-c22,isc17-c01,isc17-c02,isc17-c03 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 80530636800 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Tue Oct 23 22:39:50 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 80530636800 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 22:39:50 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 32 (2 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 75 GiB
aggregate filesize = 2400 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 3156115456.
read 43.97 78643200 16.00 0.020186 68.43 0.011702 68.45 0
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 3175940096.
read 44.26 78643200 16.00 0.009680 68.43 0.008969 68.44 1
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 3169992704.
read 44.18 78643200 16.00 0.009972 68.43 0.009044 68.43 2
Max Read: 44.26 MiB/sec (46.41 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 44.26 43.97 44.14 0.12 68.43898 0 32 2 3 0 0 1 0 0 1 80530636800 16384 3156115456 MPIIO 0
Finished: Tue Oct 23 22:43:15 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 32 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12597699 0xc039c3 0
0 12426101 0xbd9b75 0
2 12572431 0xbfd70f 0
4 12514173 0xbef37d 0
7 12462804 0xbe2ad4 0
1 12644083 0xc0eef3 0
3 12492380 0xbe9e5c 0
5 12384440 0xbcf8b8 0

598
output/COUNT:1#NN:16#PPN:2#API:POSIX#T:102400.txt Normal file
View File

@ -0,0 +1,598 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12,isc17-c13,isc17-c14,isc17-c15,isc17-c18,isc17-c22,isc17-c01,isc17-c02,isc17-c03 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 80530636800 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:16#PPN:2#API:POSIX#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Tue Oct 23 20:57:46 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 80530636800 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 20:57:46 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 32 (2 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 75 GiB
aggregate filesize = 2400 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 2576789299200.
WARNING: Using actual aggregate bytes moved = 107571609600.
write 800.40 78643200 100.00 0.001116 128.17 0.000509 128.17 0
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 2576885043200.
WARNING: Using actual aggregate bytes moved = 101109248000.
write 779.54 78643200 100.00 0.002048 123.69 0.000753 123.69 1
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 2576949964800.
WARNING: Using actual aggregate bytes moved = 102498611200.
write 815.19 78643200 100.00 0.003191 119.91 0.001021 119.91 2
Max Write: 815.19 MiB/sec (854.79 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 815.19 779.54 798.38 14.62 123.92568 0 32 2 3 1 0 1 0 0 1 80530636800 102400 107571609600 POSIX 0
Finished: Tue Oct 23 21:04:00 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12,isc17-c13,isc17-c14,isc17-c15,isc17-c18,isc17-c22,isc17-c01,isc17-c02,isc17-c03 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12,isc17-c13,isc17-c14,isc17-c15,isc17-c18,isc17-c22,isc17-c01,isc17-c02,isc17-c03 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 80530636800 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread16/file -r
+ tee -a ./output/COUNT:1#NN:16#PPN:2#API:POSIX#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Tue Oct 23 21:04:07 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 80530636800 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread16/file -r
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 21:04:07 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/indread16/file
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 32 (2 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 75 GiB
aggregate filesize = 2400 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 21941452800.
read 348.47 78643200 100.00 0.006952 60.05 0.006602 60.05 0
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 22563430400.
read 358.28 78643200 100.00 0.006607 60.06 0.006591 60.06 1
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 22746726400.
read 361.29 78643200 100.00 0.006654 60.04 0.006733 60.04 2
Max Read: 361.29 MiB/sec (378.84 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 361.29 348.47 356.01 5.48 60.05015 0 32 2 3 1 0 1 0 0 1 80530636800 102400 21941452800 POSIX 0
Finished: Tue Oct 23 21:07:07 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 32 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write.00000006
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12513137 0xbeef71 0
7 12461769 0xbe26c9 0
6 12596665 0xc035b9 0
3 12491348 0xbe9a54 0
0 12425069 0xbd976d 0
5 12383406 0xbcf4ae 0
2 12571405 0xbfd30d 0
/esfs/jtacquaviva/ioperf/file_write.00000023
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12383408 0xbcf4b0 0
2 12571407 0xbfd30f 0
4 12513144 0xbeef78 0
1 12643054 0xc0eaee 0
6 12596671 0xc035bf 0
7 12461777 0xbe26d1 0
0 12425073 0xbd9771 0
3 12491351 0xbe9a57 0
/esfs/jtacquaviva/ioperf/file_write.00000002
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12491338 0xbe9a4a 0
0 12425058 0xbd9762 0
2 12571393 0xbfd301 0
7 12461762 0xbe26c2 0
6 12596658 0xc035b2 0
4 12513132 0xbeef6c 0
5 12383397 0xbcf4a5 0
1 12643042 0xc0eae2 0
/esfs/jtacquaviva/ioperf/file_write.00000026
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12513147 0xbeef7b 0
7 12461780 0xbe26d4 0
1 12643057 0xc0eaf1 0
3 12491354 0xbe9a5a 0
5 12383412 0xbcf4b4 0
6 12596673 0xc035c1 0
0 12425075 0xbd9773 0
2 12571410 0xbfd312 0
/esfs/jtacquaviva/ioperf/file_write.00000000
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12383420 0xbcf4bc 0
6 12596681 0xc035c9 0
0 12425082 0xbd977a 0
2 12571418 0xbfd31a 0
4 12513155 0xbeef83 0
7 12461787 0xbe26db 0
1 12643065 0xc0eaf9 0
3 12491362 0xbe9a62 0
/esfs/jtacquaviva/ioperf/file_write.00000020
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12425065 0xbd9769 0
4 12513141 0xbeef75 0
1 12643051 0xc0eaeb 0
5 12383405 0xbcf4ad 0
7 12461773 0xbe26cd 0
2 12571404 0xbfd30c 0
6 12596669 0xc035bd 0
/esfs/jtacquaviva/ioperf/file_write.00000030
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12643060 0xc0eaf4 0
3 12491357 0xbe9a5d 0
5 12383415 0xbcf4b7 0
6 12596676 0xc035c4 0
0 12425077 0xbd9775 0
2 12571413 0xbfd315 0
4 12513150 0xbeef7e 0
7 12461782 0xbe26d6 0
/esfs/jtacquaviva/ioperf/file_write.00000004
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12571409 0xbfd311 0
4 12513146 0xbeef7a 0
7 12461779 0xbe26d3 0
1 12643056 0xc0eaf0 0
3 12491353 0xbe9a59 0
5 12383411 0xbcf4b3 0
6 12596672 0xc035c0 0
0 12425074 0xbd9772 0
/esfs/jtacquaviva/ioperf/file_write.00000007
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12383417 0xbcf4b9 0
6 12596678 0xc035c6 0
0 12425079 0xbd9777 0
7 12461784 0xbe26d8 0
1 12643062 0xc0eaf6 0
2 12571416 0xbfd318 0
4 12513153 0xbeef81 0
3 12491361 0xbe9a61 0
/esfs/jtacquaviva/ioperf/file_write.00000018
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12425084 0xbd977c 0
2 12571420 0xbfd31c 0
4 12513157 0xbeef85 0
7 12461789 0xbe26dd 0
1 12643067 0xc0eafb 0
3 12491364 0xbe9a64 0
5 12383422 0xbcf4be 0
6 12596683 0xc035cb 0
/esfs/jtacquaviva/ioperf/file_write.00000031
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12491344 0xbe9a50 0
4 12513136 0xbeef70 0
2 12571402 0xbfd30a 0
7 12461768 0xbe26c8 0
0 12425067 0xbd976b 0
1 12643049 0xc0eae9 0
5 12383403 0xbcf4ab 0
6 12596666 0xc035ba 0
/esfs/jtacquaviva/ioperf/file_write.00000012
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12491358 0xbe9a5e 0
5 12383416 0xbcf4b8 0
6 12596677 0xc035c5 0
0 12425078 0xbd9776 0
2 12571414 0xbfd316 0
4 12513151 0xbeef7f 0
7 12461783 0xbe26d7 0
1 12643061 0xc0eaf5 0
/esfs/jtacquaviva/ioperf/file_write.00000013
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12425063 0xbd9767 0
2 12571398 0xbfd306 0
6 12596663 0xc035b7 0
5 12383402 0xbcf4aa 0
1 12643048 0xc0eae8 0
4 12513140 0xbeef74 0
3 12491345 0xbe9a51 0
7 12461770 0xbe26ca 0
/esfs/jtacquaviva/ioperf/file_write.00000025
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12461792 0xbe26e0 0
1 12643070 0xc0eafe 0
3 12491367 0xbe9a67 0
5 12383425 0xbcf4c1 0
6 12596686 0xc035ce 0
0 12425087 0xbd977f 0
2 12571423 0xbfd31f 0
4 12513160 0xbeef88 0
/esfs/jtacquaviva/ioperf/file_write.00000022
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12596682 0xc035ca 0
0 12425083 0xbd977b 0
2 12571419 0xbfd31b 0
4 12513156 0xbeef84 0
7 12461788 0xbe26dc 0
1 12643066 0xc0eafa 0
3 12491363 0xbe9a63 0
5 12383421 0xbcf4bd 0
/esfs/jtacquaviva/ioperf/file_write.00000003
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12383401 0xbcf4a9 0
7 12461767 0xbe26c7 0
6 12596662 0xc035b6 0
3 12491343 0xbe9a4f 0
0 12425064 0xbd9768 0
1 12643046 0xc0eae6 0
2 12571399 0xbfd307 0
/esfs/jtacquaviva/ioperf/file_write.00000008
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12571421 0xbfd31d 0
4 12513158 0xbeef86 0
7 12461790 0xbe26de 0
1 12643068 0xc0eafc 0
3 12491365 0xbe9a65 0
5 12383423 0xbcf4bf 0
6 12596684 0xc035cc 0
0 12425085 0xbd977d 0
/esfs/jtacquaviva/ioperf/file_write.00000028
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12461781 0xbe26d5 0
1 12643058 0xc0eaf2 0
3 12491355 0xbe9a5b 0
5 12383413 0xbcf4b5 0
6 12596674 0xc035c2 0
0 12425076 0xbd9774 0
2 12571412 0xbfd314 0
4 12513149 0xbeef7d 0
/esfs/jtacquaviva/ioperf/file_write.00000010
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12491359 0xbe9a5f 0
5 12383419 0xbcf4bb 0
2 12571415 0xbfd317 0
6 12596680 0xc035c8 0
4 12513154 0xbeef82 0
7 12461786 0xbe26da 0
0 12425081 0xbd9779 0
1 12643064 0xc0eaf8 0
/esfs/jtacquaviva/ioperf/file_write.00000017
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12596657 0xc035b1 0
4 12513131 0xbeef6b 0
1 12643041 0xc0eae1 0
5 12383396 0xbcf4a4 0
0 12425059 0xbd9763 0
7 12461763 0xbe26c3 0
3 12491339 0xbe9a4b 0
2 12571394 0xbfd302 0
/esfs/jtacquaviva/ioperf/file_write.00000029
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12643044 0xc0eae4 0
5 12383399 0xbcf4a7 0
2 12571397 0xbfd305 0
3 12491341 0xbe9a4d 0
4 12513134 0xbeef6e 0
6 12596661 0xc035b5 0
7 12461766 0xbe26c6 0
0 12425062 0xbd9766 0
/esfs/jtacquaviva/ioperf/file_write.00000016
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12571395 0xbfd303 0
5 12383400 0xbcf4a8 0
1 12643045 0xc0eae5 0
6 12596660 0xc035b4 0
7 12461765 0xbe26c5 0
0 12425061 0xbd9765 0
3 12491342 0xbe9a4e 0
4 12513135 0xbeef6f 0
/esfs/jtacquaviva/ioperf/file_write.00000005
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12643040 0xc0eae0 0
3 12491337 0xbe9a49 0
5 12383395 0xbcf4a3 0
6 12596656 0xc035b0 0
0 12425057 0xbd9761 0
2 12571392 0xbfd300 0
4 12513130 0xbeef6a 0
7 12461761 0xbe26c1 0
/esfs/jtacquaviva/ioperf/file_write.00000014
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12513152 0xbeef80 0
3 12491360 0xbe9a60 0
5 12383418 0xbcf4ba 0
7 12461785 0xbe26d9 0
6 12596679 0xc035c7 0
1 12643063 0xc0eaf7 0
0 12425080 0xbd9778 0
2 12571417 0xbfd319 0
/esfs/jtacquaviva/ioperf/file_write.00000021
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12513159 0xbeef87 0
7 12461791 0xbe26df 0
1 12643069 0xc0eafd 0
3 12491366 0xbe9a66 0
5 12383424 0xbcf4c0 0
6 12596685 0xc035cd 0
0 12425086 0xbd977e 0
2 12571422 0xbfd31e 0
/esfs/jtacquaviva/ioperf/file_write.00000019
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12571403 0xbfd30b 0
0 12425066 0xbd976a 0
5 12383409 0xbcf4b1 0
6 12596670 0xc035be 0
1 12643055 0xc0eaef 0
7 12461778 0xbe26d2 0
4 12513145 0xbeef79 0
3 12491352 0xbe9a58 0
/esfs/jtacquaviva/ioperf/file_write.00000009
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12425070 0xbd976e 0
7 12461776 0xbe26d0 0
3 12491356 0xbe9a5c 0
2 12571411 0xbfd313 0
1 12643059 0xc0eaf3 0
4 12513148 0xbeef7c 0
5 12383414 0xbcf4b6 0
6 12596675 0xc035c3 0
/esfs/jtacquaviva/ioperf/file_write.00000015
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12571401 0xbfd309 0
6 12596664 0xc035b8 0
3 12491347 0xbe9a53 0
4 12513142 0xbeef76 0
1 12643053 0xc0eaed 0
7 12461775 0xbe26cf 0
0 12425072 0xbd9770 0
/esfs/jtacquaviva/ioperf/file_write.00000027
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12461764 0xbe26c4 0
1 12643043 0xc0eae3 0
6 12596659 0xc035b3 0
4 12513133 0xbeef6d 0
3 12491340 0xbe9a4c 0
5 12383398 0xbcf4a6 0
0 12425060 0xbd9764 0
2 12571396 0xbfd304 0
/esfs/jtacquaviva/ioperf/file_write.00000024
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12571400 0xbfd308 0
4 12513138 0xbeef72 0
3 12491346 0xbe9a52 0
1 12643050 0xc0eaea 0
5 12383404 0xbcf4ac 0
7 12461772 0xbe26cc 0
6 12596668 0xc035bc 0
0 12425071 0xbd976f 0
/esfs/jtacquaviva/ioperf/file_write.00000001
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12643047 0xc0eae7 0
0 12425068 0xbd976c 0
7 12461771 0xbe26cb 0
4 12513143 0xbeef77 0
5 12383407 0xbcf4af 0
3 12491349 0xbe9a55 0
2 12571406 0xbfd30e 0
/esfs/jtacquaviva/ioperf/file_write.00000011
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12513139 0xbeef73 0
1 12643052 0xc0eaec 0
6 12596667 0xc035bb 0
7 12461774 0xbe26ce 0
2 12571408 0xbfd310 0
5 12383410 0xbcf4b2 0
3 12491350 0xbe9a56 0

600
output/COUNT:1#NN:16#PPN:2#API:POSIX#T:16384.txt Normal file
View File

@ -0,0 +1,600 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12,isc17-c13,isc17-c14,isc17-c15,isc17-c18,isc17-c22,isc17-c01,isc17-c02,isc17-c03 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 80530636800 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:16#PPN:2#API:POSIX#T:16384.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Tue Oct 23 22:24:57 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 80530636800 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 22:24:57 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 32 (2 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 75 GiB
aggregate filesize = 2400 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 2576953704448.
WARNING: Using actual aggregate bytes moved = 43030691840.
write 248.62 78643200 16.00 0.000930 165.06 0.000633 165.06 0
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 2576886644736.
WARNING: Using actual aggregate bytes moved = 37815009280.
write 221.74 78643200 16.00 0.002044 162.63 0.000707 162.64 1
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 2576513892352.
WARNING: Using actual aggregate bytes moved = 39176388608.
write 222.27 78643200 16.00 0.019878 168.07 0.001107 168.09 2
Max Write: 248.62 MiB/sec (260.69 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 248.62 221.74 230.88 12.55 165.26129 0 32 2 3 1 0 1 0 0 1 80530636800 16384 43030691840 POSIX 0
Finished: Tue Oct 23 22:33:14 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12,isc17-c13,isc17-c14,isc17-c15,isc17-c18,isc17-c22,isc17-c01,isc17-c02,isc17-c03 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12,isc17-c13,isc17-c14,isc17-c15,isc17-c18,isc17-c22,isc17-c01,isc17-c02,isc17-c03 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 80530636800 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread16/file -r
+ tee -a ./output/COUNT:1#NN:16#PPN:2#API:POSIX#T:16384.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Tue Oct 23 22:33:18 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 80530636800 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread16/file -r
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 22:33:18 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/indread16/file
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 32 (2 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 75 GiB
aggregate filesize = 2400 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 3848159232.
read 60.88 78643200 16.00 0.001235 60.28 0.000831 60.28 0
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 4552736768.
read 72.03 78643200 16.00 0.000874 60.28 0.000890 60.28 1
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 4550787072.
read 71.99 78643200 16.00 0.000906 60.28 0.000944 60.28 2
Max Read: 72.03 MiB/sec (75.52 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 72.03 60.88 68.30 5.25 60.28195 0 32 2 3 1 0 1 0 0 1 80530636800 16384 3848159232 POSIX 0
Finished: Tue Oct 23 22:36:19 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 32 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write.00000006
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12572415 0xbfd6ff 0
4 12514157 0xbef36d 0
7 12462786 0xbe2ac2 0
1 12644066 0xc0eee2 0
3 12492363 0xbe9e4b 0
5 12384423 0xbcf8a7 0
6 12597682 0xc039b2 0
0 12426085 0xbd9b65 0
/esfs/jtacquaviva/ioperf/file_write.00000023
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12514146 0xbef362 0
7 12462776 0xbe2ab8 0
1 12644056 0xc0eed8 0
3 12492357 0xbe9e45 0
2 12572411 0xbfd6fb 0
6 12597679 0xc039af 0
5 12384420 0xbcf8a4 0
0 12426082 0xbd9b62 0
/esfs/jtacquaviva/ioperf/file_write.00000002
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12514142 0xbef35e 0
7 12462771 0xbe2ab3 0
1 12644051 0xc0eed3 0
3 12492348 0xbe9e3c 0
5 12384409 0xbcf899 0
6 12597667 0xc039a3 0
0 12426070 0xbd9b56 0
2 12572400 0xbfd6f0 0
/esfs/jtacquaviva/ioperf/file_write.00000026
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12644067 0xc0eee3 0
5 12384424 0xbcf8a8 0
0 12426086 0xbd9b66 0
2 12572416 0xbfd700 0
3 12492365 0xbe9e4d 0
6 12597684 0xc039b4 0
7 12462788 0xbe2ac4 0
/esfs/jtacquaviva/ioperf/file_write.00000000
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12384421 0xbcf8a5 0
6 12597680 0xc039b0 0
4 12514155 0xbef36b 0
7 12462784 0xbe2ac0 0
3 12492361 0xbe9e49 0
0 12426083 0xbd9b63 0
2 12572414 0xbfd6fe 0
1 12644065 0xc0eee1 0
/esfs/jtacquaviva/ioperf/file_write.00000020
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12597670 0xc039a6 0
2 12572405 0xbfd6f5 0
7 12462777 0xbe2ab9 0
4 12514148 0xbef364 0
0 12426077 0xbd9b5d 0
1 12644061 0xc0eedd 0
3 12492358 0xbe9e46 0
5 12384419 0xbcf8a3 0
/esfs/jtacquaviva/ioperf/file_write.00000030
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12384412 0xbcf89c 0
1 12644057 0xc0eed9 0
0 12426079 0xbd9b5f 0
4 12514150 0xbef366 0
3 12492356 0xbe9e44 0
7 12462781 0xbe2abd 0
6 12597675 0xc039ab 0
2 12572410 0xbfd6fa 0
/esfs/jtacquaviva/ioperf/file_write.00000004
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12572422 0xbfd706 0
5 12384431 0xbcf8af 0
0 12426092 0xbd9b6c 0
4 12514165 0xbef375 0
7 12462795 0xbe2acb 0
6 12597691 0xc039bb 0
1 12644075 0xc0eeeb 0
3 12492372 0xbe9e54 0
/esfs/jtacquaviva/ioperf/file_write.00000007
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12492349 0xbe9e3d 0
7 12462772 0xbe2ab4 0
0 12426071 0xbd9b57 0
5 12384410 0xbcf89a 0
2 12572402 0xbfd6f2 0
1 12644052 0xc0eed4 0
6 12597669 0xc039a5 0
4 12514144 0xbef360 0
/esfs/jtacquaviva/ioperf/file_write.00000018
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12492350 0xbe9e3e 0
2 12572401 0xbfd6f1 0
0 12426072 0xbd9b58 0
7 12462774 0xbe2ab6 0
5 12384411 0xbcf89b 0
4 12514145 0xbef361 0
1 12644055 0xc0eed7 0
6 12597672 0xc039a8 0
/esfs/jtacquaviva/ioperf/file_write.00000031
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12572417 0xbfd701 0
0 12426087 0xbd9b67 0
4 12514159 0xbef36f 0
1 12644069 0xc0eee5 0
3 12492367 0xbe9e4f 0
5 12384427 0xbcf8ab 0
6 12597686 0xc039b6 0
7 12462789 0xbe2ac5 0
/esfs/jtacquaviva/ioperf/file_write.00000012
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12462800 0xbe2ad0 0
1 12644080 0xc0eef0 0
3 12492377 0xbe9e59 0
5 12384437 0xbcf8b5 0
6 12597696 0xc039c0 0
0 12426098 0xbd9b72 0
2 12572428 0xbfd70c 0
4 12514170 0xbef37a 0
/esfs/jtacquaviva/ioperf/file_write.00000013
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12514169 0xbef379 0
7 12462799 0xbe2acf 0
1 12644079 0xc0eeef 0
3 12492376 0xbe9e58 0
5 12384436 0xbcf8b4 0
6 12597695 0xc039bf 0
0 12426097 0xbd9b71 0
2 12572427 0xbfd70b 0
/esfs/jtacquaviva/ioperf/file_write.00000025
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12462773 0xbe2ab5 0
0 12426073 0xbd9b59 0
2 12572403 0xbfd6f3 0
1 12644054 0xc0eed6 0
6 12597671 0xc039a7 0
4 12514147 0xbef363 0
5 12384415 0xbcf89f 0
3 12492354 0xbe9e42 0
/esfs/jtacquaviva/ioperf/file_write.00000022
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12597689 0xc039b9 0
4 12514164 0xbef374 0
7 12462793 0xbe2ac9 0
1 12644073 0xc0eee9 0
3 12492370 0xbe9e52 0
2 12572423 0xbfd707 0
0 12426093 0xbd9b6d 0
5 12384432 0xbcf8b0 0
/esfs/jtacquaviva/ioperf/file_write.00000003
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12572398 0xbfd6ee 0
4 12514140 0xbef35c 0
7 12462769 0xbe2ab1 0
1 12644049 0xc0eed1 0
3 12492346 0xbe9e3a 0
5 12384407 0xbcf897 0
6 12597665 0xc039a1 0
0 12426068 0xbd9b54 0
/esfs/jtacquaviva/ioperf/file_write.00000008
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12572413 0xbfd6fd 0
1 12644064 0xc0eee0 0
5 12384422 0xbcf8a6 0
4 12514156 0xbef36c 0
0 12426084 0xbd9b64 0
7 12462785 0xbe2ac1 0
3 12492362 0xbe9e4a 0
6 12597681 0xc039b1 0
/esfs/jtacquaviva/ioperf/file_write.00000028
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12426089 0xbd9b69 0
2 12572420 0xbfd704 0
5 12384429 0xbcf8ad 0
4 12514162 0xbef372 0
6 12597690 0xc039ba 0
7 12462794 0xbe2aca 0
1 12644074 0xc0eeea 0
3 12492371 0xbe9e53 0
/esfs/jtacquaviva/ioperf/file_write.00000010
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12492366 0xbe9e4e 0
5 12384426 0xbcf8aa 0
6 12597685 0xc039b5 0
0 12426088 0xbd9b68 0
2 12572419 0xbfd703 0
4 12514160 0xbef370 0
7 12462790 0xbe2ac6 0
1 12644070 0xc0eee6 0
/esfs/jtacquaviva/ioperf/file_write.00000017
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12492351 0xbe9e3f 0
0 12426076 0xbd9b5c 0
1 12644060 0xc0eedc 0
2 12572409 0xbfd6f9 0
6 12597676 0xc039ac 0
5 12384418 0xbcf8a2 0
4 12514154 0xbef36a 0
7 12462783 0xbe2abf 0
/esfs/jtacquaviva/ioperf/file_write.00000029
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12426074 0xbd9b5a 0
6 12597674 0xc039aa 0
5 12384416 0xbcf8a0 0
3 12492355 0xbe9e43 0
7 12462780 0xbe2abc 0
2 12572408 0xbfd6f8 0
1 12644062 0xc0eede 0
4 12514152 0xbef368 0
/esfs/jtacquaviva/ioperf/file_write.00000016
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12384428 0xbcf8ac 0
7 12462791 0xbe2ac7 0
1 12644071 0xc0eee7 0
6 12597687 0xc039b7 0
3 12492369 0xbe9e51 0
0 12426090 0xbd9b6a 0
2 12572421 0xbfd705 0
4 12514163 0xbef373 0
/esfs/jtacquaviva/ioperf/file_write.00000005
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12426075 0xbd9b5b 0
7 12462782 0xbe2abe 0
6 12597677 0xc039ad 0
4 12514153 0xbef369 0
3 12492360 0xbe9e48 0
2 12572412 0xbfd6fc 0
1 12644063 0xc0eedf 0
/esfs/jtacquaviva/ioperf/file_write.00000014
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12514161 0xbef371 0
3 12492368 0xbe9e50 0
7 12462792 0xbe2ac8 0
6 12597688 0xc039b8 0
1 12644072 0xc0eee8 0
5 12384430 0xbcf8ae 0
0 12426091 0xbd9b6b 0
/esfs/jtacquaviva/ioperf/file_write.00000021
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12426095 0xbd9b6f 0
2 12572425 0xbfd709 0
4 12514167 0xbef377 0
7 12462797 0xbe2acd 0
1 12644077 0xc0eeed 0
3 12492374 0xbe9e56 0
5 12384434 0xbcf8b2 0
6 12597693 0xc039bd 0
/esfs/jtacquaviva/ioperf/file_write.00000019
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12572426 0xbfd70a 0
4 12514168 0xbef378 0
7 12462798 0xbe2ace 0
1 12644078 0xc0eeee 0
3 12492375 0xbe9e57 0
5 12384435 0xbcf8b3 0
6 12597694 0xc039be 0
0 12426096 0xbd9b70 0
/esfs/jtacquaviva/ioperf/file_write.00000009
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12514141 0xbef35d 0
7 12462770 0xbe2ab2 0
1 12644050 0xc0eed2 0
3 12492347 0xbe9e3b 0
5 12384408 0xbcf898 0
6 12597666 0xc039a2 0
0 12426069 0xbd9b55 0
2 12572399 0xbfd6ef 0
/esfs/jtacquaviva/ioperf/file_write.00000015
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12644053 0xc0eed5 0
5 12384413 0xbcf89d 0
0 12426078 0xbd9b5e 0
2 12572406 0xbfd6f6 0
7 12462778 0xbe2aba 0
4 12514151 0xbef367 0
6 12597678 0xc039ae 0
3 12492359 0xbe9e47 0
/esfs/jtacquaviva/ioperf/file_write.00000027
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12597668 0xc039a4 0
4 12514143 0xbef35f 0
2 12572404 0xbfd6f4 0
7 12462775 0xbe2ab7 0
3 12492352 0xbe9e40 0
5 12384414 0xbcf89e 0
1 12644059 0xc0eedb 0
0 12426080 0xbd9b60 0
/esfs/jtacquaviva/ioperf/file_write.00000024
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12514158 0xbef36e 0
3 12492364 0xbe9e4c 0
6 12597683 0xc039b3 0
7 12462787 0xbe2ac3 0
5 12384425 0xbcf8a9 0
1 12644068 0xc0eee4 0
2 12572418 0xbfd702 0
/esfs/jtacquaviva/ioperf/file_write.00000001
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12597692 0xc039bc 0
0 12426094 0xbd9b6e 0
2 12572424 0xbfd708 0
4 12514166 0xbef376 0
7 12462796 0xbe2acc 0
1 12644076 0xc0eeec 0
3 12492373 0xbe9e55 0
5 12384433 0xbcf8b1 0
/esfs/jtacquaviva/ioperf/file_write.00000011
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12597673 0xc039a9 0
3 12492353 0xbe9e41 0
4 12514149 0xbef365 0
1 12644058 0xc0eeda 0
7 12462779 0xbe2abb 0
5 12384417 0xbcf8a1 0
2 12572407 0xbfd6f7 0
0 12426081 0xbd9b61 0

110
output/COUNT:1#NN:16#PPN:4#API:MPIIO#T:102400.txt Normal file
View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 64 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12,isc17-c13,isc17-c14,isc17-c15,isc17-c18,isc17-c22,isc17-c01,isc17-c02,isc17-c03 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 40265318400 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:16#PPN:4#API:MPIIO#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Tue Oct 23 20:23:34 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 40265318400 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 20:23:34 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 64 (4 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 37.50 GiB
aggregate filesize = 2400 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 2576944128000.
WARNING: Using actual aggregate bytes moved = 25013657600.
write 388.01 39321600 100.00 0.014584 61.47 0.000576 61.48 0
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 2576949248000.
WARNING: Using actual aggregate bytes moved = 25303552000.
write 392.87 39321600 100.00 0.002228 61.42 0.000658 61.42 1
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 2576967372800.
WARNING: Using actual aggregate bytes moved = 25200332800.
write 391.64 39321600 100.00 0.004342 61.36 0.000815 61.36 2
Max Write: 392.87 MiB/sec (411.95 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 392.87 388.01 390.84 2.06 61.42278 0 64 4 3 0 0 1 0 0 1 40265318400 102400 25013657600 MPIIO 0
Finished: Tue Oct 23 20:26:39 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 64 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12,isc17-c13,isc17-c14,isc17-c15,isc17-c18,isc17-c22,isc17-c01,isc17-c02,isc17-c03 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ tee -a ./output/COUNT:1#NN:16#PPN:4#API:MPIIO#T:102400.txt
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 64 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12,isc17-c13,isc17-c14,isc17-c15,isc17-c18,isc17-c22,isc17-c01,isc17-c02,isc17-c03 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 40265318400 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Tue Oct 23 20:26:46 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 40265318400 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 20:26:46 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 64 (4 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 37.50 GiB
aggregate filesize = 2400 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 23957913600.
read 372.22 39321600 100.00 0.025120 61.36 0.008300 61.38 0
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 24092364800.
read 374.41 39321600 100.00 0.004821 61.36 0.003211 61.37 1
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 24276992000.
read 377.34 39321600 100.00 0.005271 61.35 0.003289 61.36 2
Max Read: 377.34 MiB/sec (395.67 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 377.34 372.22 374.66 2.10 61.36891 0 64 4 3 0 0 1 0 0 1 40265318400 102400 23957913600 MPIIO 0
Finished: Tue Oct 23 20:29:50 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 32 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12570990 0xbfd16e 0
4 12512727 0xbeedd7 0
7 12461360 0xbe2530 0
1 12642638 0xc0e94e 0
3 12490936 0xbe98b8 0
5 12383000 0xbcf318 0
6 12596258 0xc03422 0
0 12424650 0xbd95ca 0

110
output/COUNT:1#NN:16#PPN:4#API:MPIIO#T:16384.txt Normal file
View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 64 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12,isc17-c13,isc17-c14,isc17-c15,isc17-c18,isc17-c22,isc17-c01,isc17-c02,isc17-c03 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 40265318400 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:16#PPN:4#API:MPIIO#T:16384.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Tue Oct 23 21:44:02 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 40265318400 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 21:44:02 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 64 (4 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 37.50 GiB
aggregate filesize = 2400 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 2576974430208.
WARNING: Using actual aggregate bytes moved = 4304732160.
write 59.84 39321600 16.00 0.025778 68.58 0.000717 68.61 0
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 2576943087616.
WARNING: Using actual aggregate bytes moved = 4308074496.
write 60.06 39321600 16.00 0.002767 68.41 0.000707 68.41 1
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 2576961110016.
WARNING: Using actual aggregate bytes moved = 4297834496.
write 59.83 39321600 16.00 0.002667 68.50 0.000813 68.50 2
Max Write: 60.06 MiB/sec (62.97 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 60.06 59.83 59.91 0.10 68.50740 0 64 4 3 0 0 1 0 0 1 40265318400 16384 4304732160 MPIIO 0
Finished: Tue Oct 23 21:47:28 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 64 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12,isc17-c13,isc17-c14,isc17-c15,isc17-c18,isc17-c22,isc17-c01,isc17-c02,isc17-c03 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ tee -a ./output/COUNT:1#NN:16#PPN:4#API:MPIIO#T:16384.txt
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 64 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12,isc17-c13,isc17-c14,isc17-c15,isc17-c18,isc17-c22,isc17-c01,isc17-c02,isc17-c03 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 40265318400 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Tue Oct 23 21:47:32 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 40265318400 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 21:47:32 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 64 (4 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 37.50 GiB
aggregate filesize = 2400 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 4970004480.
read 69.39 39321600 16.00 0.017438 68.29 0.004506 68.31 0
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 4990713856.
read 69.68 39321600 16.00 0.005675 68.30 0.004488 68.31 1
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 5004328960.
read 69.87 39321600 16.00 0.006482 68.30 0.004589 68.31 2
Max Read: 69.87 MiB/sec (73.26 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 69.87 69.39 69.64 0.20 68.30819 0 64 4 3 0 0 1 0 0 1 40265318400 16384 4970004480 MPIIO 0
Finished: Tue Oct 23 21:50:57 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 32 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12425668 0xbd99c4 0
2 12571999 0xbfd55f 0
4 12513736 0xbef1c8 0
7 12462370 0xbe2922 0
1 12643648 0xc0ed40 0
3 12491944 0xbe9ca8 0
5 12384005 0xbcf705 0
6 12597263 0xc0380f 0

1103
output/COUNT:1#NN:16#PPN:4#API:POSIX#T:102400.txt Normal file
View File

File diff suppressed because it is too large Load Diff

1104
output/COUNT:1#NN:16#PPN:4#API:POSIX#T:16384.txt Normal file
View File

File diff suppressed because it is too large Load Diff

110
output/COUNT:1#NN:16#PPN:6#API:MPIIO#T:102400.txt Normal file
View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 96 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12,isc17-c13,isc17-c14,isc17-c15,isc17-c18,isc17-c22,isc17-c01,isc17-c02,isc17-c03 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 26843545600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:16#PPN:6#API:MPIIO#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Tue Oct 23 20:51:27 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 26843545600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 20:51:27 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 96 (6 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 25 GiB
aggregate filesize = 2400 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 2576976076800.
WARNING: Using actual aggregate bytes moved = 27160780800.
write 416.91 26214400 100.00 0.370195 61.77 0.012444 62.13 0
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 2576954265600.
WARNING: Using actual aggregate bytes moved = 27222323200.
write 421.16 26214400 100.00 0.014395 61.64 0.012260 61.64 1
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 2576959590400.
WARNING: Using actual aggregate bytes moved = 27131596800.
write 420.14 26214400 100.00 0.017603 61.58 0.012438 61.59 2
Max Write: 421.16 MiB/sec (441.62 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 421.16 416.91 419.40 1.81 61.78596 0 96 6 3 0 0 1 0 0 1 26843545600 102400 27160780800 MPIIO 0
Finished: Tue Oct 23 20:54:33 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 96 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12,isc17-c13,isc17-c14,isc17-c15,isc17-c18,isc17-c22,isc17-c01,isc17-c02,isc17-c03 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ tee -a ./output/COUNT:1#NN:16#PPN:6#API:MPIIO#T:102400.txt
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 96 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12,isc17-c13,isc17-c14,isc17-c15,isc17-c18,isc17-c22,isc17-c01,isc17-c02,isc17-c03 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 26843545600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Tue Oct 23 20:54:40 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 26843545600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 20:54:40 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 96 (6 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 25 GiB
aggregate filesize = 2400 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 29335961600.
read 453.35 26214400 100.00 0.349667 61.37 0.007523 61.71 0
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 29731328000.
read 462.07 26214400 100.00 0.010683 61.36 0.007504 61.36 1
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 30092800000.
read 467.75 26214400 100.00 0.010186 61.35 0.007504 61.36 2
Max Read: 467.75 MiB/sec (490.47 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 467.75 453.35 461.05 5.92 61.47689 0 96 6 3 0 0 1 0 0 1 26843545600 102400 29335961600 MPIIO 0
Finished: Tue Oct 23 20:57:45 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 32 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12461695 0xbe267f 0
1 12642974 0xc0ea9e 0
3 12491271 0xbe9a07 0
5 12383330 0xbcf462 0
6 12596589 0xc0356d 0
0 12424990 0xbd971e 0
2 12571325 0xbfd2bd 0
4 12513063 0xbeef27 0

110
output/COUNT:1#NN:16#PPN:6#API:MPIIO#T:16384.txt Normal file
View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 96 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12,isc17-c13,isc17-c14,isc17-c15,isc17-c18,isc17-c22,isc17-c01,isc17-c02,isc17-c03 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 26843545600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:16#PPN:6#API:MPIIO#T:16384.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Tue Oct 23 22:18:00 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 26843545600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 22:18:00 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 96 (6 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 25 GiB
aggregate filesize = 2400 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 2576969908224.
WARNING: Using actual aggregate bytes moved = 4593238016.
write 63.61 26214400 16.00 0.351773 68.52 0.001044 68.87 0
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 2576939417600.
WARNING: Using actual aggregate bytes moved = 4604821504.
write 64.13 26214400 16.00 0.003141 68.47 0.001192 68.48 1
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 2576956604416.
WARNING: Using actual aggregate bytes moved = 4598972416.
write 64.02 26214400 16.00 0.003272 68.51 0.001262 68.51 2
Max Write: 64.13 MiB/sec (67.25 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 64.13 63.61 63.92 0.23 68.61903 0 96 6 3 0 0 1 0 0 1 26843545600 16384 4593238016 MPIIO 0
Finished: Tue Oct 23 22:21:26 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 96 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12,isc17-c13,isc17-c14,isc17-c15,isc17-c18,isc17-c22,isc17-c01,isc17-c02,isc17-c03 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ tee -a ./output/COUNT:1#NN:16#PPN:6#API:MPIIO#T:16384.txt
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 96 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12,isc17-c13,isc17-c14,isc17-c15,isc17-c18,isc17-c22,isc17-c01,isc17-c02,isc17-c03 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 26843545600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Tue Oct 23 22:21:31 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 26843545600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 22:21:31 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 96 (6 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 25 GiB
aggregate filesize = 2400 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 6348816384.
read 88.22 26214400 16.00 0.358973 68.28 0.014673 68.63 0
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 6376144896.
read 89.06 26214400 16.00 0.017285 68.27 0.014640 68.28 1
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 6384926720.
read 89.18 26214400 16.00 0.017508 68.27 0.014639 68.28 2
Max Read: 89.18 MiB/sec (93.51 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 89.18 88.22 88.82 0.42 68.39482 0 96 6 3 0 0 1 0 0 1 26843545600 16384 6348816384 MPIIO 0
Finished: Tue Oct 23 22:24:56 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 32 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12384341 0xbcf855 0
6 12597602 0xc03962 0
0 12426004 0xbd9b14 0
2 12572333 0xbfd6ad 0
4 12514076 0xbef31c 0
7 12462706 0xbe2a72 0
1 12643984 0xc0ee90 0
3 12492282 0xbe9dfa 0

1618
output/COUNT:1#NN:16#PPN:6#API:POSIX#T:102400.txt Normal file
View File

File diff suppressed because it is too large Load Diff

1619
output/COUNT:1#NN:16#PPN:6#API:POSIX#T:16384.txt Normal file
View File

File diff suppressed because it is too large Load Diff

110
output/COUNT:1#NN:16#PPN:8#API:MPIIO#T:102400.txt Normal file
View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 8 -np 128 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12,isc17-c13,isc17-c14,isc17-c15,isc17-c18,isc17-c22,isc17-c01,isc17-c02,isc17-c03 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 20132659200 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:16#PPN:8#API:MPIIO#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Tue Oct 23 20:07:46 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 20132659200 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 20:07:46 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 128 (8 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 18.75 GiB
aggregate filesize = 2400 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 2576975052800.
WARNING: Using actual aggregate bytes moved = 27621580800.
write 422.91 19660800 100.00 0.500381 61.79 0.000697 62.29 0
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 2576963379200.
WARNING: Using actual aggregate bytes moved = 28443443200.
write 440.21 19660800 100.00 0.003456 61.62 0.000779 61.62 1
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 2576969932800.
WARNING: Using actual aggregate bytes moved = 27850752000.
write 432.11 19660800 100.00 0.004572 61.46 0.000876 61.47 2
Max Write: 440.21 MiB/sec (461.59 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 440.21 422.91 431.74 7.07 61.79198 0 128 8 3 0 0 1 0 0 1 20132659200 102400 27621580800 MPIIO 0
Finished: Tue Oct 23 20:10:53 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 8 -np 128 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12,isc17-c13,isc17-c14,isc17-c15,isc17-c18,isc17-c22,isc17-c01,isc17-c02,isc17-c03 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ tee -a ./output/COUNT:1#NN:16#PPN:8#API:MPIIO#T:102400.txt
+ /opt/ddn/mvapich/bin/mpiexec -ppn 8 -np 128 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12,isc17-c13,isc17-c14,isc17-c15,isc17-c18,isc17-c22,isc17-c01,isc17-c02,isc17-c03 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 20132659200 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Tue Oct 23 20:11:03 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 20132659200 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 20:11:03 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 128 (8 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 18.75 GiB
aggregate filesize = 2400 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 33230336000.
read 512.19 19660800 100.00 0.513790 61.36 0.002868 61.87 0
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 33932390400.
read 527.39 19660800 100.00 0.005589 61.36 0.002923 61.36 1
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 34252902400.
read 532.33 19660800 100.00 0.006677 61.36 0.004403 61.36 2
Max Read: 532.33 MiB/sec (558.19 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 532.33 512.19 523.97 8.57 61.53242 0 128 8 3 0 0 1 0 0 1 20132659200 102400 33230336000 MPIIO 0
Finished: Tue Oct 23 20:14:07 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 32 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12596066 0xc03362 0
0 12424459 0xbd950b 0
2 12570798 0xbfd0ae 0
4 12512537 0xbeed19 0
7 12461168 0xbe2470 0
1 12642446 0xc0e88e 0
3 12490743 0xbe97f7 0
5 12382810 0xbcf25a 0

110
output/COUNT:1#NN:16#PPN:8#API:MPIIO#T:16384.txt Normal file
View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 8 -np 128 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12,isc17-c13,isc17-c14,isc17-c15,isc17-c18,isc17-c22,isc17-c01,isc17-c02,isc17-c03 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 20132659200 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:16#PPN:8#API:MPIIO#T:16384.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Tue Oct 23 21:25:04 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 20132659200 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 21:25:04 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 128 (8 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 18.75 GiB
aggregate filesize = 2400 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 2576979804160.
WARNING: Using actual aggregate bytes moved = 4941348864.
write 68.08 19660800 16.00 0.553325 68.66 0.000669 69.22 0
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 2576976101376.
WARNING: Using actual aggregate bytes moved = 4916150272.
write 68.46 19660800 16.00 0.002607 68.48 0.000758 68.49 1
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 2576980328448.
WARNING: Using actual aggregate bytes moved = 4860739584.
write 67.77 19660800 16.00 0.003006 68.40 0.000871 68.40 2
Max Write: 68.46 MiB/sec (71.78 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 68.46 67.77 68.10 0.28 68.70114 0 128 8 3 0 0 1 0 0 1 20132659200 16384 4941348864 MPIIO 0
Finished: Tue Oct 23 21:28:31 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 8 -np 128 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12,isc17-c13,isc17-c14,isc17-c15,isc17-c18,isc17-c22,isc17-c01,isc17-c02,isc17-c03 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ tee -a ./output/COUNT:1#NN:16#PPN:8#API:MPIIO#T:16384.txt
+ /opt/ddn/mvapich/bin/mpiexec -ppn 8 -np 128 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12,isc17-c13,isc17-c14,isc17-c15,isc17-c18,isc17-c22,isc17-c01,isc17-c02,isc17-c03 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 20132659200 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Tue Oct 23 21:28:38 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 20132659200 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 21:28:38 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 128 (8 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 18.75 GiB
aggregate filesize = 2400 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 7226949632.
read 100.26 19660800 16.00 0.499991 68.24 0.001031 68.75 0
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 7283228672.
read 101.78 19660800 16.00 0.004498 68.24 0.001824 68.24 1
WARNING: Expected aggregate file size = 2576980377600.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 7291551744.
read 101.86 19660800 16.00 0.004573 68.26 0.001081 68.27 2
Max Read: 101.86 MiB/sec (106.81 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 101.86 100.26 101.30 0.74 68.41744 0 128 8 3 0 0 1 0 0 1 20132659200 16384 7226949632 MPIIO 0
Finished: Tue Oct 23 21:32:03 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 32 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12597072 0xc03750 0
0 12425474 0xbd9902 0
2 12571809 0xbfd4a1 0
4 12513543 0xbef107 0
7 12462179 0xbe2863 0
1 12643456 0xc0ec80 0
3 12491753 0xbe9be9 0
5 12383813 0xbcf645 0

2119
output/COUNT:1#NN:16#PPN:8#API:POSIX#T:102400.txt Normal file
View File

File diff suppressed because it is too large Load Diff

2126
output/COUNT:1#NN:16#PPN:8#API:POSIX#T:16384.txt Normal file
View File

File diff suppressed because it is too large Load Diff