julian.kunkel
/
ddn-ime-evaluation
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

NN:8 experiment results

This commit is contained in:
Eugen Betke 2018-10-24 07:42:09 +02:00
parent 2060b9b5f6
commit a351cab075
40 changed files with 14710 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

110
output/COUNT:1#NN:8#PPN:1#API:MPIIO#T:102400.txt Normal file
View File

@ -0,0 +1,110 @@
+ tee -a ./output/COUNT:1#NN:8#PPN:1#API:MPIIO#T:102400.txt
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 161061273600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 05:49:36 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 161061273600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 05:49:36 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 8 (1 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 150 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288470323200.
WARNING: Using actual aggregate bytes moved = 15616716800.
write 245.15 157286400 100.00 0.009737 60.74 0.000690 60.75 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288488243200.
WARNING: Using actual aggregate bytes moved = 15217766400.
write 238.95 157286400 100.00 0.001339 60.74 0.000383 60.74 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288469913600.
WARNING: Using actual aggregate bytes moved = 15483904000.
write 242.97 157286400 100.00 0.001689 60.77 0.000406 60.77 2
Max Write: 245.15 MiB/sec (257.06 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 245.15 238.95 242.36 2.57 60.75423 0 8 1 3 0 0 1 0 0 1 161061273600 102400 15616716800 MPIIO 0
Finished: Wed Oct 24 05:52:39 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ tee -a ./output/COUNT:1#NN:8#PPN:1#API:MPIIO#T:102400.txt
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 161061273600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 05:52:42 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 161061273600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 05:52:42 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 8 (1 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 150 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 5620838400.
read 88.26 157286400 100.00 0.012851 60.72 0.000690 60.74 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 5649715200.
read 88.73 157286400 100.00 0.001083 60.72 0.000705 60.72 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 5670297600.
read 89.07 157286400 100.00 0.001049 60.71 0.000719 60.71 2
Max Read: 89.07 MiB/sec (93.39 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 89.07 88.26 88.69 0.33 60.72357 0 8 1 3 0 0 1 0 0 1 161061273600 102400 5620838400 MPIIO 0
Finished: Wed Oct 24 05:55:44 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494351 0xbea60f 0
5 12386422 0xbd0076 0
6 12599666 0xc04172 0
0 12428065 0xbda321 0
2 12574412 0xbfdecc 0
4 12516141 0xbefb2d 0
7 12464771 0xbe3283 0
1 12646052 0xc0f6a4 0

110
output/COUNT:1#NN:8#PPN:1#API:MPIIO#T:1048576.txt Normal file
View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 161061273600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:1#API:MPIIO#T:1048576.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 04:40:14 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 161061273600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 04:40:14 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 8 (1 per node)
repetitions = 3
xfersize = 1 MiB
blocksize = 150 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288461877248.
WARNING: Using actual aggregate bytes moved = 162994847744.
write 2585.87 157286400 1024.00 0.013872 60.10 0.000374 60.11 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288479703040.
WARNING: Using actual aggregate bytes moved = 156644671488.
write 2482.69 157286400 1024.00 0.089696 60.08 0.000351 60.17 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288488091648.
WARNING: Using actual aggregate bytes moved = 159087853568.
write 2520.91 157286400 1024.00 0.087149 60.10 0.000404 60.18 2
Max Write: 2585.87 MiB/sec (2711.48 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 2585.87 2482.69 2529.82 42.59 60.15621 0 8 1 3 0 0 1 0 0 1 161061273600 1048576 162994847744 MPIIO 0
Finished: Wed Oct 24 04:43:18 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ tee -a ./output/COUNT:1#NN:8#PPN:1#API:MPIIO#T:1048576.txt
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 161061273600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 04:43:25 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 161061273600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 04:43:25 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 8 (1 per node)
repetitions = 3
xfersize = 1 MiB
blocksize = 150 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 31960596480.
read 506.65 157286400 1024.00 0.076376 60.08 0.000570 60.16 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 33031192576.
read 524.23 157286400 1024.00 0.001699 60.09 0.000530 60.09 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 33508294656.
read 531.82 157286400 1024.00 0.000879 60.09 0.000547 60.09 2
Max Read: 531.82 MiB/sec (557.66 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 531.82 506.65 520.90 10.54 60.11243 0 8 1 3 0 0 1 0 0 1 161061273600 1048576 31960596480 MPIIO 0
Finished: Wed Oct 24 04:46:26 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12599151 0xc03f6f 0
0 12427557 0xbda125 0
2 12573898 0xbfdcca 0
4 12515629 0xbef92d 0
7 12464253 0xbe307d 0
1 12645541 0xc0f4a5 0
3 12493839 0xbea40f 0
5 12385907 0xbcfe73 0

110
output/COUNT:1#NN:8#PPN:1#API:MPIIO#T:10485760.txt Normal file
View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 161061273600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:1#API:MPIIO#T:10485760.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 03:33:06 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 161061273600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 03:33:06 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 8 (1 per node)
repetitions = 3
xfersize = 10 MiB
blocksize = 150 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288123187200.
WARNING: Using actual aggregate bytes moved = 105623060480.
write 1673.14 157286400 10240 0.014375 60.19 0.000682 60.20 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288228044800.
WARNING: Using actual aggregate bytes moved = 107091066880.
write 1690.88 157286400 10240 0.051064 60.35 0.000771 60.40 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288186101760.
WARNING: Using actual aggregate bytes moved = 107374182400.
write 1702.76 157286400 10240 0.054793 60.08 0.000918 60.14 2
Max Write: 1702.76 MiB/sec (1785.47 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 1702.76 1673.14 1688.92 12.17 60.24755 0 8 1 3 0 0 1 0 0 1 161061273600 10485760 105623060480 MPIIO 0
Finished: Wed Oct 24 03:36:10 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 161061273600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
+ tee -a ./output/COUNT:1#NN:8#PPN:1#API:MPIIO#T:10485760.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 03:36:17 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 161061273600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 03:36:17 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 8 (1 per node)
repetitions = 3
xfersize = 10 MiB
blocksize = 150 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 96563363840.
read 1531.88 157286400 10240 0.069546 60.05 0.004064 60.12 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 98964602880.
read 1570.90 157286400 10240 0.004699 60.08 0.004031 60.08 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 98744401920.
read 1567.39 157286400 10240 0.004338 60.08 0.004016 60.08 2
Max Read: 1570.90 MiB/sec (1647.20 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 1570.90 1531.88 1556.72 17.62 60.09226 0 8 1 3 0 0 1 0 0 1 161061273600 10485760 96563363840 MPIIO 0
Finished: Wed Oct 24 03:39:17 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12493327 0xbea20f 0
5 12385391 0xbcfc6f 0
6 12598641 0xc03d71 0
0 12427045 0xbd9f25 0
2 12573382 0xbfdac6 0
4 12515119 0xbef72f 0
7 12463739 0xbe2e7b 0
1 12645027 0xc0f2a3 0

110
output/COUNT:1#NN:8#PPN:1#API:MPIIO#T:16384.txt Normal file
View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 161061273600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:1#API:MPIIO#T:16384.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 07:03:49 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 161061273600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 07:03:49 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 8 (1 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 150 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288482652160.
WARNING: Using actual aggregate bytes moved = 2862219264.
write 42.16 157286400 16.00 0.012217 64.72 0.000924 64.74 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288479277056.
WARNING: Using actual aggregate bytes moved = 2864889856.
write 42.17 157286400 16.00 0.001418 64.78 0.000353 64.78 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288487223296.
WARNING: Using actual aggregate bytes moved = 2879717376.
write 42.40 157286400 16.00 0.001358 64.78 0.000395 64.78 2
Max Write: 42.40 MiB/sec (44.45 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 42.40 42.16 42.24 0.11 64.76696 0 8 1 3 0 0 1 0 0 1 161061273600 16384 2862219264 MPIIO 0
Finished: Wed Oct 24 07:07:04 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 161061273600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
+ tee -a ./output/COUNT:1#NN:8#PPN:1#API:MPIIO#T:16384.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 07:07:06 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 161061273600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 07:07:06 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 8 (1 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 150 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 864337920.
read 12.73 157286400 16.00 0.006994 64.72 0.000738 64.73 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 997228544.
read 14.70 157286400 16.00 0.001774 64.71 0.000756 64.71 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 1070743552.
read 15.78 157286400 16.00 0.001112 64.71 0.000835 64.71 2
Max Read: 15.78 MiB/sec (16.55 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 15.78 12.73 14.40 1.26 64.71666 0 8 1 3 0 0 1 0 0 1 161061273600 16384 864337920 MPIIO 0
Finished: Wed Oct 24 07:10:20 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12465283 0xbe3483 0
1 12646569 0xc0f8a9 0
3 12494870 0xbea816 0
5 12386930 0xbd0272 0
6 12600174 0xc0436e 0
0 12428580 0xbda524 0
2 12574926 0xbfe0ce 0
4 12516652 0xbefd2c 0

219
output/COUNT:1#NN:8#PPN:1#API:POSIX#T:102400.txt Normal file
View File

@ -0,0 +1,219 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 161061273600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:1#API:POSIX#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 05:41:43 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 161061273600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 05:41:43 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 8 (1 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 150 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288465817600.
WARNING: Using actual aggregate bytes moved = 83866828800.
write 881.60 157286400 100.00 0.000598 90.72 0.000482 90.72 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288488550400.
WARNING: Using actual aggregate bytes moved = 80570163200.
write 808.36 157286400 100.00 0.000983 95.05 0.000788 95.05 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288469094400.
WARNING: Using actual aggregate bytes moved = 83156172800.
write 857.98 157286400 100.00 0.000992 92.43 0.000664 92.43 2
Max Write: 881.60 MiB/sec (924.43 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 881.60 808.36 849.31 30.52 92.73608 0 8 1 3 1 0 1 0 0 1 161061273600 102400 83866828800 POSIX 0
Finished: Wed Oct 24 05:46:25 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ tee -a ./output/COUNT:1#NN:8#PPN:1#API:POSIX#T:102400.txt
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 161061273600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 05:46:36 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 161061273600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 05:46:36 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/indread8/file
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 8 (1 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 150 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288490188800.
WARNING: Using actual aggregate bytes moved = 4965376000.
read 78.83 157286400 100.00 0.000876 60.07 0.000693 60.07 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288490188800.
WARNING: Using actual aggregate bytes moved = 5366579200.
read 85.21 157286400 100.00 0.000662 60.06 0.000757 60.06 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288490188800.
WARNING: Using actual aggregate bytes moved = 5337292800.
read 84.75 157286400 100.00 0.000726 60.06 0.000836 60.06 2
Max Read: 85.21 MiB/sec (89.35 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 85.21 78.83 82.93 2.90 60.06219 0 8 1 3 1 0 1 0 0 1 161061273600 102400 4965376000 POSIX 0
Finished: Wed Oct 24 05:49:36 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write.00000006
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12574408 0xbfdec8 0
4 12516137 0xbefb29 0
7 12464768 0xbe3280 0
1 12646049 0xc0f6a1 0
3 12494348 0xbea60c 0
5 12386419 0xbd0073 0
6 12599663 0xc0416f 0
0 12428062 0xbda31e 0
/esfs/jtacquaviva/ioperf/file_write.00000002
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12574405 0xbfdec5 0
1 12646046 0xc0f69e 0
7 12464765 0xbe327d 0
5 12386417 0xbd0071 0
4 12516135 0xbefb27 0
3 12494347 0xbea60b 0
0 12428060 0xbda31c 0
6 12599662 0xc0416e 0
/esfs/jtacquaviva/ioperf/file_write.00000000
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428058 0xbda31a 0
7 12464764 0xbe327c 0
2 12574406 0xbfdec6 0
3 12494345 0xbea609 0
4 12516134 0xbefb26 0
1 12646048 0xc0f6a0 0
5 12386418 0xbd0072 0
6 12599661 0xc0416d 0
/esfs/jtacquaviva/ioperf/file_write.00000004
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464763 0xbe327b 0
3 12494344 0xbea608 0
1 12646047 0xc0f69f 0
5 12386416 0xbd0070 0
6 12599660 0xc0416c 0
4 12516138 0xbefb2a 0
0 12428061 0xbda31d 0
/esfs/jtacquaviva/ioperf/file_write.00000007
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12646043 0xc0f69b 0
6 12599658 0xc0416a 0
0 12428057 0xbda319 0
2 12574404 0xbfdec4 0
3 12494342 0xbea606 0
5 12386415 0xbd006f 0
4 12516133 0xbefb25 0
7 12464766 0xbe327e 0
/esfs/jtacquaviva/ioperf/file_write.00000003
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12646042 0xc0f69a 0
6 12599656 0xc04168 0
2 12574402 0xbfdec2 0
4 12516131 0xbefb23 0
3 12494341 0xbea605 0
0 12428056 0xbda318 0
5 12386413 0xbd006d 0
7 12464762 0xbe327a 0
/esfs/jtacquaviva/ioperf/file_write.00000005
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12386412 0xbd006c 0
0 12428055 0xbda317 0
7 12464761 0xbe3279 0
6 12599657 0xc04169 0
2 12574403 0xbfdec3 0
1 12646044 0xc0f69c 0
3 12494343 0xbea607 0
4 12516132 0xbefb24 0
/esfs/jtacquaviva/ioperf/file_write.00000001
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12386414 0xbd006e 0
1 12646045 0xc0f69d 0
6 12599659 0xc0416b 0
2 12574407 0xbfdec7 0
3 12494346 0xbea60a 0
4 12516136 0xbefb28 0
0 12428059 0xbda31b 0
7 12464767 0xbe327f 0

216
output/COUNT:1#NN:8#PPN:1#API:POSIX#T:1048576.txt Normal file
View File

@ -0,0 +1,216 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 161061273600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:1#API:POSIX#T:1048576.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 04:33:14 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 161061273600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 04:33:14 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 8 (1 per node)
repetitions = 3
xfersize = 1 MiB
blocksize = 150 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288445100032.
WARNING: Using actual aggregate bytes moved = 337465311232.
write 4706 157286400 1024.00 0.000716 68.38 0.000429 68.38 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288446148608.
WARNING: Using actual aggregate bytes moved = 349623549952.
write 4889 157286400 1024.00 0.000895 68.20 0.000514 68.20 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288481800192.
WARNING: Using actual aggregate bytes moved = 338388058112.
write 4738 157286400 1024.00 0.001086 68.11 0.000644 68.11 2
Max Write: 4888.95 MiB/sec (5126.43 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 4888.95 4706.45 4777.83 79.63 68.23050 0 8 1 3 1 0 1 0 0 1 161061273600 1048576 337465311232 POSIX 0
Finished: Wed Oct 24 04:36:51 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 161061273600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
+ tee -a ./output/COUNT:1#NN:8#PPN:1#API:POSIX#T:1048576.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 04:37:13 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 161061273600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 04:37:13 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/indread8/file
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 8 (1 per node)
repetitions = 3
xfersize = 1 MiB
blocksize = 150 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288490188800.
WARNING: Using actual aggregate bytes moved = 39415971840.
read 626.28 157286400 1024.00 0.000842 60.02 0.000558 60.02 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288490188800.
WARNING: Using actual aggregate bytes moved = 40653291520.
read 645.99 157286400 1024.00 0.000575 60.02 0.000583 60.02 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288490188800.
WARNING: Using actual aggregate bytes moved = 41420849152.
read 658.16 157286400 1024.00 0.000618 60.02 0.000540 60.02 2
Max Read: 658.16 MiB/sec (690.14 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 658.16 626.28 643.48 13.14 60.01856 0 8 1 3 1 0 1 0 0 1 161061273600 1048576 39415971840 POSIX 0
Finished: Wed Oct 24 04:40:13 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write.00000006
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464243 0xbe3073 0
5 12385897 0xbcfe69 0
1 12645532 0xc0f49c 0
4 12515621 0xbef925 0
2 12573890 0xbfdcc2 0
0 12427550 0xbda11e 0
6 12599144 0xc03f68 0
3 12493831 0xbea407 0
/esfs/jtacquaviva/ioperf/file_write.00000002
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12427551 0xbda11f 0
5 12385900 0xbcfe6c 0
4 12515623 0xbef927 0
2 12573892 0xbfdcc4 0
1 12645535 0xc0f49f 0
6 12599147 0xc03f6b 0
3 12493835 0xbea40b 0
/esfs/jtacquaviva/ioperf/file_write.00000000
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12599142 0xc03f66 0
1 12645531 0xc0f49b 0
4 12515620 0xbef924 0
2 12573889 0xbfdcc1 0
0 12427549 0xbda11d 0
5 12385899 0xbcfe6b 0
7 12464248 0xbe3078 0
3 12493833 0xbea409 0
/esfs/jtacquaviva/ioperf/file_write.00000004
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12427552 0xbda120 0
3 12493832 0xbea408 0
7 12464249 0xbe3079 0
5 12385903 0xbcfe6f 0
6 12599146 0xc03f6a 0
2 12573895 0xbfdcc7 0
1 12645538 0xc0f4a2 0
4 12515626 0xbef92a 0
/esfs/jtacquaviva/ioperf/file_write.00000007
lmm_stripe_count: 6
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12645533 0xc0f49d 0
7 12464244 0xbe3074 0
3 12493830 0xbea406 0
4 12515622 0xbef926 0
5 12385901 0xbcfe6d 0
2 12573893 0xbfdcc5 0
/esfs/jtacquaviva/ioperf/file_write.00000003
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573888 0xbfdcc0 0
3 12493829 0xbea405 0
7 12464245 0xbe3075 0
5 12385902 0xbcfe6e 0
6 12599145 0xc03f69 0
1 12645537 0xc0f4a1 0
4 12515625 0xbef929 0
0 12427554 0xbda122 0
/esfs/jtacquaviva/ioperf/file_write.00000005
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464246 0xbe3076 0
2 12573891 0xbfdcc3 0
0 12427553 0xbda121 0
1 12645536 0xc0f4a0 0
6 12599148 0xc03f6c 0
5 12385904 0xbcfe70 0
3 12493836 0xbea40c 0
/esfs/jtacquaviva/ioperf/file_write.00000001
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12427548 0xbda11c 0
5 12385898 0xbcfe6a 0
1 12645534 0xc0f49e 0
6 12599143 0xc03f67 0
7 12464247 0xbe3077 0
3 12493834 0xbea40a 0
4 12515624 0xbef928 0
2 12573894 0xbfdcc6 0

217
output/COUNT:1#NN:8#PPN:1#API:POSIX#T:10485760.txt Normal file
View File

@ -0,0 +1,217 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 161061273600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:1#API:POSIX#T:10485760.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 03:26:07 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 161061273600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 03:26:07 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 8 (1 per node)
repetitions = 3
xfersize = 10 MiB
blocksize = 150 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288490188800.
WARNING: Using actual aggregate bytes moved = 384177274880.
write 5452 157286400 10240 0.000488 67.20 0.000300 67.20 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288269987840.
WARNING: Using actual aggregate bytes moved = 383233556480.
write 5361 157286400 10240 0.002442 68.17 0.000870 68.18 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288060272640.
WARNING: Using actual aggregate bytes moved = 382551982080.
write 5416 157286400 10240 0.001030 67.36 0.000696 67.37 2
Max Write: 5452.10 MiB/sec (5716.94 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 5452.10 5360.72 5409.48 37.56 67.58117 0 8 1 3 1 0 1 0 0 1 161061273600 10485760 384177274880 POSIX 0
Finished: Wed Oct 24 03:29:42 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ tee -a ./output/COUNT:1#NN:8#PPN:1#API:POSIX#T:10485760.txt
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 161061273600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 03:30:04 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 161061273600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 03:30:04 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/indread8/file
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 8 (1 per node)
repetitions = 3
xfersize = 10 MiB
blocksize = 150 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288490188800.
WARNING: Using actual aggregate bytes moved = 45329940480.
read 719.44 157286400 10240 0.000782 60.09 0.000533 60.09 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288490188800.
WARNING: Using actual aggregate bytes moved = 46472888320.
read 737.10 157286400 10240 0.000535 60.13 0.000640 60.13 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288490188800.
WARNING: Using actual aggregate bytes moved = 49377443840.
read 783.26 157286400 10240 0.000899 60.12 0.000746 60.12 2
Max Read: 783.26 MiB/sec (821.30 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 783.26 719.44 746.60 26.90 60.11220 0 8 1 3 1 0 1 0 0 1 161061273600 10485760 45329940480 POSIX 0
Finished: Wed Oct 24 03:33:05 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write.00000006
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12598633 0xc03d69 0
0 12427039 0xbd9f1f 0
2 12573378 0xbfdac2 0
7 12463736 0xbe2e78 0
4 12515115 0xbef72b 0
5 12385388 0xbcfc6c 0
1 12645024 0xc0f2a0 0
/esfs/jtacquaviva/ioperf/file_write.00000002
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12463730 0xbe2e72 0
0 12427035 0xbd9f1b 0
4 12515109 0xbef725 0
3 12493319 0xbea207 0
6 12598632 0xc03d68 0
1 12645018 0xc0f29a 0
5 12385383 0xbcfc67 0
2 12573374 0xbfdabe 0
/esfs/jtacquaviva/ioperf/file_write.00000000
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573376 0xbfdac0 0
4 12515112 0xbef728 0
1 12645022 0xc0f29e 0
5 12385387 0xbcfc6b 0
6 12598638 0xc03d6e 0
3 12493324 0xbea20c 0
0 12427041 0xbd9f21 0
/esfs/jtacquaviva/ioperf/file_write.00000004
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12463734 0xbe2e76 0
1 12645020 0xc0f29c 0
3 12493321 0xbea209 0
4 12515114 0xbef72a 0
5 12385386 0xbcfc6a 0
6 12598636 0xc03d6c 0
0 12427040 0xbd9f20 0
/esfs/jtacquaviva/ioperf/file_write.00000007
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12493318 0xbea206 0
2 12573372 0xbfdabc 0
1 12645017 0xc0f299 0
0 12427036 0xbd9f1c 0
4 12515110 0xbef726 0
5 12385384 0xbcfc68 0
7 12463733 0xbe2e75 0
6 12598634 0xc03d6a 0
/esfs/jtacquaviva/ioperf/file_write.00000003
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12493320 0xbea208 0
5 12385382 0xbcfc66 0
7 12463732 0xbe2e74 0
0 12427037 0xbd9f1d 0
2 12573375 0xbfdabf 0
6 12598635 0xc03d6b 0
1 12645021 0xc0f29d 0
4 12515113 0xbef729 0
/esfs/jtacquaviva/ioperf/file_write.00000005
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573377 0xbfdac1 0
5 12385385 0xbcfc69 0
7 12463735 0xbe2e77 0
3 12493323 0xbea20b 0
6 12598637 0xc03d6d 0
1 12645023 0xc0f29f 0
4 12515116 0xbef72c 0
0 12427042 0xbd9f22 0
/esfs/jtacquaviva/ioperf/file_write.00000001
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12598631 0xc03d67 0
7 12463731 0xbe2e73 0
5 12385381 0xbcfc65 0
2 12573373 0xbfdabd 0
4 12515111 0xbef727 0
0 12427038 0xbd9f1e 0
1 12645019 0xc0f29b 0
3 12493322 0xbea20a 0

220
output/COUNT:1#NN:8#PPN:1#API:POSIX#T:16384.txt Normal file
View File

@ -0,0 +1,220 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 161061273600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:1#API:POSIX#T:16384.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 06:54:51 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 161061273600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 06:54:51 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 8 (1 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 150 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288487698432.
WARNING: Using actual aggregate bytes moved = 31234883584.
write 240.77 157286400 16.00 0.000595 123.72 0.000485 123.72 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288485863424.
WARNING: Using actual aggregate bytes moved = 26906968064.
write 224.12 157286400 16.00 0.000990 114.50 0.000522 114.50 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288469069824.
WARNING: Using actual aggregate bytes moved = 27960279040.
write 241.93 157286400 16.00 0.001059 110.22 0.000543 110.22 2
Max Write: 241.93 MiB/sec (253.68 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 241.93 224.12 235.60 8.14 116.14498 0 8 1 3 1 0 1 0 0 1 161061273600 16384 31234883584 POSIX 0
Finished: Wed Oct 24 07:00:41 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 161061273600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
+ tee -a ./output/COUNT:1#NN:8#PPN:1#API:POSIX#T:16384.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 07:00:47 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 161061273600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 07:00:47 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/indread8/file
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 8 (1 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 150 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288490188800.
WARNING: Using actual aggregate bytes moved = 709541888.
read 11.16 157286400 16.00 0.000829 60.62 0.000619 60.62 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288490188800.
WARNING: Using actual aggregate bytes moved = 1032896512.
read 16.25 157286400 16.00 0.000626 60.62 0.000688 60.62 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288490188800.
WARNING: Using actual aggregate bytes moved = 1021591552.
read 16.07 157286400 16.00 0.000473 60.62 0.000740 60.62 2
Max Read: 16.25 MiB/sec (17.04 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 16.25 11.16 14.49 2.36 60.62368 0 8 1 3 1 0 1 0 0 1 161061273600 16384 709541888 POSIX 0
Finished: Wed Oct 24 07:03:49 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write.00000006
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12516643 0xbefd23 0
0 12428571 0xbda51b 0
6 12600165 0xc04365 0
3 12494863 0xbea80f 0
1 12646562 0xc0f8a2 0
5 12386924 0xbd026c 0
2 12574921 0xbfe0c9 0
7 12465278 0xbe347e 0
/esfs/jtacquaviva/ioperf/file_write.00000002
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12574918 0xbfe0c6 0
6 12600167 0xc04367 0
7 12465277 0xbe347d 0
4 12516646 0xbefd26 0
1 12646563 0xc0f8a3 0
3 12494867 0xbea813 0
0 12428577 0xbda521 0
5 12386927 0xbd026f 0
/esfs/jtacquaviva/ioperf/file_write.00000000
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12465276 0xbe347c 0
2 12574920 0xbfe0c8 0
0 12428575 0xbda51f 0
3 12494866 0xbea812 0
5 12386926 0xbd026e 0
4 12516648 0xbefd28 0
1 12646566 0xc0f8a6 0
6 12600170 0xc0436a 0
/esfs/jtacquaviva/ioperf/file_write.00000004
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428570 0xbda51a 0
3 12494860 0xbea80c 0
2 12574916 0xbfe0c4 0
5 12386920 0xbd0268 0
4 12516642 0xbefd22 0
6 12600163 0xc04363 0
1 12646559 0xc0f89f 0
7 12465273 0xbe3479 0
/esfs/jtacquaviva/ioperf/file_write.00000007
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494861 0xbea80d 0
1 12646560 0xc0f8a0 0
7 12465274 0xbe347a 0
4 12516644 0xbefd24 0
2 12574917 0xbfe0c5 0
0 12428572 0xbda51c 0
6 12600166 0xc04366 0
5 12386923 0xbd026b 0
/esfs/jtacquaviva/ioperf/file_write.00000003
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12646561 0xc0f8a1 0
7 12465275 0xbe347b 0
5 12386921 0xbd0269 0
3 12494864 0xbea810 0
4 12516647 0xbefd27 0
2 12574922 0xbfe0ca 0
0 12428576 0xbda520 0
6 12600169 0xc04369 0
/esfs/jtacquaviva/ioperf/file_write.00000005
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12600164 0xc04364 0
5 12386922 0xbd026a 0
0 12428574 0xbda51e 0
3 12494865 0xbea811 0
1 12646564 0xc0f8a4 0
7 12465279 0xbe347f 0
2 12574923 0xbfe0cb 0
4 12516649 0xbefd29 0
/esfs/jtacquaviva/ioperf/file_write.00000001
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494862 0xbea80e 0
4 12516645 0xbefd25 0
0 12428573 0xbda51d 0
2 12574919 0xbfe0c7 0
6 12600168 0xc04368 0
5 12386925 0xbd026d 0
1 12646565 0xc0f8a5 0
7 12465280 0xbe3480 0

110
output/COUNT:1#NN:8#PPN:2#API:MPIIO#T:102400.txt Normal file
View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 80530636800 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:2#API:MPIIO#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 06:17:34 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 80530636800 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 06:17:34 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 16 (2 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 75 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288487628800.
WARNING: Using actual aggregate bytes moved = 18781184000.
write 295.02 78643200 100.00 0.019334 60.69 0.000530 60.71 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288482713600.
WARNING: Using actual aggregate bytes moved = 18567884800.
write 291.60 78643200 100.00 0.004748 60.72 0.000514 60.73 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288483635200.
WARNING: Using actual aggregate bytes moved = 18424524800.
write 289.30 78643200 100.00 0.001615 60.73 0.000613 60.74 2
Max Write: 295.02 MiB/sec (309.35 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 295.02 289.30 291.97 2.35 60.72489 0 16 2 3 0 0 1 0 0 1 80530636800 102400 18781184000 MPIIO 0
Finished: Wed Oct 24 06:20:37 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ tee -a ./output/COUNT:1#NN:8#PPN:2#API:MPIIO#T:102400.txt
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 80530636800 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 06:20:44 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 80530636800 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 06:20:44 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 16 (2 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 75 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 10075750400.
read 158.23 78643200 100.00 0.021584 60.70 0.007377 60.73 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 10156134400.
read 159.58 78643200 100.00 0.001522 60.69 0.000973 60.69 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 10192384000.
read 160.14 78643200 100.00 0.001574 60.70 0.001101 60.70 2
Max Read: 160.14 MiB/sec (167.92 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 160.14 158.23 159.32 0.80 60.70752 0 16 2 3 0 0 1 0 0 1 80530636800 102400 10075750400 MPIIO 0
Finished: Wed Oct 24 06:23:46 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494550 0xbea6d6 0
5 12386617 0xbd0139 0
6 12599857 0xc04231 0
0 12428261 0xbda3e5 0
2 12574609 0xbfdf91 0
4 12516335 0xbefbef 0
7 12464965 0xbe3345 0
1 12646248 0xc0f768 0

110
output/COUNT:1#NN:8#PPN:2#API:MPIIO#T:1048576.txt Normal file
View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 80530636800 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:2#API:MPIIO#T:1048576.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 05:06:45 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 80530636800 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 05:06:45 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 16 (2 per node)
repetitions = 3
xfersize = 1 MiB
blocksize = 75 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288479703040.
WARNING: Using actual aggregate bytes moved = 210129387520.
write 3330.68 78643200 1024.00 0.021551 60.14 0.000686 60.17 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288484945920.
WARNING: Using actual aggregate bytes moved = 204985073664.
write 3242.99 78643200 1024.00 0.166736 60.11 0.000621 60.28 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288479703040.
WARNING: Using actual aggregate bytes moved = 207656845312.
write 3293.97 78643200 1024.00 0.023080 60.10 0.000735 60.12 2
Max Write: 3330.68 MiB/sec (3492.47 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 3330.68 3242.99 3289.21 35.96 60.18935 0 16 2 3 0 0 1 0 0 1 80530636800 1048576 210129387520 MPIIO 0
Finished: Wed Oct 24 05:09:51 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ tee -a ./output/COUNT:1#NN:8#PPN:2#API:MPIIO#T:1048576.txt
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 80530636800 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 05:10:07 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 80530636800 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 05:10:07 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 16 (2 per node)
repetitions = 3
xfersize = 1 MiB
blocksize = 75 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 49119494144.
read 777.58 78643200 1024.00 0.154625 60.09 0.001569 60.24 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 51002736640.
read 809.35 78643200 1024.00 0.001924 60.10 0.001515 60.10 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 51521781760.
read 817.73 78643200 1024.00 0.001927 60.09 0.001569 60.09 2
Max Read: 817.73 MiB/sec (857.45 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 817.73 777.58 801.55 17.29 60.14263 0 16 2 3 0 0 1 0 0 1 80530636800 1048576 49119494144 MPIIO 0
Finished: Wed Oct 24 05:13:08 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12386103 0xbcff37 0
6 12599346 0xc04032 0
0 12427750 0xbda1e6 0
2 12574096 0xbfdd90 0
4 12515825 0xbef9f1 0
7 12464450 0xbe3142 0
1 12645738 0xc0f56a 0
3 12494036 0xbea4d4 0

110
output/COUNT:1#NN:8#PPN:2#API:MPIIO#T:10485760.txt Normal file
View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 80530636800 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:2#API:MPIIO#T:10485760.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 03:59:44 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 80530636800 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 03:59:44 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 16 (2 per node)
repetitions = 3
xfersize = 10 MiB
blocksize = 75 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288448245760.
WARNING: Using actual aggregate bytes moved = 131512401920.
write 2072.03 78643200 10240 0.014707 60.51 0.000592 60.53 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288353873920.
WARNING: Using actual aggregate bytes moved = 131082485760.
write 2073.23 78643200 10240 0.102562 60.19 0.000437 60.30 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288490188800.
WARNING: Using actual aggregate bytes moved = 125944463360.
write 1988.73 78643200 10240 0.141789 60.25 0.000477 60.40 2
Max Write: 2073.23 MiB/sec (2173.94 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 2073.23 1988.73 2044.66 39.55 60.40748 0 16 2 3 0 0 1 0 0 1 80530636800 10485760 131512401920 MPIIO 0
Finished: Wed Oct 24 04:02:50 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ tee -a ./output/COUNT:1#NN:8#PPN:2#API:MPIIO#T:10485760.txt
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 80530636800 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 04:03:05 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 80530636800 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 04:03:05 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 16 (2 per node)
repetitions = 3
xfersize = 10 MiB
blocksize = 75 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 134962216960.
read 2139.43 78643200 10240 0.081175 60.08 0.007885 60.16 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 138013573120.
read 2190.36 78643200 10240 0.003115 60.09 0.002665 60.09 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 139901009920.
read 2221.11 78643200 10240 0.003155 60.07 0.002717 60.07 2
Max Read: 2221.11 MiB/sec (2329.01 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 2221.11 2139.43 2183.63 33.69 60.10681 0 16 2 3 0 0 1 0 0 1 80530636800 10485760 134962216960 MPIIO 0
Finished: Wed Oct 24 04:06:06 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12598833 0xc03e31 0
0 12427240 0xbd9fe8 0
2 12573579 0xbfdb8b 0
4 12515313 0xbef7f1 0
7 12463935 0xbe2f3f 0
1 12645224 0xc0f368 0
3 12493522 0xbea2d2 0
5 12385588 0xbcfd34 0

110
output/COUNT:1#NN:8#PPN:2#API:MPIIO#T:16384.txt Normal file
View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 80530636800 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:2#API:MPIIO#T:16384.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 07:35:02 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 80530636800 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 07:35:02 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 16 (2 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 75 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288483028992.
WARNING: Using actual aggregate bytes moved = 3432415232.
write 50.85 78643200 16.00 0.017243 64.35 0.005945 64.37 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288482881536.
WARNING: Using actual aggregate bytes moved = 3391700992.
write 50.21 78643200 16.00 0.001709 64.42 0.000566 64.42 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288478015488.
WARNING: Using actual aggregate bytes moved = 3398025216.
write 50.14 78643200 16.00 0.001647 64.62 0.000643 64.62 2
Max Write: 50.85 MiB/sec (53.32 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 50.85 50.14 50.40 0.32 64.47118 0 16 2 3 0 0 1 0 0 1 80530636800 16384 3432415232 MPIIO 0
Finished: Wed Oct 24 07:38:16 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ tee -a ./output/COUNT:1#NN:8#PPN:2#API:MPIIO#T:16384.txt
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 80530636800 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 07:38:19 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 80530636800 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 07:38:19 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 16 (2 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 75 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 1962360832.
read 29.07 78643200 16.00 0.014638 64.36 0.001892 64.37 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 1974894592.
read 29.27 78643200 16.00 0.002467 64.35 0.001945 64.36 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 1981202432.
read 29.36 78643200 16.00 0.002380 64.36 0.002002 64.36 2
Max Read: 29.36 MiB/sec (30.78 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 29.36 29.07 29.23 0.12 64.36194 0 16 2 3 0 0 1 0 0 1 80530636800 16384 1962360832 MPIIO 0
Finished: Wed Oct 24 07:41:32 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12387130 0xbd033a 0
6 12600376 0xc04438 0
0 12428779 0xbda5eb 0
2 12575126 0xbfe196 0
4 12516852 0xbefdf4 0
7 12465488 0xbe3550 0
1 12646767 0xc0f96f 0
3 12495071 0xbea8df 0

347
output/COUNT:1#NN:8#PPN:2#API:POSIX#T:102400.txt Normal file
View File

@ -0,0 +1,347 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 80530636800 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:2#API:POSIX#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 06:09:44 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 80530636800 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 06:09:44 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 16 (2 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 75 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288446668800.
WARNING: Using actual aggregate bytes moved = 84816793600.
write 906.70 78643200 100.00 0.000991 89.21 0.000457 89.21 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288423731200.
WARNING: Using actual aggregate bytes moved = 74972160000.
write 776.88 78643200 100.00 0.001311 92.03 0.001350 92.03 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288459468800.
WARNING: Using actual aggregate bytes moved = 77627289600.
write 798.18 78643200 100.00 0.001265 92.75 0.000567 92.75 2
Max Write: 906.70 MiB/sec (950.75 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 906.70 776.88 827.25 56.85 91.33173 0 16 2 3 1 0 1 0 0 1 80530636800 102400 84816793600 POSIX 0
Finished: Wed Oct 24 06:14:21 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ tee -a ./output/COUNT:1#NN:8#PPN:2#API:POSIX#T:102400.txt
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 80530636800 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 06:14:33 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 80530636800 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 06:14:33 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/indread8/file
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 16 (2 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 75 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 2576980377600.
WARNING: Using actual aggregate bytes moved = 11614617600.
read 184.49 78643200 100.00 0.001346 60.04 0.001002 60.04 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 2576980377600.
WARNING: Using actual aggregate bytes moved = 11714048000.
read 186.08 78643200 100.00 0.001052 60.04 0.001039 60.04 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 2576980377600.
WARNING: Using actual aggregate bytes moved = 11938304000.
read 189.62 78643200 100.00 0.001054 60.04 0.001043 60.04 2
Max Read: 189.62 MiB/sec (198.83 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 189.62 184.49 186.73 2.15 60.03849 0 16 2 3 1 0 1 0 0 1 80530636800 102400 11614617600 POSIX 0
Finished: Wed Oct 24 06:17:33 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write.00000006
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12646231 0xc0f757 0
3 12494534 0xbea6c6 0
6 12599842 0xc04222 0
0 12428245 0xbda3d5 0
2 12574594 0xbfdf82 0
5 12386604 0xbd012c 0
7 12464952 0xbe3338 0
4 12516324 0xbefbe4 0
/esfs/jtacquaviva/ioperf/file_write.00000002
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428243 0xbda3d3 0
4 12516317 0xbefbdd 0
6 12599840 0xc04220 0
1 12646230 0xc0f756 0
2 12574590 0xbfdf7e 0
3 12494532 0xbea6c4 0
5 12386599 0xbd0127 0
7 12464947 0xbe3333 0
/esfs/jtacquaviva/ioperf/file_write.00000000
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428244 0xbda3d4 0
3 12494533 0xbea6c5 0
5 12386600 0xbd0128 0
4 12516318 0xbefbde 0
7 12464948 0xbe3334 0
6 12599843 0xc04223 0
1 12646235 0xc0f75b 0
2 12574595 0xbfdf83 0
/esfs/jtacquaviva/ioperf/file_write.00000004
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12386602 0xbd012a 0
2 12574592 0xbfdf80 0
0 12428246 0xbda3d6 0
4 12516323 0xbefbe3 0
3 12494538 0xbea6ca 0
7 12464953 0xbe3339 0
6 12599844 0xc04224 0
1 12646237 0xc0f75d 0
/esfs/jtacquaviva/ioperf/file_write.00000007
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12386611 0xbd0133 0
6 12599851 0xc0422b 0
0 12428256 0xbda3e0 0
2 12574604 0xbfdf8c 0
4 12516330 0xbefbea 0
7 12464960 0xbe3340 0
1 12646244 0xc0f764 0
3 12494547 0xbea6d3 0
/esfs/jtacquaviva/ioperf/file_write.00000012
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12599853 0xc0422d 0
4 12516331 0xbefbeb 0
7 12464961 0xbe3341 0
3 12494546 0xbea6d2 0
0 12428257 0xbda3e1 0
1 12646245 0xc0f765 0
5 12386614 0xbd0136 0
2 12574605 0xbfdf8d 0
/esfs/jtacquaviva/ioperf/file_write.00000013
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12646233 0xc0f759 0
2 12574593 0xbfdf81 0
4 12516322 0xbefbe2 0
0 12428249 0xbda3d9 0
5 12386606 0xbd012e 0
7 12464954 0xbe333a 0
6 12599846 0xc04226 0
3 12494539 0xbea6cb 0
/esfs/jtacquaviva/ioperf/file_write.00000003
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12599847 0xc04227 0
0 12428251 0xbda3db 0
2 12574598 0xbfdf86 0
4 12516325 0xbefbe5 0
7 12464955 0xbe333b 0
1 12646238 0xc0f75e 0
3 12494540 0xbea6cc 0
5 12386607 0xbd012f 0
/esfs/jtacquaviva/ioperf/file_write.00000008
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12516326 0xbefbe6 0
7 12464957 0xbe333d 0
1 12646240 0xc0f760 0
3 12494542 0xbea6ce 0
5 12386609 0xbd0131 0
6 12599849 0xc04229 0
0 12428253 0xbda3dd 0
2 12574600 0xbfdf88 0
/esfs/jtacquaviva/ioperf/file_write.00000010
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428252 0xbda3dc 0
7 12464956 0xbe333c 0
1 12646239 0xc0f75f 0
3 12494541 0xbea6cd 0
5 12386608 0xbd0130 0
6 12599848 0xc04228 0
2 12574599 0xbfdf87 0
4 12516327 0xbefbe7 0
/esfs/jtacquaviva/ioperf/file_write.00000005
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12574603 0xbfdf8b 0
1 12646243 0xc0f763 0
5 12386613 0xbd0135 0
4 12516332 0xbefbec 0
3 12494545 0xbea6d1 0
7 12464962 0xbe3342 0
6 12599854 0xc0422e 0
0 12428258 0xbda3e2 0
/esfs/jtacquaviva/ioperf/file_write.00000014
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494543 0xbea6cf 0
6 12599850 0xc0422a 0
2 12574601 0xbfdf89 0
7 12464959 0xbe333f 0
5 12386612 0xbd0134 0
0 12428255 0xbda3df 0
4 12516329 0xbefbe9 0
1 12646242 0xc0f762 0
/esfs/jtacquaviva/ioperf/file_write.00000009
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12386601 0xbd0129 0
2 12574591 0xbfdf7f 0
6 12599841 0xc04221 0
1 12646234 0xc0f75a 0
7 12464950 0xbe3336 0
4 12516321 0xbefbe1 0
0 12428247 0xbda3d7 0
3 12494537 0xbea6c9 0
/esfs/jtacquaviva/ioperf/file_write.00000015
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12516319 0xbefbdf 0
3 12494535 0xbea6c7 0
7 12464949 0xbe3335 0
1 12646236 0xc0f75c 0
5 12386605 0xbd012d 0
6 12599845 0xc04225 0
2 12574597 0xbfdf85 0
0 12428250 0xbda3da 0
/esfs/jtacquaviva/ioperf/file_write.00000001
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464958 0xbe333e 0
5 12386610 0xbd0132 0
0 12428254 0xbda3de 0
4 12516328 0xbefbe8 0
1 12646241 0xc0f761 0
6 12599852 0xc0422c 0
2 12574602 0xbfdf8a 0
3 12494544 0xbea6d0 0
/esfs/jtacquaviva/ioperf/file_write.00000011
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12646232 0xc0f758 0
4 12516320 0xbefbe0 0
7 12464951 0xbe3337 0
3 12494536 0xbea6c8 0
5 12386603 0xbd012b 0
0 12428248 0xbda3d8 0
2 12574596 0xbfdf84 0

347
output/COUNT:1#NN:8#PPN:2#API:POSIX#T:1048576.txt Normal file
View File

@ -0,0 +1,347 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 80530636800 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:2#API:POSIX#T:1048576.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 04:59:49 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 80530636800 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 04:59:49 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 16 (2 per node)
repetitions = 3
xfersize = 1 MiB
blocksize = 75 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288456634368.
WARNING: Using actual aggregate bytes moved = 357283397632.
write 4918 78643200 1024.00 0.000937 69.28 0.000542 69.28 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288490188800.
WARNING: Using actual aggregate bytes moved = 343360405504.
write 4761 78643200 1024.00 0.001388 68.77 0.001163 68.78 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288473411584.
WARNING: Using actual aggregate bytes moved = 351454363648.
write 4870 78643200 1024.00 0.001704 68.82 0.000890 68.82 2
Max Write: 4917.97 MiB/sec (5156.86 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 4917.97 4761.16 4849.85 65.65 68.95912 0 16 2 3 1 0 1 0 0 1 80530636800 1048576 357283397632 POSIX 0
Finished: Wed Oct 24 05:03:25 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 80530636800 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
+ tee -a ./output/COUNT:1#NN:8#PPN:2#API:POSIX#T:1048576.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 05:03:44 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 80530636800 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 05:03:44 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/indread8/file
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 16 (2 per node)
repetitions = 3
xfersize = 1 MiB
blocksize = 75 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 2576980377600.
WARNING: Using actual aggregate bytes moved = 78327578624.
read 1244.50 78643200 1024.00 0.000621 60.02 0.000589 60.02 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 2576980377600.
WARNING: Using actual aggregate bytes moved = 83850428416.
read 1332.06 78643200 1024.00 0.000527 60.03 0.000671 60.03 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 2576980377600.
WARNING: Using actual aggregate bytes moved = 90405076992.
read 1436.48 78643200 1024.00 0.000686 60.02 0.000852 60.02 2
Max Read: 1436.48 MiB/sec (1506.26 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 1436.48 1244.50 1337.68 78.48 60.02489 0 16 2 3 1 0 1 0 0 1 80530636800 1048576 78327578624 POSIX 0
Finished: Wed Oct 24 05:06:44 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write.00000006
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12386085 0xbcff25 0
0 12427732 0xbda1d4 0
2 12574079 0xbfdd7f 0
3 12494019 0xbea4c3 0
6 12599330 0xc04022 0
4 12515808 0xbef9e0 0
7 12464434 0xbe3132 0
1 12645721 0xc0f559 0
/esfs/jtacquaviva/ioperf/file_write.00000002
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464433 0xbe3131 0
0 12427734 0xbda1d6 0
4 12515807 0xbef9df 0
2 12574080 0xbfdd80 0
3 12494021 0xbea4c5 0
5 12386087 0xbcff27 0
1 12645722 0xc0f55a 0
/esfs/jtacquaviva/ioperf/file_write.00000000
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12515821 0xbef9ed 0
7 12464447 0xbe313f 0
1 12645735 0xc0f567 0
3 12494033 0xbea4d1 0
5 12386100 0xbcff34 0
6 12599343 0xc0402f 0
0 12427747 0xbda1e3 0
2 12574093 0xbfdd8d 0
/esfs/jtacquaviva/ioperf/file_write.00000004
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12645729 0xc0f561 0
2 12574090 0xbfdd8a 0
3 12494028 0xbea4cc 0
5 12386094 0xbcff2e 0
4 12515819 0xbef9eb 0
6 12599339 0xc0402b 0
7 12464446 0xbe313e 0
0 12427745 0xbda1e1 0
/esfs/jtacquaviva/ioperf/file_write.00000007
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12427739 0xbda1db 0
1 12645727 0xc0f55f 0
6 12599335 0xc04027 0
7 12464440 0xbe3138 0
4 12515815 0xbef9e7 0
2 12574087 0xbfdd87 0
3 12494027 0xbea4cb 0
5 12386095 0xbcff2f 0
/esfs/jtacquaviva/ioperf/file_write.00000012
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12574091 0xbfdd8b 0
4 12515816 0xbef9e8 0
7 12464443 0xbe313b 0
1 12645733 0xc0f565 0
3 12494031 0xbea4cf 0
5 12386097 0xbcff31 0
6 12599341 0xc0402d 0
0 12427746 0xbda1e2 0
/esfs/jtacquaviva/ioperf/file_write.00000013
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12645720 0xc0f558 0
2 12574078 0xbfdd7e 0
3 12494018 0xbea4c2 0
4 12515806 0xbef9de 0
6 12599329 0xc04021 0
7 12464432 0xbe3130 0
0 12427733 0xbda1d5 0
5 12386086 0xbcff26 0
/esfs/jtacquaviva/ioperf/file_write.00000003
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12427741 0xbda1dd 0
2 12574088 0xbfdd88 0
1 12645731 0xc0f563 0
3 12494029 0xbea4cd 0
6 12599338 0xc0402a 0
7 12464445 0xbe313d 0
4 12515820 0xbef9ec 0
5 12386098 0xbcff32 0
/esfs/jtacquaviva/ioperf/file_write.00000008
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12427742 0xbda1de 0
2 12574092 0xbfdd8c 0
4 12515817 0xbef9e9 0
7 12464444 0xbe313c 0
1 12645734 0xc0f566 0
3 12494032 0xbea4d0 0
5 12386099 0xbcff33 0
6 12599342 0xc0402e 0
/esfs/jtacquaviva/ioperf/file_write.00000010
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12386092 0xbcff2c 0
7 12464439 0xbe3137 0
3 12494025 0xbea4c9 0
0 12427740 0xbda1dc 0
4 12515814 0xbef9e6 0
2 12574086 0xbfdd86 0
1 12645730 0xc0f562 0
6 12599337 0xc04029 0
/esfs/jtacquaviva/ioperf/file_write.00000005
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494020 0xbea4c4 0
6 12599332 0xc04024 0
4 12515810 0xbef9e2 0
5 12386089 0xbcff29 0
0 12427736 0xbda1d8 0
2 12574081 0xbfdd81 0
1 12645725 0xc0f55d 0
7 12464437 0xbe3135 0
/esfs/jtacquaviva/ioperf/file_write.00000014
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12515813 0xbef9e5 0
5 12386093 0xbcff2d 0
2 12574085 0xbfdd85 0
1 12645728 0xc0f560 0
7 12464441 0xbe3139 0
3 12494026 0xbea4ca 0
6 12599336 0xc04028 0
0 12427743 0xbda1df 0
/esfs/jtacquaviva/ioperf/file_write.00000009
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494022 0xbea4c6 0
5 12386090 0xbcff2a 0
6 12599334 0xc04026 0
0 12427738 0xbda1da 0
2 12574084 0xbfdd84 0
4 12515812 0xbef9e4 0
7 12464438 0xbe3136 0
1 12645726 0xc0f55e 0
/esfs/jtacquaviva/ioperf/file_write.00000015
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12427735 0xbda1d7 0
7 12464435 0xbe3133 0
4 12515809 0xbef9e1 0
5 12386088 0xbcff28 0
1 12645724 0xc0f55c 0
3 12494024 0xbea4c8 0
2 12574082 0xbfdd82 0
6 12599333 0xc04025 0
/esfs/jtacquaviva/ioperf/file_write.00000001
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464442 0xbe313a 0
2 12574089 0xbfdd89 0
1 12645732 0xc0f564 0
4 12515818 0xbef9ea 0
3 12494030 0xbea4ce 0
5 12386096 0xbcff30 0
6 12599340 0xc0402c 0
0 12427744 0xbda1e0 0
/esfs/jtacquaviva/ioperf/file_write.00000011
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12599331 0xc04023 0
7 12464436 0xbe3134 0
1 12645723 0xc0f55b 0
3 12494023 0xbea4c7 0
2 12574083 0xbfdd83 0
0 12427737 0xbda1d9 0
5 12386091 0xbcff2b 0
4 12515811 0xbef9e3 0

343
output/COUNT:1#NN:8#PPN:2#API:POSIX#T:10485760.txt Normal file
View File

@ -0,0 +1,343 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 80530636800 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:2#API:POSIX#T:10485760.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 03:52:43 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 80530636800 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 03:52:43 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 16 (2 per node)
repetitions = 3
xfersize = 10 MiB
blocksize = 75 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1287546470400.
WARNING: Using actual aggregate bytes moved = 390038814720.
write 5481 78643200 10240 0.000734 67.87 0.000542 67.87 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288490188800.
WARNING: Using actual aggregate bytes moved = 396298813440.
write 5558 78643200 10240 0.001981 68.00 0.000589 68.00 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288322416640.
WARNING: Using actual aggregate bytes moved = 391653621760.
write 5490 78643200 10240 0.001679 68.04 0.000678 68.04 2
Max Write: 5557.87 MiB/sec (5827.85 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 5557.87 5480.65 5509.41 34.47 67.96959 0 16 2 3 1 0 1 0 0 1 80530636800 10485760 390038814720 POSIX 0
Finished: Wed Oct 24 03:56:17 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ tee -a ./output/COUNT:1#NN:8#PPN:2#API:POSIX#T:10485760.txt
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 80530636800 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 03:56:42 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 80530636800 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 03:56:42 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/indread8/file
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 16 (2 per node)
repetitions = 3
xfersize = 10 MiB
blocksize = 75 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 2576980377600.
WARNING: Using actual aggregate bytes moved = 90177536000.
read 1430.03 78643200 10240 0.002533 60.14 0.002128 60.14 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 2576980377600.
WARNING: Using actual aggregate bytes moved = 97989427200.
read 1544.39 78643200 10240 0.002137 60.51 0.002124 60.51 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 2576980377600.
WARNING: Using actual aggregate bytes moved = 106472407040.
read 1688.81 78643200 10240 0.002131 60.12 0.002109 60.13 2
Max Read: 1688.81 MiB/sec (1770.85 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 1688.81 1430.03 1554.41 105.88 60.25765 0 16 2 3 1 0 1 0 0 1 80530636800 10485760 90177536000 POSIX 0
Finished: Wed Oct 24 03:59:43 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write.00000006
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12427227 0xbd9fdb 0
1 12645213 0xc0f35d 0
2 12573566 0xbfdb7e 0
4 12515301 0xbef7e5 0
6 12598822 0xc03e26 0
5 12385577 0xbcfd29 0
7 12463921 0xbe2f31 0
3 12493512 0xbea2c8 0
/esfs/jtacquaviva/ioperf/file_write.00000002
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12463928 0xbe2f38 0
1 12645218 0xc0f362 0
5 12385582 0xbcfd2e 0
0 12427235 0xbd9fe3 0
4 12515308 0xbef7ec 0
3 12493519 0xbea2cf 0
6 12598830 0xc03e2e 0
2 12573575 0xbfdb87 0
/esfs/jtacquaviva/ioperf/file_write.00000000
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12493513 0xbea2c9 0
5 12385579 0xbcfd2b 0
6 12598824 0xc03e28 0
0 12427232 0xbd9fe0 0
2 12573571 0xbfdb83 0
4 12515304 0xbef7e8 0
7 12463925 0xbe2f35 0
1 12645215 0xc0f35f 0
/esfs/jtacquaviva/ioperf/file_write.00000004
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573574 0xbfdb86 0
3 12493517 0xbea2cd 0
6 12598828 0xc03e2c 0
7 12463931 0xbe2f3b 0
5 12385584 0xbcfd30 0
0 12427236 0xbd9fe4 0
1 12645221 0xc0f365 0
4 12515309 0xbef7ed 0
/esfs/jtacquaviva/ioperf/file_write.00000007
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12463930 0xbe2f3a 0
1 12645220 0xc0f364 0
3 12493518 0xbea2ce 0
5 12385585 0xbcfd31 0
6 12598829 0xc03e2d 0
0 12427237 0xbd9fe5 0
2 12573576 0xbfdb88 0
4 12515310 0xbef7ee 0
/esfs/jtacquaviva/ioperf/file_write.00000012
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573573 0xbfdb85 0
7 12463927 0xbe2f37 0
3 12493515 0xbea2cb 0
6 12598827 0xc03e2b 0
4 12515307 0xbef7eb 0
1 12645219 0xc0f363 0
5 12385583 0xbcfd2f 0
/esfs/jtacquaviva/ioperf/file_write.00000013
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12645210 0xc0f35a 0
6 12598820 0xc03e24 0
2 12573568 0xbfdb80 0
5 12385575 0xbcfd27 0
3 12493510 0xbea2c6 0
0 12427231 0xbd9fdf 0
7 12463923 0xbe2f33 0
/esfs/jtacquaviva/ioperf/file_write.00000003
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12515300 0xbef7e4 0
6 12598821 0xc03e25 0
5 12385576 0xbcfd28 0
0 12427230 0xbd9fde 0
1 12645214 0xc0f35e 0
2 12573570 0xbfdb82 0
7 12463924 0xbe2f34 0
/esfs/jtacquaviva/ioperf/file_write.00000008
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12598826 0xc03e2a 0
4 12515306 0xbef7ea 0
1 12645217 0xc0f361 0
5 12385581 0xbcfd2d 0
0 12427234 0xbd9fe2 0
7 12463929 0xbe2f39 0
3 12493516 0xbea2cc 0
/esfs/jtacquaviva/ioperf/file_write.00000010
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12385580 0xbcfd2c 0
6 12598825 0xc03e29 0
0 12427233 0xbd9fe1 0
2 12573572 0xbfdb84 0
4 12515305 0xbef7e9 0
7 12463926 0xbe2f36 0
1 12645216 0xc0f360 0
3 12493514 0xbea2ca 0
/esfs/jtacquaviva/ioperf/file_write.00000005
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12645211 0xc0f35b 0
0 12427228 0xbd9fdc 0
5 12385574 0xbcfd26 0
4 12515303 0xbef7e7 0
2 12573569 0xbfdb81 0
3 12493511 0xbea2c7 0
7 12463922 0xbe2f32 0
6 12598823 0xc03e27 0
/esfs/jtacquaviva/ioperf/file_write.00000014
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12493508 0xbea2c4 0
2 12573564 0xbfdb7c 0
0 12427226 0xbd9fda 0
4 12515299 0xbef7e3 0
7 12463919 0xbe2f2f 0
5 12385573 0xbcfd25 0
6 12598819 0xc03e23 0
/esfs/jtacquaviva/ioperf/file_write.00000009
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12598818 0xc03e22 0
3 12493509 0xbea2c5 0
1 12645212 0xc0f35c 0
7 12463920 0xbe2f30 0
2 12573567 0xbfdb7f 0
4 12515302 0xbef7e6 0
0 12427229 0xbd9fdd 0
5 12385578 0xbcfd2a 0
/esfs/jtacquaviva/ioperf/file_write.00000015
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573562 0xbfdb7a 0
4 12515296 0xbef7e0 0
7 12463916 0xbe2f2c 0
1 12645207 0xc0f357 0
3 12493505 0xbea2c1 0
5 12385570 0xbcfd22 0
6 12598815 0xc03e1f 0
0 12427223 0xbd9fd7 0
/esfs/jtacquaviva/ioperf/file_write.00000001
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12515297 0xbef7e1 0
7 12463917 0xbe2f2d 0
5 12385571 0xbcfd23 0
6 12598816 0xc03e20 0
0 12427224 0xbd9fd8 0
1 12645208 0xc0f358 0
3 12493506 0xbea2c2 0
2 12573563 0xbfdb7b 0
/esfs/jtacquaviva/ioperf/file_write.00000011
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12598817 0xc03e21 0
3 12493507 0xbea2c3 0
0 12427225 0xbd9fd9 0
5 12385572 0xbcfd24 0
7 12463918 0xbe2f2e 0
4 12515298 0xbef7e2 0
1 12645209 0xc0f359 0
2 12573565 0xbfdb7d 0

343
output/COUNT:1#NN:8#PPN:2#API:POSIX#T:16384.txt Normal file
View File

@ -0,0 +1,343 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 80530636800 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:2#API:POSIX#T:16384.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 07:25:56 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 80530636800 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 07:25:56 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 16 (2 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 75 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288470528000.
WARNING: Using actual aggregate bytes moved = 32117735424.
write 253.16 78643200 16.00 0.000766 120.99 0.000383 120.99 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288479817728.
WARNING: Using actual aggregate bytes moved = 26547306496.
write 211.75 78643200 16.00 0.001222 119.56 0.000635 119.57 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288482062336.
WARNING: Using actual aggregate bytes moved = 28408348672.
write 231.65 78643200 16.00 0.001139 116.95 0.000387 116.95 2
Max Write: 253.16 MiB/sec (265.45 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 253.16 211.75 232.19 16.91 119.16970 0 16 2 3 1 0 1 0 0 1 80530636800 16384 32117735424 POSIX 0
Finished: Wed Oct 24 07:31:55 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ tee -a ./output/COUNT:1#NN:8#PPN:2#API:POSIX#T:16384.txt
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 80530636800 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 07:32:00 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 80530636800 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 07:32:00 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/indread8/file
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 16 (2 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 75 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 2576980377600.
WARNING: Using actual aggregate bytes moved = 1782382592.
read 28.19 78643200 16.00 0.003095 60.29 0.002781 60.29 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 2576980377600.
WARNING: Using actual aggregate bytes moved = 2341502976.
read 37.04 78643200 16.00 0.002794 60.29 0.002829 60.29 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 2576980377600.
WARNING: Using actual aggregate bytes moved = 2328739840.
read 36.84 78643200 16.00 0.002892 60.28 0.002882 60.28 2
Max Read: 37.04 MiB/sec (38.84 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 37.04 28.19 34.03 4.12 60.28596 0 16 2 3 1 0 1 0 0 1 80530636800 16384 1782382592 POSIX 0
Finished: Wed Oct 24 07:35:01 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write.00000006
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12646748 0xc0f95c 0
4 12516833 0xbefde1 0
3 12495053 0xbea8cd 0
6 12600358 0xc04426 0
0 12428764 0xbda5dc 0
2 12575110 0xbfe186 0
7 12465472 0xbe3540 0
5 12387116 0xbd032c 0
/esfs/jtacquaviva/ioperf/file_write.00000002
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12387119 0xbd032f 0
6 12600364 0xc0442c 0
4 12516841 0xbefde9 0
7 12465476 0xbe3544 0
1 12646756 0xc0f964 0
0 12428768 0xbda5e0 0
2 12575115 0xbfe18b 0
3 12495062 0xbea8d6 0
/esfs/jtacquaviva/ioperf/file_write.00000000
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12646749 0xc0f95d 0
0 12428762 0xbda5da 0
4 12516836 0xbefde4 0
7 12465471 0xbe353f 0
5 12387115 0xbd032b 0
3 12495057 0xbea8d1 0
6 12600361 0xc04429 0
/esfs/jtacquaviva/ioperf/file_write.00000004
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428767 0xbda5df 0
1 12646757 0xc0f965 0
3 12495061 0xbea8d5 0
4 12516842 0xbefdea 0
5 12387121 0xbd0331 0
6 12600366 0xc0442e 0
2 12575116 0xbfe18c 0
7 12465478 0xbe3546 0
/esfs/jtacquaviva/ioperf/file_write.00000007
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12600369 0xc04431 0
0 12428772 0xbda5e4 0
2 12575119 0xbfe18f 0
4 12516846 0xbefdee 0
7 12465481 0xbe3549 0
1 12646761 0xc0f969 0
3 12495065 0xbea8d9 0
5 12387124 0xbd0334 0
/esfs/jtacquaviva/ioperf/file_write.00000012
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12575105 0xbfe181 0
1 12646747 0xc0f95b 0
5 12387110 0xbd0326 0
3 12495051 0xbea8cb 0
4 12516831 0xbefddf 0
6 12600354 0xc04422 0
7 12465466 0xbe353a 0
0 12428758 0xbda5d6 0
/esfs/jtacquaviva/ioperf/file_write.00000013
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12495058 0xbea8d2 0
0 12428766 0xbda5de 0
7 12465475 0xbe3543 0
2 12575113 0xbfe189 0
5 12387118 0xbd032e 0
6 12600363 0xc0442b 0
4 12516840 0xbefde8 0
1 12646755 0xc0f963 0
/esfs/jtacquaviva/ioperf/file_write.00000003
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12600357 0xc04425 0
7 12465469 0xbe353d 0
1 12646751 0xc0f95f 0
5 12387114 0xbd032a 0
2 12575109 0xbfe185 0
4 12516837 0xbefde5 0
0 12428765 0xbda5dd 0
/esfs/jtacquaviva/ioperf/file_write.00000008
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12575112 0xbfe188 0
4 12516839 0xbefde7 0
7 12465474 0xbe3542 0
5 12387117 0xbd032d 0
6 12600362 0xc0442a 0
1 12646754 0xc0f962 0
3 12495059 0xbea8d3 0
/esfs/jtacquaviva/ioperf/file_write.00000010
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12575114 0xbfe18a 0
3 12495060 0xbea8d4 0
0 12428769 0xbda5e1 0
6 12600365 0xc0442d 0
5 12387120 0xbd0330 0
4 12516843 0xbefdeb 0
7 12465477 0xbe3545 0
1 12646758 0xc0f966 0
/esfs/jtacquaviva/ioperf/file_write.00000005
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12516834 0xbefde2 0
5 12387113 0xbd0329 0
6 12600359 0xc04427 0
0 12428763 0xbda5db 0
2 12575108 0xbfe184 0
3 12495056 0xbea8d0 0
1 12646753 0xc0f961 0
7 12465473 0xbe3541 0
/esfs/jtacquaviva/ioperf/file_write.00000014
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428760 0xbda5d8 0
5 12387112 0xbd0328 0
6 12600356 0xc04424 0
1 12646750 0xc0f95e 0
3 12495054 0xbea8ce 0
7 12465470 0xbe353e 0
2 12575111 0xbfe187 0
4 12516838 0xbefde6 0
/esfs/jtacquaviva/ioperf/file_write.00000009
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12465467 0xbe353b 0
3 12495052 0xbea8cc 0
4 12516832 0xbefde0 0
0 12428759 0xbda5d7 0
5 12387111 0xbd0327 0
2 12575106 0xbfe182 0
6 12600355 0xc04423 0
/esfs/jtacquaviva/ioperf/file_write.00000015
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12495063 0xbea8d7 0
5 12387122 0xbd0332 0
6 12600367 0xc0442f 0
0 12428770 0xbda5e2 0
2 12575117 0xbfe18d 0
4 12516844 0xbefdec 0
7 12465479 0xbe3547 0
1 12646759 0xc0f967 0
/esfs/jtacquaviva/ioperf/file_write.00000001
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12387123 0xbd0333 0
6 12600368 0xc04430 0
0 12428771 0xbda5e3 0
2 12575118 0xbfe18e 0
4 12516845 0xbefded 0
7 12465480 0xbe3548 0
1 12646760 0xc0f968 0
3 12495064 0xbea8d8 0
/esfs/jtacquaviva/ioperf/file_write.00000011
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428761 0xbda5d9 0
2 12575107 0xbfe183 0
7 12465468 0xbe353c 0
4 12516835 0xbefde3 0
6 12600360 0xc04428 0
3 12495055 0xbea8cf 0
1 12646752 0xc0f960 0

110
output/COUNT:1#NN:8#PPN:4#API:MPIIO#T:102400.txt Normal file
View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 40265318400 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:4#API:MPIIO#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 05:35:28 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 40265318400 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 05:35:28 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 32 (4 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 37.50 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288488243200.
WARNING: Using actual aggregate bytes moved = 22185164800.
write 348.41 39321600 100.00 0.019215 60.71 0.000534 60.73 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288483942400.
WARNING: Using actual aggregate bytes moved = 22497484800.
write 353.21 39321600 100.00 0.001917 60.74 0.000539 60.74 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288487424000.
WARNING: Using actual aggregate bytes moved = 22288896000.
write 349.81 39321600 100.00 0.002036 60.76 0.000559 60.76 2
Max Write: 353.21 MiB/sec (370.37 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 353.21 348.41 350.48 2.01 60.74474 0 32 4 3 0 0 1 0 0 1 40265318400 102400 22185164800 MPIIO 0
Finished: Wed Oct 24 05:38:31 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 40265318400 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
+ tee -a ./output/COUNT:1#NN:8#PPN:4#API:MPIIO#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 05:38:40 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 40265318400 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 05:38:40 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 32 (4 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 37.50 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 16698368000.
read 262.32 39321600 100.00 0.014407 60.69 0.006799 60.71 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 16845721600.
read 264.70 39321600 100.00 0.005292 60.69 0.004282 60.69 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 16842854400.
read 264.62 39321600 100.00 0.005390 60.70 0.004318 60.70 2
Max Read: 264.70 MiB/sec (277.56 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 264.70 262.32 263.88 1.10 60.70035 0 32 4 3 0 0 1 0 0 1 40265318400 102400 16698368000 MPIIO 0
Finished: Wed Oct 24 05:41:43 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464743 0xbe3267 0
1 12646026 0xc0f68a 0
3 12494324 0xbea5f4 0
5 12386395 0xbd005b 0
6 12599639 0xc04157 0
0 12428038 0xbda306 0
2 12574385 0xbfdeb1 0
4 12516115 0xbefb13 0

110
output/COUNT:1#NN:8#PPN:4#API:MPIIO#T:1048576.txt Normal file
View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 40265318400 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:4#API:MPIIO#T:1048576.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 04:26:49 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 40265318400 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 04:26:49 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 32 (4 per node)
repetitions = 3
xfersize = 1 MiB
blocksize = 37.50 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288488091648.
WARNING: Using actual aggregate bytes moved = 261622857728.
write 4151 39321600 1024.00 0.009951 60.09 0.000559 60.10 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288488091648.
WARNING: Using actual aggregate bytes moved = 258843082752.
write 4096 39321600 1024.00 0.180245 60.09 0.000508 60.27 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288484945920.
WARNING: Using actual aggregate bytes moved = 262620053504.
write 4156 39321600 1024.00 0.143859 60.12 0.000498 60.27 2
Max Write: 4155.83 MiB/sec (4357.70 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 4155.83 4095.85 4134.27 27.23 60.21318 0 32 4 3 0 0 1 0 0 1 40265318400 1048576 261622857728 MPIIO 0
Finished: Wed Oct 24 04:29:56 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 40265318400 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
+ tee -a ./output/COUNT:1#NN:8#PPN:4#API:MPIIO#T:1048576.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 04:30:13 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 40265318400 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 04:30:13 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 32 (4 per node)
repetitions = 3
xfersize = 1 MiB
blocksize = 37.50 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 73459040256.
read 1161.48 39321600 1024.00 0.183081 60.14 0.008206 60.32 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 75210162176.
read 1193.01 39321600 1024.00 0.009586 60.12 0.008296 60.12 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 76188483584.
read 1208.65 39321600 1024.00 0.009621 60.11 0.008259 60.12 2
Max Read: 1208.65 MiB/sec (1267.36 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 1208.65 1161.48 1187.71 19.62 60.18458 0 32 4 3 0 0 1 0 0 1 40265318400 1048576 73459040256 MPIIO 0
Finished: Wed Oct 24 04:33:13 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12515603 0xbef913 0
7 12464228 0xbe3064 0
1 12645515 0xc0f48b 0
3 12493813 0xbea3f5 0
5 12385880 0xbcfe58 0
6 12599126 0xc03f56 0
0 12427531 0xbda10b 0
2 12573871 0xbfdcaf 0

110
output/COUNT:1#NN:8#PPN:4#API:MPIIO#T:10485760.txt Normal file
View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 40265318400 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:4#API:MPIIO#T:10485760.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 03:19:42 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 40265318400 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 03:19:42 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 32 (4 per node)
repetitions = 3
xfersize = 10 MiB
blocksize = 37.50 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288458731520.
WARNING: Using actual aggregate bytes moved = 174157987840.
write 2753.77 39321600 10240 0.010886 60.30 0.000964 60.31 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288437760000.
WARNING: Using actual aggregate bytes moved = 176737484800.
write 2792.94 39321600 10240 0.081348 60.27 0.000673 60.35 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288427274240.
WARNING: Using actual aggregate bytes moved = 171987435520.
write 2726.40 39321600 10240 0.026415 60.13 0.000841 60.16 2
Max Write: 2792.94 MiB/sec (2928.61 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 2792.94 2726.40 2757.70 27.31 60.27411 0 32 4 3 0 0 1 0 0 1 40265318400 10485760 174157987840 MPIIO 0
Finished: Wed Oct 24 03:22:49 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ tee -a ./output/COUNT:1#NN:8#PPN:4#API:MPIIO#T:10485760.txt
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 40265318400 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 03:23:06 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 40265318400 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 03:23:06 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 32 (4 per node)
repetitions = 3
xfersize = 10 MiB
blocksize = 37.50 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 174000701440.
read 2754.27 39321600 10240 0.143731 60.11 0.004690 60.25 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 177880432640.
read 2821.21 39321600 10240 0.005724 60.13 0.004749 60.13 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 180124385280.
read 2858.10 39321600 10240 0.006035 60.10 0.004731 60.10 2
Max Read: 2858.10 MiB/sec (2996.94 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 2858.10 2754.27 2811.19 42.98 60.16048 0 32 4 3 0 0 1 0 0 1 40265318400 10485760 174000701440 MPIIO 0
Finished: Wed Oct 24 03:26:07 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12515092 0xbef714 0
7 12463713 0xbe2e61 0
1 12645000 0xc0f288 0
3 12493301 0xbea1f5 0
5 12385364 0xbcfc54 0
6 12598615 0xc03d57 0
0 12427019 0xbd9f0b 0
2 12573356 0xbfdaac 0

110
output/COUNT:1#NN:8#PPN:4#API:MPIIO#T:16384.txt Normal file
View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 40265318400 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:4#API:MPIIO#T:16384.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 06:48:20 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 40265318400 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 06:48:20 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 32 (4 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 37.50 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288482373632.
WARNING: Using actual aggregate bytes moved = 4082876416.
write 60.46 39321600 16.00 0.012821 64.39 0.000421 64.41 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288484339712.
WARNING: Using actual aggregate bytes moved = 4046667776.
write 60.03 39321600 16.00 0.001576 64.29 0.000718 64.29 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288487600128.
WARNING: Using actual aggregate bytes moved = 4043341824.
write 59.99 39321600 16.00 0.002535 64.27 0.001013 64.27 2
Max Write: 60.46 MiB/sec (63.39 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 60.46 59.99 60.16 0.21 64.32336 0 32 4 3 0 0 1 0 0 1 40265318400 16384 4082876416 MPIIO 0
Finished: Wed Oct 24 06:51:33 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ tee -a ./output/COUNT:1#NN:8#PPN:4#API:MPIIO#T:16384.txt
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 40265318400 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 06:51:37 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 40265318400 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 06:51:37 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 32 (4 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 37.50 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 3402989568.
read 50.52 39321600 16.00 0.007093 64.23 0.000988 64.24 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 3412885504.
read 50.68 39321600 16.00 0.001842 64.22 0.000982 64.22 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 3433807872.
read 50.99 39321600 16.00 0.002303 64.22 0.000941 64.22 2
Max Read: 50.99 MiB/sec (53.47 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 50.99 50.52 50.73 0.20 64.22841 0 32 4 3 0 0 1 0 0 1 40265318400 16384 3402989568 MPIIO 0
Finished: Wed Oct 24 06:54:50 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428552 0xbda508 0
2 12574900 0xbfe0b4 0
4 12516626 0xbefd12 0
7 12465256 0xbe3468 0
1 12646543 0xc0f88f 0
3 12494843 0xbea7fb 0
5 12386905 0xbd0259 0
6 12600146 0xc04352 0

600
output/COUNT:1#NN:8#PPN:4#API:POSIX#T:102400.txt Normal file
View File

@ -0,0 +1,600 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 40265318400 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:4#API:POSIX#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 05:27:34 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 40265318400 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 05:27:34 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 32 (4 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 37.50 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288460697600.
WARNING: Using actual aggregate bytes moved = 85053952000.
write 899.15 39321600 100.00 0.001971 90.21 0.000328 90.21 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288355840000.
WARNING: Using actual aggregate bytes moved = 75213209600.
write 759.79 39321600 100.00 0.002503 94.40 0.000411 94.41 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288453222400.
WARNING: Using actual aggregate bytes moved = 76511436800.
write 796.06 39321600 100.00 0.008759 91.65 0.000430 91.66 2
Max Write: 899.15 MiB/sec (942.83 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 899.15 759.79 818.33 59.04 92.09254 0 32 4 3 1 0 1 0 0 1 40265318400 102400 85053952000 POSIX 0
Finished: Wed Oct 24 05:32:12 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 40265318400 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
+ tee -a ./output/COUNT:1#NN:8#PPN:4#API:POSIX#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 05:32:26 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 40265318400 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 05:32:26 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/indread8/file
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 32 (4 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 37.50 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 5153960755200.
WARNING: Using actual aggregate bytes moved = 24710860800.
read 392.53 39321600 100.00 0.001527 60.03 0.000924 60.04 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 5153960755200.
WARNING: Using actual aggregate bytes moved = 25415168000.
read 403.74 39321600 100.00 0.001171 60.03 0.000952 60.03 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 5153960755200.
WARNING: Using actual aggregate bytes moved = 25638297600.
read 407.24 39321600 100.00 0.000996 60.04 0.000961 60.04 2
Max Read: 407.24 MiB/sec (427.02 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 407.24 392.53 401.17 6.27 60.03603 0 32 4 3 1 0 1 0 0 1 40265318400 102400 24710860800 POSIX 0
Finished: Wed Oct 24 05:35:26 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write.00000006
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428022 0xbda2f6 0
2 12574368 0xbfdea0 0
4 12516100 0xbefb04 0
5 12386380 0xbd004c 0
7 12464727 0xbe3257 0
1 12646011 0xc0f67b 0
3 12494309 0xbea5e5 0
6 12599623 0xc04147 0
/esfs/jtacquaviva/ioperf/file_write.00000023
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12574370 0xbfdea2 0
4 12516101 0xbefb05 0
7 12464728 0xbe3258 0
1 12646012 0xc0f67c 0
3 12494310 0xbea5e6 0
5 12386381 0xbd004d 0
6 12599624 0xc04148 0
0 12428025 0xbda2f9 0
/esfs/jtacquaviva/ioperf/file_write.00000002
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464714 0xbe324a 0
1 12645997 0xc0f66d 0
3 12494297 0xbea5d9 0
5 12386367 0xbd003f 0
2 12574358 0xbfde96 0
0 12428012 0xbda2ec 0
6 12599610 0xc0413a 0
4 12516089 0xbefaf9 0
/esfs/jtacquaviva/ioperf/file_write.00000026
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494317 0xbea5ed 0
5 12386390 0xbd0056 0
7 12464737 0xbe3261 0
6 12599633 0xc04151 0
0 12428034 0xbda302 0
2 12574380 0xbfdeac 0
4 12516111 0xbefb0f 0
1 12646020 0xc0f684 0
/esfs/jtacquaviva/ioperf/file_write.00000000
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12646000 0xc0f670 0
2 12574359 0xbfde97 0
3 12494299 0xbea5db 0
0 12428014 0xbda2ee 0
5 12386369 0xbd0041 0
7 12464717 0xbe324d 0
4 12516090 0xbefafa 0
6 12599612 0xc0413c 0
/esfs/jtacquaviva/ioperf/file_write.00000020
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12386388 0xbd0054 0
0 12428031 0xbda2ff 0
3 12494318 0xbea5ee 0
2 12574378 0xbfdeaa 0
7 12464738 0xbe3262 0
6 12599634 0xc04152 0
4 12516110 0xbefb0e 0
1 12646021 0xc0f685 0
/esfs/jtacquaviva/ioperf/file_write.00000030
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12574353 0xbfde91 0
1 12645994 0xc0f66a 0
5 12386363 0xbd003b 0
0 12428008 0xbda2e8 0
4 12516084 0xbefaf4 0
3 12494294 0xbea5d6 0
6 12599607 0xc04137 0
7 12464712 0xbe3248 0
/esfs/jtacquaviva/ioperf/file_write.00000004
lmm_stripe_count: 6
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464719 0xbe324f 0
6 12599614 0xc0413e 0
5 12386372 0xbd0044 0
0 12428017 0xbda2f1 0
1 12646003 0xc0f673 0
2 12574362 0xbfde9a 0
/esfs/jtacquaviva/ioperf/file_write.00000007
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464711 0xbe3247 0
3 12494293 0xbea5d5 0
6 12599606 0xc04136 0
2 12574354 0xbfde92 0
1 12645995 0xc0f66b 0
5 12386364 0xbd003c 0
0 12428009 0xbda2e9 0
4 12516085 0xbefaf5 0
/esfs/jtacquaviva/ioperf/file_write.00000018
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12646017 0xc0f681 0
3 12494315 0xbea5eb 0
5 12386386 0xbd0052 0
6 12599629 0xc0414d 0
0 12428029 0xbda2fd 0
2 12574376 0xbfdea8 0
4 12516106 0xbefb0a 0
7 12464735 0xbe325f 0
/esfs/jtacquaviva/ioperf/file_write.00000031
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428024 0xbda2f8 0
2 12574372 0xbfdea4 0
4 12516102 0xbefb06 0
7 12464730 0xbe325a 0
1 12646013 0xc0f67d 0
3 12494311 0xbea5e7 0
5 12386382 0xbd004e 0
6 12599625 0xc04149 0
/esfs/jtacquaviva/ioperf/file_write.00000012
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12574356 0xbfde94 0
4 12516087 0xbefaf7 0
5 12386368 0xbd0040 0
1 12645999 0xc0f66f 0
3 12494298 0xbea5da 0
0 12428013 0xbda2ed 0
7 12464716 0xbe324c 0
6 12599611 0xc0413b 0
/esfs/jtacquaviva/ioperf/file_write.00000013
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12599635 0xc04153 0
0 12428035 0xbda303 0
2 12574382 0xbfdeae 0
4 12516112 0xbefb10 0
7 12464740 0xbe3264 0
1 12646023 0xc0f687 0
3 12494321 0xbea5f1 0
5 12386392 0xbd0058 0
/esfs/jtacquaviva/ioperf/file_write.00000025
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12645996 0xc0f66c 0
3 12494295 0xbea5d7 0
5 12386365 0xbd003d 0
6 12599609 0xc04139 0
0 12428010 0xbda2ea 0
7 12464713 0xbe3249 0
4 12516086 0xbefaf6 0
2 12574357 0xbfde95 0
/esfs/jtacquaviva/ioperf/file_write.00000022
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12516093 0xbefafd 0
3 12494302 0xbea5de 0
7 12464722 0xbe3252 0
6 12599616 0xc04140 0
2 12574363 0xbfde9b 0
5 12386375 0xbd0047 0
1 12646006 0xc0f676 0
/esfs/jtacquaviva/ioperf/file_write.00000003
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464733 0xbe325d 0
1 12646016 0xc0f680 0
3 12494314 0xbea5ea 0
5 12386385 0xbd0051 0
6 12599628 0xc0414c 0
0 12428028 0xbda2fc 0
2 12574375 0xbfdea7 0
4 12516105 0xbefb09 0
/esfs/jtacquaviva/ioperf/file_write.00000008
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12574373 0xbfdea5 0
4 12516103 0xbefb07 0
7 12464731 0xbe325b 0
1 12646014 0xc0f67e 0
3 12494312 0xbea5e8 0
5 12386383 0xbd004f 0
6 12599626 0xc0414a 0
0 12428026 0xbda2fa 0
/esfs/jtacquaviva/ioperf/file_write.00000028
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12516107 0xbefb0b 0
7 12464736 0xbe3260 0
6 12599631 0xc0414f 0
1 12646019 0xc0f683 0
5 12386391 0xbd0057 0
0 12428033 0xbda301 0
3 12494319 0xbea5ef 0
2 12574381 0xbfdead 0
/esfs/jtacquaviva/ioperf/file_write.00000010
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428006 0xbda2e6 0
4 12516082 0xbefaf2 0
3 12494291 0xbea5d3 0
2 12574352 0xbfde90 0
7 12464710 0xbe3246 0
1 12645993 0xc0f669 0
5 12386362 0xbd003a 0
6 12599605 0xc04135 0
/esfs/jtacquaviva/ioperf/file_write.00000017
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428015 0xbda2ef 0
5 12386371 0xbd0043 0
7 12464718 0xbe324e 0
1 12646002 0xc0f672 0
2 12574361 0xbfde99 0
4 12516091 0xbefafb 0
3 12494301 0xbea5dd 0
6 12599615 0xc0413f 0
/esfs/jtacquaviva/ioperf/file_write.00000029
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12516094 0xbefafe 0
5 12386373 0xbd0045 0
6 12599617 0xc04141 0
1 12646005 0xc0f675 0
2 12574364 0xbfde9c 0
0 12428018 0xbda2f2 0
3 12494304 0xbea5e0 0
7 12464723 0xbe3253 0
/esfs/jtacquaviva/ioperf/file_write.00000016
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12516098 0xbefb02 0
7 12464726 0xbe3256 0
5 12386378 0xbd004a 0
2 12574369 0xbfdea1 0
1 12646009 0xc0f679 0
6 12599621 0xc04145 0
3 12494308 0xbea5e4 0
/esfs/jtacquaviva/ioperf/file_write.00000005
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464734 0xbe325e 0
1 12646018 0xc0f682 0
3 12494316 0xbea5ec 0
5 12386387 0xbd0053 0
6 12599630 0xc0414e 0
0 12428030 0xbda2fe 0
2 12574377 0xbfdea9 0
4 12516108 0xbefb0c 0
/esfs/jtacquaviva/ioperf/file_write.00000014
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428020 0xbda2f4 0
2 12574366 0xbfde9e 0
4 12516096 0xbefb00 0
7 12464724 0xbe3254 0
1 12646007 0xc0f677 0
3 12494305 0xbea5e1 0
5 12386376 0xbd0048 0
6 12599619 0xc04143 0
/esfs/jtacquaviva/ioperf/file_write.00000021
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12386361 0xbd0039 0
2 12574351 0xbfde8f 0
7 12464709 0xbe3245 0
1 12645992 0xc0f668 0
6 12599604 0xc04134 0
0 12428007 0xbda2e7 0
4 12516083 0xbefaf3 0
3 12494292 0xbea5d4 0
/esfs/jtacquaviva/ioperf/file_write.00000019
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12599608 0xc04138 0
2 12574355 0xbfde93 0
3 12494296 0xbea5d8 0
7 12464715 0xbe324b 0
5 12386366 0xbd003e 0
1 12645998 0xc0f66e 0
4 12516088 0xbefaf8 0
0 12428011 0xbda2eb 0
/esfs/jtacquaviva/ioperf/file_write.00000009
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464721 0xbe3251 0
1 12646004 0xc0f674 0
5 12386374 0xbd0046 0
6 12599618 0xc04142 0
4 12516095 0xbefaff 0
3 12494303 0xbea5df 0
0 12428019 0xbda2f3 0
2 12574365 0xbfde9d 0
/esfs/jtacquaviva/ioperf/file_write.00000015
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12516104 0xbefb08 0
7 12464732 0xbe325c 0
1 12646015 0xc0f67f 0
3 12494313 0xbea5e9 0
5 12386384 0xbd0050 0
6 12599627 0xc0414b 0
0 12428027 0xbda2fb 0
2 12574374 0xbfdea6 0
/esfs/jtacquaviva/ioperf/file_write.00000027
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12386370 0xbd0042 0
1 12646001 0xc0f671 0
6 12599613 0xc0413d 0
3 12494300 0xbea5dc 0
2 12574360 0xbfde98 0
7 12464720 0xbe3250 0
0 12428016 0xbda2f0 0
4 12516092 0xbefafc 0
/esfs/jtacquaviva/ioperf/file_write.00000024
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494307 0xbea5e3 0
4 12516099 0xbefb03 0
5 12386379 0xbd004b 0
0 12428023 0xbda2f7 0
1 12646010 0xc0f67a 0
2 12574371 0xbfdea3 0
7 12464729 0xbe3259 0
6 12599622 0xc04146 0
/esfs/jtacquaviva/ioperf/file_write.00000001
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12574367 0xbfde9f 0
4 12516097 0xbefb01 0
7 12464725 0xbe3255 0
1 12646008 0xc0f678 0
3 12494306 0xbea5e2 0
5 12386377 0xbd0049 0
6 12599620 0xc04144 0
0 12428021 0xbda2f5 0
/esfs/jtacquaviva/ioperf/file_write.00000011
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12386389 0xbd0055 0
6 12599632 0xc04150 0
0 12428032 0xbda300 0
2 12574379 0xbfdeab 0
4 12516109 0xbefb0d 0
7 12464739 0xbe3263 0
1 12646022 0xc0f686 0
3 12494320 0xbea5f0 0

602
output/COUNT:1#NN:8#PPN:4#API:POSIX#T:1048576.txt Normal file
View File

@ -0,0 +1,602 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 40265318400 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:4#API:POSIX#T:1048576.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 04:19:56 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 40265318400 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 04:19:56 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 32 (4 per node)
repetitions = 3
xfersize = 1 MiB
blocksize = 37.50 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288410497024.
WARNING: Using actual aggregate bytes moved = 351248842752.
write 4842 39321600 1024.00 0.002099 69.18 0.000774 69.18 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288187150336.
WARNING: Using actual aggregate bytes moved = 352358236160.
write 4855 39321600 1024.00 0.008233 69.21 0.000361 69.22 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288462925824.
WARNING: Using actual aggregate bytes moved = 346617282560.
write 4787 39321600 1024.00 0.001639 69.06 0.000374 69.06 2
Max Write: 4854.87 MiB/sec (5090.70 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 4854.87 4786.51 4827.75 29.64 69.15340 0 32 4 3 1 0 1 0 0 1 40265318400 1048576 351248842752 POSIX 0
Finished: Wed Oct 24 04:23:30 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ tee -a ./output/COUNT:1#NN:8#PPN:4#API:POSIX#T:1048576.txt
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 40265318400 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 04:23:47 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 40265318400 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 04:23:47 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/indread8/file
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 32 (4 per node)
repetitions = 3
xfersize = 1 MiB
blocksize = 37.50 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 5153960755200.
WARNING: Using actual aggregate bytes moved = 151530766336.
read 2407.47 39321600 1024.00 0.001461 60.02 0.000884 60.03 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 5153960755200.
WARNING: Using actual aggregate bytes moved = 172089147392.
read 2734.60 39321600 1024.00 0.000905 60.01 0.001178 60.01 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 5153960755200.
WARNING: Using actual aggregate bytes moved = 189633921024.
read 3013.02 39321600 1024.00 0.000915 60.02 0.000907 60.02 2
Max Read: 3013.02 MiB/sec (3159.38 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 3013.02 2407.47 2718.37 247.48 60.02112 0 32 4 3 1 0 1 0 0 1 40265318400 1048576 151530766336 POSIX 0
Finished: Wed Oct 24 04:26:47 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write.00000006
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573838 0xbfdc8e 0
1 12645483 0xc0f46b 0
6 12599095 0xc03f37 0
5 12385849 0xbcfe39 0
4 12515573 0xbef8f5 0
3 12493784 0xbea3d8 0
7 12464200 0xbe3048 0
/esfs/jtacquaviva/ioperf/file_write.00000023
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12515582 0xbef8fe 0
1 12645495 0xc0f477 0
5 12385858 0xbcfe42 0
0 12427512 0xbda0f8 0
7 12464209 0xbe3051 0
3 12493794 0xbea3e2 0
6 12599107 0xc03f43 0
2 12573852 0xbfdc9c 0
/esfs/jtacquaviva/ioperf/file_write.00000002
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12645498 0xc0f47a 0
3 12493796 0xbea3e4 0
5 12385862 0xbcfe46 0
6 12599109 0xc03f45 0
0 12427514 0xbda0fa 0
2 12573854 0xbfdc9e 0
4 12515586 0xbef902 0
7 12464211 0xbe3053 0
/esfs/jtacquaviva/ioperf/file_write.00000026
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573845 0xbfdc95 0
4 12515577 0xbef8f9 0
7 12464202 0xbe304a 0
1 12645489 0xc0f471 0
3 12493787 0xbea3db 0
5 12385853 0xbcfe3d 0
6 12599100 0xc03f3c 0
0 12427505 0xbda0f1 0
/esfs/jtacquaviva/ioperf/file_write.00000000
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12427522 0xbda102 0
2 12573862 0xbfdca6 0
4 12515594 0xbef90a 0
7 12464219 0xbe305b 0
1 12645506 0xc0f482 0
3 12493804 0xbea3ec 0
5 12385870 0xbcfe4e 0
6 12599117 0xc03f4d 0
/esfs/jtacquaviva/ioperf/file_write.00000020
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464223 0xbe305f 0
1 12645510 0xc0f486 0
5 12385875 0xbcfe53 0
0 12427526 0xbda106 0
4 12515598 0xbef90e 0
3 12493809 0xbea3f1 0
6 12599122 0xc03f52 0
2 12573867 0xbfdcab 0
/esfs/jtacquaviva/ioperf/file_write.00000030
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464196 0xbe3044 0
5 12385847 0xbcfe37 0
3 12493781 0xbea3d5 0
2 12573843 0xbfdc93 0
1 12645485 0xc0f46d 0
6 12599097 0xc03f39 0
4 12515574 0xbef8f6 0
0 12427504 0xbda0f0 0
/esfs/jtacquaviva/ioperf/file_write.00000004
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464204 0xbe304c 0
1 12645491 0xc0f473 0
3 12493789 0xbea3dd 0
5 12385855 0xbcfe3f 0
6 12599102 0xc03f3e 0
0 12427507 0xbda0f3 0
2 12573847 0xbfdc97 0
4 12515579 0xbef8fb 0
/esfs/jtacquaviva/ioperf/file_write.00000007
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12493797 0xbea3e5 0
5 12385863 0xbcfe47 0
6 12599110 0xc03f46 0
0 12427515 0xbda0fb 0
2 12573855 0xbfdc9f 0
4 12515587 0xbef903 0
7 12464212 0xbe3054 0
1 12645499 0xc0f47b 0
/esfs/jtacquaviva/ioperf/file_write.00000018
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12599104 0xc03f40 0
4 12515580 0xbef8fc 0
3 12493792 0xbea3e0 0
0 12427511 0xbda0f7 0
1 12645494 0xc0f476 0
2 12573850 0xbfdc9a 0
7 12464208 0xbe3050 0
5 12385859 0xbcfe43 0
/esfs/jtacquaviva/ioperf/file_write.00000031
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12385873 0xbcfe51 0
3 12493808 0xbea3f0 0
6 12599121 0xc03f51 0
2 12573866 0xbfdcaa 0
7 12464224 0xbe3060 0
0 12427527 0xbda107 0
4 12515599 0xbef90f 0
1 12645511 0xc0f487 0
/esfs/jtacquaviva/ioperf/file_write.00000012
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12385864 0xbcfe48 0
6 12599111 0xc03f47 0
0 12427516 0xbda0fc 0
2 12573856 0xbfdca0 0
4 12515588 0xbef904 0
7 12464213 0xbe3055 0
1 12645500 0xc0f47c 0
3 12493798 0xbea3e6 0
/esfs/jtacquaviva/ioperf/file_write.00000013
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12385846 0xbcfe36 0
6 12599094 0xc03f36 0
4 12515571 0xbef8f3 0
2 12573840 0xbfdc90 0
3 12493782 0xbea3d6 0
1 12645484 0xc0f46c 0
0 12427502 0xbda0ee 0
7 12464199 0xbe3047 0
/esfs/jtacquaviva/ioperf/file_write.00000025
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12599116 0xc03f4c 0
0 12427521 0xbda101 0
2 12573861 0xbfdca5 0
4 12515593 0xbef909 0
7 12464218 0xbe305a 0
1 12645505 0xc0f481 0
3 12493803 0xbea3eb 0
5 12385869 0xbcfe4d 0
/esfs/jtacquaviva/ioperf/file_write.00000022
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12427518 0xbda0fe 0
2 12573858 0xbfdca2 0
4 12515590 0xbef906 0
7 12464215 0xbe3057 0
1 12645502 0xc0f47e 0
3 12493800 0xbea3e8 0
5 12385866 0xbcfe4a 0
6 12599113 0xc03f49 0
/esfs/jtacquaviva/ioperf/file_write.00000003
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464194 0xbe3042 0
6 12599093 0xc03f35 0
3 12493780 0xbea3d4 0
1 12645482 0xc0f46a 0
0 12427498 0xbda0ea 0
2 12573839 0xbfdc8f 0
5 12385848 0xbcfe38 0
4 12515572 0xbef8f4 0
/esfs/jtacquaviva/ioperf/file_write.00000008
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12493791 0xbea3df 0
0 12427509 0xbda0f5 0
2 12573849 0xbfdc99 0
7 12464206 0xbe304e 0
1 12645493 0xc0f475 0
5 12385857 0xbcfe41 0
6 12599105 0xc03f41 0
4 12515583 0xbef8ff 0
/esfs/jtacquaviva/ioperf/file_write.00000028
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573859 0xbfdca3 0
4 12515591 0xbef907 0
7 12464216 0xbe3058 0
1 12645503 0xc0f47f 0
6 12599114 0xc03f4a 0
0 12427519 0xbda0ff 0
3 12493801 0xbea3e9 0
5 12385868 0xbcfe4c 0
/esfs/jtacquaviva/ioperf/file_write.00000010
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573841 0xbfdc91 0
6 12599096 0xc03f38 0
0 12427501 0xbda0ed 0
7 12464198 0xbe3046 0
5 12385850 0xbcfe3a 0
1 12645488 0xc0f470 0
3 12493786 0xbea3da 0
/esfs/jtacquaviva/ioperf/file_write.00000017
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12599112 0xc03f48 0
0 12427517 0xbda0fd 0
2 12573857 0xbfdca1 0
4 12515589 0xbef905 0
7 12464214 0xbe3056 0
1 12645501 0xc0f47d 0
3 12493799 0xbea3e7 0
5 12385865 0xbcfe49 0
/esfs/jtacquaviva/ioperf/file_write.00000029
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12427510 0xbda0f6 0
7 12464207 0xbe304f 0
3 12493793 0xbea3e1 0
6 12599106 0xc03f42 0
2 12573851 0xbfdc9b 0
5 12385860 0xbcfe44 0
1 12645496 0xc0f478 0
4 12515584 0xbef900 0
/esfs/jtacquaviva/ioperf/file_write.00000016
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573842 0xbfdc92 0
7 12464197 0xbe3045 0
1 12645486 0xc0f46e 0
0 12427503 0xbda0ef 0
3 12493785 0xbea3d9 0
5 12385851 0xbcfe3b 0
6 12599098 0xc03f3a 0
4 12515575 0xbef8f7 0
/esfs/jtacquaviva/ioperf/file_write.00000005
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573863 0xbfdca7 0
4 12515595 0xbef90b 0
7 12464220 0xbe305c 0
1 12645507 0xc0f483 0
3 12493805 0xbea3ed 0
5 12385871 0xbcfe4f 0
6 12599118 0xc03f4e 0
0 12427523 0xbda103 0
/esfs/jtacquaviva/ioperf/file_write.00000014
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12515596 0xbef90c 0
7 12464221 0xbe305d 0
1 12645508 0xc0f484 0
3 12493806 0xbea3ee 0
5 12385872 0xbcfe50 0
6 12599119 0xc03f4f 0
0 12427524 0xbda104 0
2 12573864 0xbfdca8 0
/esfs/jtacquaviva/ioperf/file_write.00000021
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12493783 0xbea3d7 0
0 12427500 0xbda0ec 0
1 12645487 0xc0f46f 0
7 12464201 0xbe3049 0
2 12573844 0xbfdc94 0
5 12385852 0xbcfe3c 0
4 12515576 0xbef8f8 0
6 12599099 0xc03f3b 0
/esfs/jtacquaviva/ioperf/file_write.00000019
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464222 0xbe305e 0
1 12645509 0xc0f485 0
3 12493807 0xbea3ef 0
5 12385874 0xbcfe52 0
6 12599120 0xc03f50 0
0 12427525 0xbda105 0
2 12573865 0xbfdca9 0
4 12515597 0xbef90d 0
/esfs/jtacquaviva/ioperf/file_write.00000009
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12385876 0xbcfe54 0
6 12599123 0xc03f53 0
0 12427528 0xbda108 0
2 12573868 0xbfdcac 0
4 12515600 0xbef910 0
7 12464225 0xbe3061 0
1 12645512 0xc0f488 0
3 12493810 0xbea3f2 0
/esfs/jtacquaviva/ioperf/file_write.00000015
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12645492 0xc0f474 0
3 12493790 0xbea3de 0
5 12385856 0xbcfe40 0
6 12599103 0xc03f3f 0
0 12427508 0xbda0f4 0
2 12573848 0xbfdc98 0
4 12515581 0xbef8fd 0
7 12464205 0xbe304d 0
/esfs/jtacquaviva/ioperf/file_write.00000027
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464210 0xbe3052 0
1 12645497 0xc0f479 0
3 12493795 0xbea3e3 0
5 12385861 0xbcfe45 0
6 12599108 0xc03f44 0
0 12427513 0xbda0f9 0
2 12573853 0xbfdc9d 0
4 12515585 0xbef901 0
/esfs/jtacquaviva/ioperf/file_write.00000024
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12645481 0xc0f469 0
6 12599092 0xc03f34 0
5 12385845 0xbcfe35 0
3 12493779 0xbea3d3 0
2 12573837 0xbfdc8d 0
7 12464195 0xbe3043 0
4 12515570 0xbef8f2 0
0 12427499 0xbda0eb 0
/esfs/jtacquaviva/ioperf/file_write.00000001
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12515578 0xbef8fa 0
7 12464203 0xbe304b 0
1 12645490 0xc0f472 0
3 12493788 0xbea3dc 0
5 12385854 0xbcfe3e 0
6 12599101 0xc03f3d 0
0 12427506 0xbda0f2 0
2 12573846 0xbfdc96 0
/esfs/jtacquaviva/ioperf/file_write.00000011
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12385867 0xbcfe4b 0
6 12599115 0xc03f4b 0
7 12464217 0xbe3059 0
1 12645504 0xc0f480 0
0 12427520 0xbda100 0
2 12573860 0xbfdca4 0
4 12515592 0xbef908 0
3 12493802 0xbea3ea 0

599
output/COUNT:1#NN:8#PPN:4#API:POSIX#T:10485760.txt Normal file
View File

@ -0,0 +1,599 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 40265318400 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:4#API:POSIX#T:10485760.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 03:12:53 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 40265318400 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 03:12:53 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 32 (4 per node)
repetitions = 3
xfersize = 10 MiB
blocksize = 37.50 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288154644480.
WARNING: Using actual aggregate bytes moved = 396697272320.
write 5553 39321600 10240 0.001622 68.12 0.000543 68.13 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1284966973440.
WARNING: Using actual aggregate bytes moved = 391297105920.
write 5525 39321600 10240 0.009448 67.54 0.000672 67.55 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1286602752000.
WARNING: Using actual aggregate bytes moved = 391611678720.
write 5503 39321600 10240 0.015847 67.86 0.000832 67.87 2
Max Write: 5553.20 MiB/sec (5822.95 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 5553.20 5502.54 5526.77 20.74 67.84871 0 32 4 3 1 0 1 0 0 1 40265318400 10485760 396697272320 POSIX 0
Finished: Wed Oct 24 03:16:22 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 40265318400 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
+ tee -a ./output/COUNT:1#NN:8#PPN:4#API:POSIX#T:10485760.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 03:16:39 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 40265318400 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 03:16:39 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/indread8/file
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 32 (4 per node)
repetitions = 3
xfersize = 10 MiB
blocksize = 37.50 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 5153960755200.
WARNING: Using actual aggregate bytes moved = 169198223360.
read 2681.48 39321600 10240 0.001316 60.17 0.000816 60.18 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 5153960755200.
WARNING: Using actual aggregate bytes moved = 191574835200.
read 3033.32 39321600 10240 0.000779 60.23 0.001073 60.23 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 5153960755200.
WARNING: Using actual aggregate bytes moved = 211235635200.
read 3351.82 39321600 10240 0.000796 60.10 0.000797 60.10 2
Max Read: 3351.82 MiB/sec (3514.63 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 3351.82 2681.48 3022.21 273.78 60.16949 0 32 4 3 1 0 1 0 0 1 40265318400 10485760 169198223360 POSIX 0
Finished: Wed Oct 24 03:19:40 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write.00000006
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12493276 0xbea1dc 0
4 12515068 0xbef6fc 0
5 12385340 0xbcfc3c 0
7 12463689 0xbe2e49 0
1 12644976 0xc0f270 0
6 12598591 0xc03d3f 0
0 12426995 0xbd9ef3 0
2 12573333 0xbfda95 0
/esfs/jtacquaviva/ioperf/file_write.00000023
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12515059 0xbef6f3 0
3 12493267 0xbea1d3 0
2 12573324 0xbfda8c 0
6 12598585 0xc03d39 0
5 12385335 0xbcfc37 0
7 12463684 0xbe2e44 0
1 12644972 0xc0f26c 0
0 12426991 0xbd9eef 0
/esfs/jtacquaviva/ioperf/file_write.00000002
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12463704 0xbe2e58 0
1 12644991 0xc0f27f 0
3 12493293 0xbea1ed 0
5 12385355 0xbcfc4b 0
6 12598607 0xc03d4f 0
0 12427011 0xbd9f03 0
2 12573348 0xbfdaa4 0
4 12515084 0xbef70c 0
/esfs/jtacquaviva/ioperf/file_write.00000026
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12426986 0xbd9eea 0
4 12515061 0xbef6f5 0
5 12385334 0xbcfc36 0
1 12644969 0xc0f269 0
3 12493272 0xbea1d8 0
7 12463682 0xbe2e42 0
2 12573328 0xbfda90 0
/esfs/jtacquaviva/ioperf/file_write.00000000
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12463695 0xbe2e4f 0
4 12515073 0xbef701 0
6 12598598 0xc03d46 0
2 12573339 0xbfda9b 0
1 12644983 0xc0f277 0
3 12493287 0xbea1e7 0
5 12385350 0xbcfc46 0
0 12427006 0xbd9efe 0
/esfs/jtacquaviva/ioperf/file_write.00000020
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12598589 0xc03d3d 0
7 12463687 0xbe2e47 0
0 12426993 0xbd9ef1 0
3 12493275 0xbea1db 0
5 12385338 0xbcfc3a 0
2 12573330 0xbfda92 0
1 12644974 0xc0f26e 0
/esfs/jtacquaviva/ioperf/file_write.00000030
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12493297 0xbea1f1 0
5 12385359 0xbcfc4f 0
6 12598611 0xc03d53 0
0 12427015 0xbd9f07 0
2 12573352 0xbfdaa8 0
4 12515088 0xbef710 0
7 12463709 0xbe2e5d 0
1 12644996 0xc0f284 0
/esfs/jtacquaviva/ioperf/file_write.00000004
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12385332 0xbcfc34 0
6 12598583 0xc03d37 0
7 12463680 0xbe2e40 0
1 12644968 0xc0f268 0
3 12493269 0xbea1d5 0
2 12573327 0xbfda8f 0
4 12515064 0xbef6f8 0
0 12426990 0xbd9eee 0
/esfs/jtacquaviva/ioperf/file_write.00000007
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12644993 0xc0f281 0
3 12493294 0xbea1ee 0
0 12427012 0xbd9f04 0
2 12573349 0xbfdaa5 0
4 12515085 0xbef70d 0
5 12385356 0xbcfc4c 0
6 12598609 0xc03d51 0
7 12463707 0xbe2e5b 0
/esfs/jtacquaviva/ioperf/file_write.00000018
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12515069 0xbef6fd 0
5 12385342 0xbcfc3e 0
2 12573334 0xbfda96 0
3 12493280 0xbea1e0 0
7 12463693 0xbe2e4d 0
6 12598594 0xc03d42 0
0 12426999 0xbd9ef7 0
1 12644980 0xc0f274 0
/esfs/jtacquaviva/ioperf/file_write.00000031
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12644995 0xc0f283 0
3 12493296 0xbea1f0 0
5 12385358 0xbcfc4e 0
6 12598610 0xc03d52 0
0 12427014 0xbd9f06 0
2 12573351 0xbfdaa7 0
4 12515087 0xbef70f 0
7 12463708 0xbe2e5c 0
/esfs/jtacquaviva/ioperf/file_write.00000012
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12598590 0xc03d3e 0
4 12515067 0xbef6fb 0
5 12385339 0xbcfc3b 0
1 12644975 0xc0f26f 0
7 12463688 0xbe2e48 0
0 12426994 0xbd9ef2 0
3 12493277 0xbea1dd 0
2 12573331 0xbfda93 0
/esfs/jtacquaviva/ioperf/file_write.00000013
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12515082 0xbef70a 0
7 12463705 0xbe2e59 0
1 12644992 0xc0f280 0
3 12493292 0xbea1ec 0
5 12385354 0xbcfc4a 0
6 12598606 0xc03d4e 0
0 12427010 0xbd9f02 0
2 12573347 0xbfdaa3 0
/esfs/jtacquaviva/ioperf/file_write.00000025
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12493285 0xbea1e5 0
5 12385348 0xbcfc44 0
6 12598602 0xc03d4a 0
0 12427005 0xbd9efd 0
2 12573343 0xbfda9f 0
4 12515079 0xbef707 0
7 12463700 0xbe2e54 0
1 12644987 0xc0f27b 0
/esfs/jtacquaviva/ioperf/file_write.00000022
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12463694 0xbe2e4e 0
5 12385344 0xbcfc40 0
1 12644982 0xc0f276 0
6 12598597 0xc03d45 0
3 12493286 0xbea1e6 0
0 12427002 0xbd9efa 0
2 12573340 0xbfda9c 0
4 12515077 0xbef705 0
/esfs/jtacquaviva/ioperf/file_write.00000003
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573332 0xbfda94 0
1 12644977 0xc0f271 0
4 12515070 0xbef6fe 0
0 12426997 0xbd9ef5 0
3 12493279 0xbea1df 0
6 12598592 0xc03d40 0
7 12463692 0xbe2e4c 0
/esfs/jtacquaviva/ioperf/file_write.00000008
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573345 0xbfdaa1 0
4 12515081 0xbef709 0
7 12463703 0xbe2e57 0
1 12644989 0xc0f27d 0
3 12493291 0xbea1eb 0
5 12385353 0xbcfc49 0
6 12598604 0xc03d4c 0
0 12427009 0xbd9f01 0
/esfs/jtacquaviva/ioperf/file_write.00000028
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12385331 0xbcfc33 0
2 12573322 0xbfda8a 0
7 12463679 0xbe2e3f 0
4 12515060 0xbef6f4 0
3 12493268 0xbea1d4 0
0 12426989 0xbd9eed 0
6 12598586 0xc03d3a 0
1 12644971 0xc0f26b 0
/esfs/jtacquaviva/ioperf/file_write.00000010
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12427000 0xbd9ef8 0
3 12493284 0xbea1e4 0
5 12385347 0xbcfc43 0
6 12598600 0xc03d48 0
2 12573341 0xbfda9d 0
4 12515076 0xbef704 0
7 12463698 0xbe2e52 0
1 12644985 0xc0f279 0
/esfs/jtacquaviva/ioperf/file_write.00000017
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573337 0xbfda99 0
1 12644981 0xc0f275 0
3 12493282 0xbea1e2 0
6 12598596 0xc03d44 0
5 12385345 0xbcfc41 0
0 12427001 0xbd9ef9 0
4 12515075 0xbef703 0
7 12463697 0xbe2e51 0
/esfs/jtacquaviva/ioperf/file_write.00000029
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12385360 0xbcfc50 0
6 12598612 0xc03d54 0
0 12427016 0xbd9f08 0
2 12573353 0xbfdaa9 0
4 12515089 0xbef711 0
7 12463710 0xbe2e5e 0
1 12644997 0xc0f285 0
3 12493298 0xbea1f2 0
/esfs/jtacquaviva/ioperf/file_write.00000016
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12598584 0xc03d38 0
7 12463681 0xbe2e41 0
0 12426988 0xbd9eec 0
2 12573326 0xbfda8e 0
3 12493273 0xbea1d9 0
5 12385336 0xbcfc38 0
4 12515065 0xbef6f9 0
/esfs/jtacquaviva/ioperf/file_write.00000005
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12385346 0xbcfc42 0
6 12598599 0xc03d47 0
0 12427003 0xbd9efb 0
2 12573342 0xbfda9e 0
4 12515078 0xbef706 0
7 12463699 0xbe2e53 0
1 12644986 0xc0f27a 0
3 12493288 0xbea1e8 0
/esfs/jtacquaviva/ioperf/file_write.00000014
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12644967 0xc0f267 0
6 12598582 0xc03d36 0
0 12426985 0xbd9ee9 0
5 12385333 0xbcfc35 0
4 12515062 0xbef6f6 0
2 12573325 0xbfda8d 0
3 12493271 0xbea1d7 0
7 12463683 0xbe2e43 0
/esfs/jtacquaviva/ioperf/file_write.00000021
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12385351 0xbcfc47 0
2 12573344 0xbfdaa0 0
4 12515080 0xbef708 0
1 12644988 0xc0f27c 0
6 12598603 0xc03d4b 0
7 12463702 0xbe2e56 0
3 12493290 0xbea1ea 0
0 12427008 0xbd9f00 0
/esfs/jtacquaviva/ioperf/file_write.00000019
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12427007 0xbd9eff 0
7 12463701 0xbe2e55 0
3 12493289 0xbea1e9 0
5 12385352 0xbcfc48 0
2 12573346 0xbfdaa2 0
1 12644990 0xc0f27e 0
4 12515083 0xbef70b 0
6 12598605 0xc03d4d 0
/esfs/jtacquaviva/ioperf/file_write.00000009
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12463690 0xbe2e4a 0
5 12385341 0xbcfc3d 0
4 12515071 0xbef6ff 0
2 12573335 0xbfda97 0
0 12426998 0xbd9ef6 0
1 12644979 0xc0f273 0
3 12493281 0xbea1e1 0
6 12598595 0xc03d43 0
/esfs/jtacquaviva/ioperf/file_write.00000015
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573338 0xbfda9a 0
7 12463696 0xbe2e50 0
3 12493283 0xbea1e3 0
4 12515074 0xbef702 0
5 12385349 0xbcfc45 0
6 12598601 0xc03d49 0
0 12427004 0xbd9efc 0
1 12644984 0xc0f278 0
/esfs/jtacquaviva/ioperf/file_write.00000027
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12598608 0xc03d50 0
7 12463706 0xbe2e5a 0
1 12644994 0xc0f282 0
0 12427013 0xbd9f05 0
2 12573350 0xbfdaa6 0
3 12493295 0xbea1ef 0
5 12385357 0xbcfc4d 0
4 12515086 0xbef70e 0
/esfs/jtacquaviva/ioperf/file_write.00000024
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12493278 0xbea1de 0
0 12426996 0xbd9ef4 0
1 12644978 0xc0f272 0
7 12463691 0xbe2e4b 0
6 12598593 0xc03d41 0
4 12515072 0xbef700 0
5 12385343 0xbcfc3f 0
2 12573336 0xbfda98 0
/esfs/jtacquaviva/ioperf/file_write.00000001
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573323 0xbfda8b 0
0 12426987 0xbd9eeb 0
4 12515063 0xbef6f7 0
3 12493270 0xbea1d6 0
1 12644970 0xc0f26a 0
7 12463685 0xbe2e45 0
6 12598587 0xc03d3b 0
/esfs/jtacquaviva/ioperf/file_write.00000011
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12644973 0xc0f26d 0
3 12493274 0xbea1da 0
5 12385337 0xbcfc39 0
6 12598588 0xc03d3c 0
0 12426992 0xbd9ef0 0
2 12573329 0xbfda91 0
4 12515066 0xbef6fa 0
7 12463686 0xbe2e46 0

600
output/COUNT:1#NN:8#PPN:4#API:POSIX#T:16384.txt Normal file
View File

@ -0,0 +1,600 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 40265318400 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:4#API:POSIX#T:16384.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 06:39:18 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 40265318400 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 06:39:18 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 32 (4 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 37.50 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288488091648.
WARNING: Using actual aggregate bytes moved = 32587415552.
write 267.29 39321600 16.00 0.002044 116.27 0.000366 116.27 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288455798784.
WARNING: Using actual aggregate bytes moved = 27252424704.
write 221.19 39321600 16.00 0.014096 117.49 0.000384 117.50 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288337129472.
WARNING: Using actual aggregate bytes moved = 27642986496.
write 220.63 39321600 16.00 0.002227 119.49 0.000433 119.49 2
Max Write: 267.29 MiB/sec (280.27 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 267.29 220.63 236.37 21.87 117.75372 0 32 4 3 1 0 1 0 0 1 40265318400 16384 32587415552 POSIX 0
Finished: Wed Oct 24 06:45:12 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ tee -a ./output/COUNT:1#NN:8#PPN:4#API:POSIX#T:16384.txt
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 40265318400 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 06:45:17 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 40265318400 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 06:45:17 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/indread8/file
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 32 (4 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 37.50 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 5153960755200.
WARNING: Using actual aggregate bytes moved = 3915972608.
read 62.08 39321600 16.00 0.001261 60.15 0.000709 60.15 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 5153960755200.
WARNING: Using actual aggregate bytes moved = 5149704192.
read 81.65 39321600 16.00 0.000742 60.15 0.000776 60.15 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 5153960755200.
WARNING: Using actual aggregate bytes moved = 5127798784.
read 81.31 39321600 16.00 0.001570 60.14 0.000759 60.15 2
Max Read: 81.65 MiB/sec (85.61 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 81.65 62.08 75.01 9.14 60.15091 0 32 4 3 1 0 1 0 0 1 40265318400 16384 3915972608 POSIX 0
Finished: Wed Oct 24 06:48:18 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write.00000006
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12386874 0xbd023a 0
6 12600115 0xc04333 0
4 12516594 0xbefcf2 0
7 12465225 0xbe3449 0
1 12646510 0xc0f86e 0
0 12428520 0xbda4e8 0
2 12574869 0xbfe095 0
3 12494813 0xbea7dd 0
/esfs/jtacquaviva/ioperf/file_write.00000023
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494814 0xbea7de 0
5 12386877 0xbd023d 0
6 12600118 0xc04336 0
1 12646513 0xc0f871 0
7 12465229 0xbe344d 0
4 12516598 0xbefcf6 0
2 12574872 0xbfe098 0
0 12428525 0xbda4ed 0
/esfs/jtacquaviva/ioperf/file_write.00000002
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12386883 0xbd0243 0
6 12600124 0xc0433c 0
0 12428531 0xbda4f3 0
4 12516605 0xbefcfd 0
1 12646522 0xc0f87a 0
7 12465236 0xbe3454 0
2 12574881 0xbfe0a1 0
3 12494826 0xbea7ea 0
/esfs/jtacquaviva/ioperf/file_write.00000026
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12574892 0xbfe0ac 0
4 12516618 0xbefd0a 0
7 12465248 0xbe3460 0
1 12646534 0xc0f886 0
3 12494835 0xbea7f3 0
5 12386897 0xbd0251 0
6 12600138 0xc0434a 0
0 12428544 0xbda500 0
/esfs/jtacquaviva/ioperf/file_write.00000000
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12465228 0xbe344c 0
1 12646515 0xc0f873 0
3 12494816 0xbea7e0 0
2 12574874 0xbfe09a 0
4 12516600 0xbefcf8 0
6 12600121 0xc04339 0
5 12386879 0xbd023f 0
0 12428527 0xbda4ef 0
/esfs/jtacquaviva/ioperf/file_write.00000020
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12465240 0xbe3458 0
1 12646526 0xc0f87e 0
3 12494827 0xbea7eb 0
5 12386889 0xbd0249 0
6 12600130 0xc04342 0
0 12428536 0xbda4f8 0
2 12574884 0xbfe0a4 0
4 12516610 0xbefd02 0
/esfs/jtacquaviva/ioperf/file_write.00000030
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12600133 0xc04345 0
0 12428539 0xbda4fb 0
2 12574887 0xbfe0a7 0
1 12646529 0xc0f881 0
3 12494830 0xbea7ee 0
4 12516613 0xbefd05 0
5 12386893 0xbd024d 0
7 12465244 0xbe345c 0
/esfs/jtacquaviva/ioperf/file_write.00000004
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12646523 0xc0f87b 0
6 12600126 0xc0433e 0
5 12386887 0xbd0247 0
7 12465238 0xbe3456 0
0 12428534 0xbda4f6 0
4 12516609 0xbefd01 0
2 12574883 0xbfe0a3 0
/esfs/jtacquaviva/ioperf/file_write.00000007
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12465243 0xbe345b 0
5 12386892 0xbd024c 0
2 12574888 0xbfe0a8 0
1 12646530 0xc0f882 0
6 12600134 0xc04346 0
0 12428540 0xbda4fc 0
3 12494832 0xbea7f0 0
4 12516614 0xbefd06 0
/esfs/jtacquaviva/ioperf/file_write.00000018
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12574879 0xbfe09f 0
3 12494825 0xbea7e9 0
6 12600127 0xc0433f 0
0 12428533 0xbda4f5 0
1 12646524 0xc0f87c 0
4 12516608 0xbefd00 0
7 12465239 0xbe3457 0
/esfs/jtacquaviva/ioperf/file_write.00000031
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12386898 0xbd0252 0
1 12646536 0xc0f888 0
6 12600140 0xc0434c 0
0 12428546 0xbda502 0
2 12574894 0xbfe0ae 0
4 12516620 0xbefd0c 0
7 12465250 0xbe3462 0
3 12494837 0xbea7f5 0
/esfs/jtacquaviva/ioperf/file_write.00000012
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428542 0xbda4fe 0
7 12465246 0xbe345e 0
3 12494834 0xbea7f2 0
2 12574890 0xbfe0aa 0
4 12516616 0xbefd08 0
1 12646532 0xc0f884 0
5 12386896 0xbd0250 0
6 12600137 0xc04349 0
/esfs/jtacquaviva/ioperf/file_write.00000013
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494824 0xbea7e8 0
4 12516606 0xbefcfe 0
2 12574880 0xbfe0a0 0
7 12465237 0xbe3455 0
1 12646525 0xc0f87d 0
6 12600129 0xc04341 0
0 12428535 0xbda4f7 0
5 12386888 0xbd0248 0
/esfs/jtacquaviva/ioperf/file_write.00000025
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12516597 0xbefcf5 0
3 12494817 0xbea7e1 0
7 12465230 0xbe344e 0
2 12574875 0xbfe09b 0
5 12386878 0xbd023e 0
1 12646517 0xc0f875 0
0 12428526 0xbda4ee 0
/esfs/jtacquaviva/ioperf/file_write.00000022
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12646518 0xc0f876 0
2 12574877 0xbfe09d 0
7 12465233 0xbe3451 0
4 12516603 0xbefcfb 0
0 12428530 0xbda4f2 0
6 12600123 0xc0433b 0
5 12386885 0xbd0245 0
3 12494821 0xbea7e5 0
/esfs/jtacquaviva/ioperf/file_write.00000003
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12600141 0xc0434d 0
0 12428547 0xbda503 0
2 12574895 0xbfe0af 0
4 12516621 0xbefd0d 0
7 12465251 0xbe3463 0
1 12646537 0xc0f889 0
3 12494838 0xbea7f6 0
5 12386900 0xbd0254 0
/esfs/jtacquaviva/ioperf/file_write.00000008
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12646527 0xc0f87f 0
3 12494828 0xbea7ec 0
0 12428537 0xbda4f9 0
2 12574885 0xbfe0a5 0
7 12465241 0xbe3459 0
5 12386891 0xbd024b 0
6 12600132 0xc04344 0
4 12516612 0xbefd04 0
/esfs/jtacquaviva/ioperf/file_write.00000028
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12465235 0xbe3453 0
3 12494822 0xbea7e6 0
1 12646521 0xc0f879 0
5 12386886 0xbd0246 0
0 12428532 0xbda4f4 0
4 12516607 0xbefcff 0
2 12574882 0xbfe0a2 0
6 12600128 0xc04340 0
/esfs/jtacquaviva/ioperf/file_write.00000010
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428523 0xbda4eb 0
1 12646514 0xc0f872 0
2 12574871 0xbfe097 0
3 12494818 0xbea7e2 0
4 12516599 0xbefcf7 0
6 12600120 0xc04338 0
7 12465232 0xbe3450 0
5 12386881 0xbd0241 0
/esfs/jtacquaviva/ioperf/file_write.00000017
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494833 0xbea7f1 0
5 12386895 0xbd024f 0
2 12574891 0xbfe0ab 0
4 12516617 0xbefd09 0
1 12646533 0xc0f885 0
6 12600136 0xc04348 0
0 12428543 0xbda4ff 0
7 12465247 0xbe345f 0
/esfs/jtacquaviva/ioperf/file_write.00000029
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12574868 0xbfe094 0
3 12494812 0xbea7dc 0
4 12516595 0xbefcf3 0
7 12465226 0xbe344a 0
5 12386875 0xbd023b 0
6 12600116 0xc04334 0
1 12646511 0xc0f86f 0
0 12428521 0xbda4e9 0
/esfs/jtacquaviva/ioperf/file_write.00000016
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12516596 0xbefcf4 0
7 12465227 0xbe344b 0
5 12386876 0xbd023c 0
1 12646512 0xc0f870 0
2 12574870 0xbfe096 0
0 12428522 0xbda4ea 0
6 12600117 0xc04335 0
3 12494815 0xbea7df 0
/esfs/jtacquaviva/ioperf/file_write.00000005
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12516619 0xbefd0b 0
7 12465249 0xbe3461 0
1 12646535 0xc0f887 0
6 12600139 0xc0434b 0
0 12428545 0xbda501 0
2 12574893 0xbfe0ad 0
3 12494836 0xbea7f4 0
5 12386899 0xbd0253 0
/esfs/jtacquaviva/ioperf/file_write.00000014
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12646538 0xc0f88a 0
6 12600142 0xc0434e 0
3 12494839 0xbea7f7 0
0 12428549 0xbda505 0
5 12386902 0xbd0256 0
2 12574897 0xbfe0b1 0
4 12516623 0xbefd0f 0
7 12465253 0xbe3465 0
/esfs/jtacquaviva/ioperf/file_write.00000021
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12465223 0xbe3447 0
5 12386872 0xbd0238 0
0 12428518 0xbda4e6 0
2 12574866 0xbfe092 0
1 12646508 0xc0f86c 0
3 12494811 0xbea7db 0
6 12600114 0xc04332 0
4 12516593 0xbefcf1 0
/esfs/jtacquaviva/ioperf/file_write.00000019
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428548 0xbda504 0
2 12574896 0xbfe0b0 0
4 12516622 0xbefd0e 0
3 12494840 0xbea7f8 0
5 12386901 0xbd0255 0
7 12465252 0xbe3464 0
6 12600143 0xc0434f 0
1 12646539 0xc0f88b 0
/esfs/jtacquaviva/ioperf/file_write.00000009
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12646520 0xc0f878 0
0 12428529 0xbda4f1 0
4 12516604 0xbefcfc 0
5 12386884 0xbd0244 0
6 12600125 0xc0433d 0
2 12574878 0xbfe09e 0
3 12494823 0xbea7e7 0
/esfs/jtacquaviva/ioperf/file_write.00000015
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428524 0xbda4ec 0
6 12600119 0xc04337 0
2 12574873 0xbfe099 0
1 12646516 0xc0f874 0
3 12494819 0xbea7e3 0
7 12465231 0xbe344f 0
5 12386880 0xbd0240 0
4 12516601 0xbefcf9 0
/esfs/jtacquaviva/ioperf/file_write.00000027
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12516602 0xbefcfa 0
6 12600122 0xc0433a 0
2 12574876 0xbfe09c 0
3 12494820 0xbea7e4 0
5 12386882 0xbd0242 0
0 12428528 0xbda4f0 0
1 12646519 0xc0f877 0
7 12465234 0xbe3452 0
/esfs/jtacquaviva/ioperf/file_write.00000024
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12600131 0xc04343 0
4 12516611 0xbefd03 0
1 12646528 0xc0f880 0
0 12428538 0xbda4fa 0
5 12386890 0xbd024a 0
2 12574886 0xbfe0a6 0
7 12465242 0xbe345a 0
3 12494829 0xbea7ed 0
/esfs/jtacquaviva/ioperf/file_write.00000001
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12646531 0xc0f883 0
3 12494831 0xbea7ef 0
5 12386894 0xbd024e 0
6 12600135 0xc04347 0
0 12428541 0xbda4fd 0
2 12574889 0xbfe0a9 0
4 12516615 0xbefd07 0
7 12465245 0xbe345d 0
/esfs/jtacquaviva/ioperf/file_write.00000011
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494810 0xbea7da 0
6 12600113 0xc04331 0
4 12516592 0xbefcf0 0
5 12386873 0xbd0239 0
0 12428519 0xbda4e7 0
2 12574867 0xbfe093 0
7 12465224 0xbe3448 0
1 12646509 0xc0f86d 0

110
output/COUNT:1#NN:8#PPN:6#API:MPIIO#T:102400.txt Normal file
View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 48 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 26843545600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:6#API:MPIIO#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 06:03:30 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 26843545600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 06:03:30 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 48 (6 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 25 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288489574400.
WARNING: Using actual aggregate bytes moved = 24692736000.
write 387.51 26214400 100.00 0.010085 60.76 0.000485 60.77 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288487116800.
WARNING: Using actual aggregate bytes moved = 24790630400.
write 389.21 26214400 100.00 0.001933 60.74 0.000494 60.74 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288489881600.
WARNING: Using actual aggregate bytes moved = 24570265600.
write 385.76 26214400 100.00 0.002338 60.74 0.000528 60.74 2
Max Write: 389.21 MiB/sec (408.11 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 389.21 385.76 387.49 1.41 60.75249 0 48 6 3 0 0 1 0 0 1 26843545600 102400 24692736000 MPIIO 0
Finished: Wed Oct 24 06:06:33 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 48 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 48 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 26843545600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
+ tee -a ./output/COUNT:1#NN:8#PPN:6#API:MPIIO#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 06:06:41 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 26843545600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 06:06:41 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 48 (6 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 25 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 21360230400.
read 335.58 26214400 100.00 0.018368 60.68 0.000767 60.70 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 21834342400.
read 343.08 26214400 100.00 0.002198 60.69 0.000792 60.69 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 22139699200.
read 347.93 26214400 100.00 0.002467 60.68 0.000849 60.69 2
Max Read: 347.93 MiB/sec (364.83 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 347.93 335.58 342.19 5.08 60.69428 0 48 6 3 0 0 1 0 0 1 26843545600 102400 21360230400 MPIIO 0
Finished: Wed Oct 24 06:09:43 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494498 0xbea6a2 0
5 12386568 0xbd0108 0
6 12599809 0xc04201 0
0 12428211 0xbda3b3 0
2 12574557 0xbfdf5d 0
4 12516284 0xbefbbc 0
7 12464914 0xbe3312 0
1 12646197 0xc0f735 0

110
output/COUNT:1#NN:8#PPN:6#API:MPIIO#T:1048576.txt Normal file
View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 48 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 26843545600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:6#API:MPIIO#T:1048576.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 04:53:21 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 26843545600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 04:53:21 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 48 (6 per node)
repetitions = 3
xfersize = 1 MiB
blocksize = 25 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288490188800.
WARNING: Using actual aggregate bytes moved = 294345768960.
write 4669 26214400 1024.00 0.010255 60.10 0.007472 60.12 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288490188800.
WARNING: Using actual aggregate bytes moved = 286901927936.
write 4535 26214400 1024.00 0.224817 60.11 0.000744 60.33 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288470265856.
WARNING: Using actual aggregate bytes moved = 287090671616.
write 4544 26214400 1024.00 0.143342 60.11 0.000853 60.25 2
Max Write: 4669.30 MiB/sec (4896.11 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 4669.30 4535.12 4582.85 61.24 60.23384 0 48 6 3 0 0 1 0 0 1 26843545600 1048576 294345768960 MPIIO 0
Finished: Wed Oct 24 04:56:28 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 48 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 48 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 26843545600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
+ tee -a ./output/COUNT:1#NN:8#PPN:6#API:MPIIO#T:1048576.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 04:56:47 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 26843545600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 04:56:47 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 48 (6 per node)
repetitions = 3
xfersize = 1 MiB
blocksize = 25 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 89326092288.
read 1414.48 26214400 1024.00 0.116877 60.11 0.002345 60.23 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 91959066624.
read 1458.33 26214400 1024.00 0.003724 60.13 0.002358 60.14 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 92708798464.
read 1470.67 26214400 1024.00 0.004265 60.11 0.002330 60.12 2
Max Read: 1470.67 MiB/sec (1542.10 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 1470.67 1414.48 1447.83 24.11 60.16018 0 48 6 3 0 0 1 0 0 1 26843545600 1048576 89326092288 MPIIO 0
Finished: Wed Oct 24 04:59:48 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12574044 0xbfdd5c 0
4 12515773 0xbef9bd 0
7 12464399 0xbe310f 0
1 12645688 0xc0f538 0
3 12493985 0xbea4a1 0
5 12386053 0xbcff05 0
6 12599297 0xc04001 0
0 12427702 0xbda1b6 0

110
output/COUNT:1#NN:8#PPN:6#API:MPIIO#T:10485760.txt Normal file
View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 48 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 26843545600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:6#API:MPIIO#T:10485760.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 03:46:15 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 26843545600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 03:46:15 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 48 (6 per node)
repetitions = 3
xfersize = 10 MiB
blocksize = 25 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288427274240.
WARNING: Using actual aggregate bytes moved = 203077713920.
write 3215.70 26214400 10240 0.015478 60.21 0.000596 60.23 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288217559040.
WARNING: Using actual aggregate bytes moved = 195790110720.
write 3081.74 26214400 10240 0.176834 60.41 0.000714 60.59 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288469217280.
WARNING: Using actual aggregate bytes moved = 202343710720.
write 3169.02 26214400 10240 0.021270 60.87 0.000804 60.89 2
Max Write: 3215.70 MiB/sec (3371.90 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 3215.70 3081.74 3155.49 55.52 60.56940 0 48 6 3 0 0 1 0 0 1 26843545600 10485760 203077713920 MPIIO 0
Finished: Wed Oct 24 03:49:22 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 48 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 48 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 26843545600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
+ tee -a ./output/COUNT:1#NN:8#PPN:6#API:MPIIO#T:10485760.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 03:49:41 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 26843545600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 03:49:41 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 48 (6 per node)
repetitions = 3
xfersize = 10 MiB
blocksize = 25 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 195171450880.
read 3089.98 26214400 10240 0.062367 60.17 0.008497 60.24 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 199470612480.
read 3163.62 26214400 10240 0.006616 60.13 0.005305 60.13 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 200057815040.
read 3171.48 26214400 10240 0.007063 60.16 0.005277 60.16 2
Max Read: 3171.48 MiB/sec (3325.53 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 3171.48 3089.98 3141.69 36.70 60.17508 0 48 6 3 0 0 1 0 0 1 26843545600 10485760 195171450880 MPIIO 0
Finished: Wed Oct 24 03:52:42 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12515263 0xbef7bf 0
7 12463885 0xbe2f0d 0
1 12645174 0xc0f336 0
3 12493473 0xbea2a1 0
5 12385538 0xbcfd02 0
6 12598783 0xc03dff 0
0 12427191 0xbd9fb7 0
2 12573529 0xbfdb59 0

110
output/COUNT:1#NN:8#PPN:6#API:MPIIO#T:16384.txt Normal file
View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 48 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 26843545600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:6#API:MPIIO#T:16384.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 07:19:24 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 26843545600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 07:19:24 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 48 (6 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 25 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288483831808.
WARNING: Using actual aggregate bytes moved = 4463050752.
write 66.13 26214400 16.00 0.015352 64.34 0.006864 64.36 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288486223872.
WARNING: Using actual aggregate bytes moved = 4473749504.
write 66.08 26214400 16.00 0.001987 64.56 0.000559 64.57 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288484454400.
WARNING: Using actual aggregate bytes moved = 4464164864.
write 66.30 26214400 16.00 0.002209 64.21 0.000578 64.22 2
Max Write: 66.30 MiB/sec (69.52 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 66.30 66.08 66.17 0.09 64.38060 0 48 6 3 0 0 1 0 0 1 26843545600 16384 4463050752 MPIIO 0
Finished: Wed Oct 24 07:22:38 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 48 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ tee -a ./output/COUNT:1#NN:8#PPN:6#API:MPIIO#T:16384.txt
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 48 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 26843545600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 07:22:43 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 26843545600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 07:22:43 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 48 (6 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 25 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 4522311680.
read 67.19 26214400 16.00 0.015215 64.18 0.000954 64.19 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 4550836224.
read 67.61 26214400 16.00 0.002446 64.19 0.001371 64.19 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 4560273408.
read 67.76 26214400 16.00 0.003257 64.18 0.001555 64.18 2
Max Read: 67.76 MiB/sec (71.05 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 67.76 67.19 67.52 0.24 64.18861 0 48 6 3 0 0 1 0 0 1 26843545600 16384 4522311680 MPIIO 0
Finished: Wed Oct 24 07:25:55 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12387076 0xbd0304 0
6 12600319 0xc043ff 0
0 12428724 0xbda5b4 0
2 12575073 0xbfe161 0
4 12516798 0xbefdbe 0
7 12465431 0xbe3517 0
1 12646714 0xc0f93a 0
3 12495017 0xbea8a9 0

850
output/COUNT:1#NN:8#PPN:6#API:POSIX#T:102400.txt Normal file
View File

@ -0,0 +1,850 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 48 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 26843545600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:6#API:POSIX#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 05:55:47 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 26843545600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 05:55:47 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 48 (6 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 25 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288431206400.
WARNING: Using actual aggregate bytes moved = 86527283200.
write 918.39 26214400 100.00 0.003204 89.85 0.000397 89.85 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288349696000.
WARNING: Using actual aggregate bytes moved = 75418931200.
write 799.92 26214400 100.00 0.005326 89.91 0.000442 89.92 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288346009600.
WARNING: Using actual aggregate bytes moved = 77376921600.
write 807.16 26214400 100.00 0.006983 91.42 0.000547 91.42 2
Max Write: 918.39 MiB/sec (963.00 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 918.39 799.92 841.82 54.22 90.39670 0 48 6 3 1 0 1 0 0 1 26843545600 102400 86527283200 POSIX 0
Finished: Wed Oct 24 06:00:19 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 48 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 48 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 26843545600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
+ tee -a ./output/COUNT:1#NN:8#PPN:6#API:POSIX#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 06:00:28 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 26843545600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 06:00:28 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/indread8/file
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 48 (6 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 25 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 7730941132800.
WARNING: Using actual aggregate bytes moved = 35547340800.
read 564.70 26214400 100.00 0.004925 60.03 0.004252 60.03 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 7730941132800.
WARNING: Using actual aggregate bytes moved = 37631078400.
read 597.80 26214400 100.00 0.004236 60.03 0.004165 60.03 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 7730941132800.
WARNING: Using actual aggregate bytes moved = 38628454400.
read 613.67 26214400 100.00 0.004246 60.03 0.004178 60.03 2
Max Read: 613.67 MiB/sec (643.48 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 613.67 564.70 592.06 20.40 60.03218 0 48 6 3 1 0 1 0 0 1 26843545600 102400 35547340800 POSIX 0
Finished: Wed Oct 24 06:03:28 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write.00000006
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428180 0xbda394 0
2 12574527 0xbfdf3f 0
3 12494470 0xbea686 0
1 12646170 0xc0f71a 0
5 12386541 0xbd00ed 0
6 12599782 0xc041e6 0
4 12516257 0xbefba1 0
7 12464886 0xbe32f6 0
/esfs/jtacquaviva/ioperf/file_write.00000023
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428167 0xbda387 0
7 12464871 0xbe32e7 0
1 12646155 0xc0f70b 0
5 12386525 0xbd00dd 0
2 12574513 0xbfdf31 0
3 12494456 0xbea678 0
4 12516242 0xbefb92 0
6 12599769 0xc041d9 0
/esfs/jtacquaviva/ioperf/file_write.00000044
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12574515 0xbfdf33 0
4 12516243 0xbefb93 0
5 12386529 0xbd00e1 0
7 12464875 0xbe32eb 0
6 12599772 0xc041dc 0
0 12428171 0xbda38b 0
1 12646160 0xc0f710 0
3 12494460 0xbea67c 0
/esfs/jtacquaviva/ioperf/file_write.00000040
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12599786 0xc041ea 0
0 12428187 0xbda39b 0
2 12574533 0xbfdf45 0
4 12516261 0xbefba5 0
7 12464890 0xbe32fa 0
1 12646174 0xc0f71e 0
3 12494475 0xbea68b 0
5 12386545 0xbd00f1 0
/esfs/jtacquaviva/ioperf/file_write.00000002
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12599778 0xc041e2 0
4 12516252 0xbefb9c 0
5 12386536 0xbd00e8 0
7 12464883 0xbe32f3 0
1 12646166 0xc0f716 0
2 12574526 0xbfdf3e 0
3 12494468 0xbea684 0
0 12428182 0xbda396 0
/esfs/jtacquaviva/ioperf/file_write.00000026
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428174 0xbda38e 0
3 12494462 0xbea67e 0
2 12574521 0xbfdf39 0
5 12386533 0xbd00e5 0
4 12516250 0xbefb9a 0
1 12646164 0xc0f714 0
7 12464879 0xbe32ef 0
6 12599776 0xc041e0 0
/esfs/jtacquaviva/ioperf/file_write.00000033
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464894 0xbe32fe 0
5 12386549 0xbd00f5 0
0 12428191 0xbda39f 0
4 12516265 0xbefba9 0
1 12646178 0xc0f722 0
6 12599791 0xc041ef 0
2 12574538 0xbfdf4a 0
/esfs/jtacquaviva/ioperf/file_write.00000000
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494453 0xbea675 0
0 12428165 0xbda385 0
2 12574511 0xbfdf2f 0
7 12464869 0xbe32e5 0
5 12386523 0xbd00db 0
6 12599767 0xc041d7 0
1 12646154 0xc0f70a 0
4 12516240 0xbefb90 0
/esfs/jtacquaviva/ioperf/file_write.00000020
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494486 0xbea696 0
5 12386556 0xbd00fc 0
6 12599797 0xc041f5 0
0 12428199 0xbda3a7 0
2 12574544 0xbfdf50 0
4 12516272 0xbefbb0 0
7 12464902 0xbe3306 0
1 12646185 0xc0f729 0
/esfs/jtacquaviva/ioperf/file_write.00000030
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12646184 0xc0f728 0
3 12494485 0xbea695 0
5 12386555 0xbd00fb 0
6 12599796 0xc041f4 0
0 12428198 0xbda3a6 0
2 12574543 0xbfdf4f 0
4 12516271 0xbefbaf 0
7 12464901 0xbe3305 0
/esfs/jtacquaviva/ioperf/file_write.00000032
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12646152 0xc0f708 0
3 12494452 0xbea674 0
5 12386522 0xbd00da 0
6 12599765 0xc041d5 0
0 12428164 0xbda384 0
2 12574510 0xbfdf2e 0
4 12516238 0xbefb8e 0
7 12464868 0xbe32e4 0
/esfs/jtacquaviva/ioperf/file_write.00000004
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464878 0xbe32ee 0
1 12646163 0xc0f713 0
0 12428175 0xbda38f 0
3 12494463 0xbea67f 0
4 12516248 0xbefb98 0
2 12574522 0xbfdf3a 0
5 12386534 0xbd00e6 0
/esfs/jtacquaviva/ioperf/file_write.00000007
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12574535 0xbfdf47 0
4 12516263 0xbefba7 0
7 12464892 0xbe32fc 0
1 12646176 0xc0f720 0
3 12494477 0xbea68d 0
5 12386547 0xbd00f3 0
6 12599788 0xc041ec 0
0 12428189 0xbda39d 0
/esfs/jtacquaviva/ioperf/file_write.00000018
lmm_stripe_count: 6
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494479 0xbea68f 0
6 12599790 0xc041ee 0
2 12574537 0xbfdf49 0
7 12464895 0xbe32ff 0
5 12386550 0xbd00f6 0
0 12428192 0xbda3a0 0
/esfs/jtacquaviva/ioperf/file_write.00000041
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12516241 0xbefb91 0
3 12494455 0xbea677 0
6 12599768 0xc041d8 0
7 12464872 0xbe32e8 0
5 12386526 0xbd00de 0
1 12646156 0xc0f70c 0
0 12428168 0xbda388 0
2 12574514 0xbfdf32 0
/esfs/jtacquaviva/ioperf/file_write.00000031
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464873 0xbe32e9 0
5 12386528 0xbd00e0 0
4 12516245 0xbefb95 0
6 12599771 0xc041db 0
1 12646159 0xc0f70f 0
3 12494459 0xbea67b 0
0 12428172 0xbda38c 0
2 12574518 0xbfdf36 0
/esfs/jtacquaviva/ioperf/file_write.00000012
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428179 0xbda393 0
1 12646169 0xc0f719 0
5 12386538 0xbd00ea 0
4 12516255 0xbefb9f 0
2 12574529 0xbfdf41 0
3 12494471 0xbea687 0
6 12599781 0xc041e5 0
/esfs/jtacquaviva/ioperf/file_write.00000013
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428188 0xbda39c 0
2 12574534 0xbfdf46 0
4 12516262 0xbefba6 0
7 12464891 0xbe32fb 0
1 12646175 0xc0f71f 0
3 12494476 0xbea68c 0
5 12386546 0xbd00f2 0
6 12599787 0xc041eb 0
/esfs/jtacquaviva/ioperf/file_write.00000042
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12516276 0xbefbb4 0
7 12464906 0xbe330a 0
5 12386560 0xbd0100 0
6 12599801 0xc041f9 0
2 12574548 0xbfdf54 0
1 12646189 0xc0f72d 0
0 12428204 0xbda3ac 0
3 12494491 0xbea69b 0
/esfs/jtacquaviva/ioperf/file_write.00000025
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12599804 0xc041fc 0
0 12428206 0xbda3ae 0
2 12574551 0xbfdf57 0
4 12516279 0xbefbb7 0
7 12464909 0xbe330d 0
1 12646192 0xc0f730 0
3 12494493 0xbea69d 0
5 12386563 0xbd0103 0
/esfs/jtacquaviva/ioperf/file_write.00000046
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464899 0xbe3303 0
1 12646183 0xc0f727 0
3 12494483 0xbea693 0
5 12386554 0xbd00fa 0
6 12599795 0xc041f3 0
0 12428197 0xbda3a5 0
2 12574542 0xbfdf4e 0
4 12516270 0xbefbae 0
/esfs/jtacquaviva/ioperf/file_write.00000039
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12386531 0xbd00e3 0
1 12646161 0xc0f711 0
6 12599774 0xc041de 0
3 12494461 0xbea67d 0
2 12574519 0xbfdf37 0
7 12464877 0xbe32ed 0
0 12428173 0xbda38d 0
4 12516247 0xbefb97 0
/esfs/jtacquaviva/ioperf/file_write.00000022
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12574523 0xbfdf3b 0
4 12516251 0xbefb9b 0
7 12464881 0xbe32f1 0
1 12646165 0xc0f715 0
3 12494465 0xbea681 0
0 12428177 0xbda391 0
5 12386535 0xbd00e7 0
6 12599777 0xc041e1 0
/esfs/jtacquaviva/ioperf/file_write.00000034
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494473 0xbea689 0
5 12386543 0xbd00ef 0
6 12599784 0xc041e8 0
0 12428185 0xbda399 0
2 12574531 0xbfdf43 0
4 12516259 0xbefba3 0
7 12464888 0xbe32f8 0
1 12646172 0xc0f71c 0
/esfs/jtacquaviva/ioperf/file_write.00000003
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12386544 0xbd00f0 0
6 12599785 0xc041e9 0
0 12428186 0xbda39a 0
2 12574532 0xbfdf44 0
4 12516260 0xbefba4 0
7 12464889 0xbe32f9 0
1 12646173 0xc0f71d 0
3 12494474 0xbea68a 0
/esfs/jtacquaviva/ioperf/file_write.00000008
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12646157 0xc0f70d 0
3 12494458 0xbea67a 0
0 12428170 0xbda38a 0
5 12386530 0xbd00e2 0
2 12574517 0xbfdf35 0
6 12599773 0xc041dd 0
7 12464876 0xbe32ec 0
4 12516246 0xbefb96 0
/esfs/jtacquaviva/ioperf/file_write.00000038
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12386552 0xbd00f8 0
7 12464897 0xbe3301 0
0 12428195 0xbda3a3 0
2 12574541 0xbfdf4d 0
1 12646181 0xc0f725 0
4 12516269 0xbefbad 0
3 12494482 0xbea692 0
/esfs/jtacquaviva/ioperf/file_write.00000028
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428207 0xbda3af 0
2 12574552 0xbfdf58 0
4 12516280 0xbefbb8 0
7 12464910 0xbe330e 0
1 12646193 0xc0f731 0
3 12494494 0xbea69e 0
5 12386564 0xbd0104 0
6 12599805 0xc041fd 0
/esfs/jtacquaviva/ioperf/file_write.00000010
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12574508 0xbfdf2c 0
1 12646150 0xc0f706 0
3 12494450 0xbea672 0
6 12599763 0xc041d3 0
4 12516236 0xbefb8c 0
5 12386521 0xbd00d9 0
0 12428163 0xbda383 0
7 12464867 0xbe32e3 0
/esfs/jtacquaviva/ioperf/file_write.00000043
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494467 0xbea683 0
6 12599780 0xc041e4 0
2 12574528 0xbfdf40 0
5 12386539 0xbd00eb 0
4 12516256 0xbefba0 0
7 12464885 0xbe32f5 0
0 12428183 0xbda397 0
/esfs/jtacquaviva/ioperf/file_write.00000036
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12574546 0xbfdf52 0
1 12646187 0xc0f72b 0
5 12386558 0xbd00fe 0
4 12516275 0xbefbb3 0
3 12494489 0xbea699 0
7 12464905 0xbe3309 0
6 12599800 0xc041f8 0
0 12428202 0xbda3aa 0
/esfs/jtacquaviva/ioperf/file_write.00000017
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428194 0xbda3a2 0
3 12494481 0xbea691 0
2 12574540 0xbfdf4c 0
7 12464898 0xbe3302 0
6 12599793 0xc041f1 0
4 12516268 0xbefbac 0
1 12646182 0xc0f726 0
/esfs/jtacquaviva/ioperf/file_write.00000029
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12599766 0xc041d6 0
4 12516239 0xbefb8f 0
1 12646153 0xc0f709 0
0 12428166 0xbda386 0
3 12494454 0xbea676 0
2 12574512 0xbfdf30 0
7 12464870 0xbe32e6 0
5 12386524 0xbd00dc 0
/esfs/jtacquaviva/ioperf/file_write.00000016
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464866 0xbe32e2 0
5 12386520 0xbd00d8 0
0 12428162 0xbda382 0
1 12646151 0xc0f707 0
3 12494451 0xbea673 0
6 12599764 0xc041d4 0
2 12574509 0xbfdf2d 0
4 12516237 0xbefb8d 0
/esfs/jtacquaviva/ioperf/file_write.00000005
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494480 0xbea690 0
5 12386551 0xbd00f7 0
6 12599792 0xc041f0 0
0 12428193 0xbda3a1 0
2 12574539 0xbfdf4b 0
4 12516266 0xbefbaa 0
7 12464896 0xbe3300 0
1 12646179 0xc0f723 0
/esfs/jtacquaviva/ioperf/file_write.00000014
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494457 0xbea679 0
6 12599770 0xc041da 0
5 12386527 0xbd00df 0
0 12428169 0xbda389 0
7 12464874 0xbe32ea 0
2 12574516 0xbfdf34 0
4 12516244 0xbefb94 0
1 12646158 0xc0f70e 0
/esfs/jtacquaviva/ioperf/file_write.00000021
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12646171 0xc0f71b 0
3 12494472 0xbea688 0
5 12386542 0xbd00ee 0
6 12599783 0xc041e7 0
0 12428184 0xbda398 0
2 12574530 0xbfdf42 0
4 12516258 0xbefba2 0
7 12464887 0xbe32f7 0
/esfs/jtacquaviva/ioperf/file_write.00000045
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428161 0xbda381 0
2 12574507 0xbfdf2b 0
4 12516235 0xbefb8b 0
7 12464865 0xbe32e1 0
1 12646149 0xc0f705 0
3 12494449 0xbea671 0
5 12386519 0xbd00d7 0
6 12599762 0xc041d2 0
/esfs/jtacquaviva/ioperf/file_write.00000019
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12646162 0xc0f712 0
2 12574520 0xbfdf38 0
5 12386532 0xbd00e4 0
4 12516249 0xbefb99 0
6 12599775 0xc041df 0
7 12464880 0xbe32f0 0
0 12428176 0xbda390 0
3 12494464 0xbea680 0
/esfs/jtacquaviva/ioperf/file_write.00000047
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12516264 0xbefba8 0
7 12464893 0xbe32fd 0
1 12646177 0xc0f721 0
3 12494478 0xbea68e 0
5 12386548 0xbd00f4 0
6 12599789 0xc041ed 0
0 12428190 0xbda39e 0
2 12574536 0xbfdf48 0
/esfs/jtacquaviva/ioperf/file_write.00000035
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494466 0xbea682 0
7 12464882 0xbe32f2 0
2 12574524 0xbfdf3c 0
5 12386537 0xbd00e9 0
6 12599779 0xc041e3 0
0 12428178 0xbda392 0
1 12646168 0xc0f718 0
4 12516253 0xbefb9d 0
/esfs/jtacquaviva/ioperf/file_write.00000009
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494490 0xbea69a 0
0 12428203 0xbda3ab 0
4 12516277 0xbefbb5 0
7 12464907 0xbe330b 0
5 12386561 0xbd0101 0
2 12574549 0xbfdf55 0
6 12599802 0xc041fa 0
1 12646190 0xc0f72e 0
/esfs/jtacquaviva/ioperf/file_write.00000015
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12599799 0xc041f7 0
4 12516274 0xbefbb2 0
7 12464904 0xbe3308 0
3 12494488 0xbea698 0
0 12428201 0xbda3a9 0
1 12646188 0xc0f72c 0
2 12574547 0xbfdf53 0
5 12386559 0xbd00ff 0
/esfs/jtacquaviva/ioperf/file_write.00000027
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12386562 0xbd0102 0
6 12599803 0xc041fb 0
0 12428205 0xbda3ad 0
2 12574550 0xbfdf56 0
4 12516278 0xbefbb6 0
7 12464908 0xbe330c 0
1 12646191 0xc0f72f 0
3 12494492 0xbea69c 0
/esfs/jtacquaviva/ioperf/file_write.00000024
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12574553 0xbfdf59 0
4 12516281 0xbefbb9 0
7 12464911 0xbe330f 0
1 12646194 0xc0f732 0
3 12494495 0xbea69f 0
5 12386565 0xbd0105 0
6 12599806 0xc041fe 0
0 12428208 0xbda3b0 0
/esfs/jtacquaviva/ioperf/file_write.00000001
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12386557 0xbd00fd 0
6 12599798 0xc041f6 0
0 12428200 0xbda3a8 0
2 12574545 0xbfdf51 0
4 12516273 0xbefbb1 0
7 12464903 0xbe3307 0
1 12646186 0xc0f72a 0
3 12494487 0xbea697 0
/esfs/jtacquaviva/ioperf/file_write.00000037
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12574525 0xbfdf3d 0
1 12646167 0xc0f717 0
0 12428181 0xbda395 0
4 12516254 0xbefb9e 0
3 12494469 0xbea685 0
5 12386540 0xbd00ec 0
7 12464884 0xbe32f4 0
/esfs/jtacquaviva/ioperf/file_write.00000011
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12516267 0xbefbab 0
1 12646180 0xc0f724 0
6 12599794 0xc041f2 0
7 12464900 0xbe3304 0
5 12386553 0xbd00f9 0
3 12494484 0xbea694 0
0 12428196 0xbda3a4 0

856
output/COUNT:1#NN:8#PPN:6#API:POSIX#T:1048576.txt Normal file
View File

@ -0,0 +1,856 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 48 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 26843545600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:6#API:POSIX#T:1048576.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 04:46:28 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 26843545600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 04:46:28 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 48 (6 per node)
repetitions = 3
xfersize = 1 MiB
blocksize = 25 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288490188800.
WARNING: Using actual aggregate bytes moved = 354606383104.
write 4887 26214400 1024.00 0.003007 69.19 0.000488 69.19 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288490188800.
WARNING: Using actual aggregate bytes moved = 336111599616.
write 4583 26214400 1024.00 0.004174 69.94 0.000649 69.95 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288199733248.
WARNING: Using actual aggregate bytes moved = 347011547136.
write 4804 26214400 1024.00 0.003389 68.88 0.000886 68.89 2
Max Write: 4887.50 MiB/sec (5124.91 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 4887.50 4582.56 4758.03 128.67 69.34262 0 48 6 3 1 0 1 0 0 1 26843545600 1048576 354606383104 POSIX 0
Finished: Wed Oct 24 04:50:01 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 48 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 48 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 26843545600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
+ tee -a ./output/COUNT:1#NN:8#PPN:6#API:POSIX#T:1048576.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 04:50:18 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 26843545600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 04:50:18 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/indread8/file
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 48 (6 per node)
repetitions = 3
xfersize = 1 MiB
blocksize = 25 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 7730941132800.
WARNING: Using actual aggregate bytes moved = 216919965696.
read 3446.39 26214400 1024.00 0.001722 60.02 0.000834 60.03 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 7730941132800.
WARNING: Using actual aggregate bytes moved = 255836815360.
read 4065 26214400 1024.00 0.000895 60.02 0.000866 60.02 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 7730941132800.
WARNING: Using actual aggregate bytes moved = 276677263360.
read 4396 26214400 1024.00 0.000878 60.02 0.000887 60.02 2
Max Read: 4395.92 MiB/sec (4609.45 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 4395.92 3446.39 3969.01 393.51 60.02471 0 48 6 3 1 0 1 0 0 1 26843545600 1048576 216919965696 POSIX 0
Finished: Wed Oct 24 04:53:18 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write.00000006
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12427694 0xbda1ae 0
2 12574036 0xbfdd54 0
4 12515765 0xbef9b5 0
7 12464391 0xbe3107 0
1 12645680 0xc0f530 0
3 12493976 0xbea498 0
5 12386045 0xbcfefd 0
6 12599289 0xc03ff9 0
/esfs/jtacquaviva/ioperf/file_write.00000023
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12645678 0xc0f52e 0
0 12427691 0xbda1ab 0
4 12515763 0xbef9b3 0
3 12493975 0xbea497 0
2 12574035 0xbfdd53 0
7 12464390 0xbe3106 0
6 12599288 0xc03ff8 0
5 12386044 0xbcfefc 0
/esfs/jtacquaviva/ioperf/file_write.00000044
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464353 0xbe30e1 0
1 12645643 0xc0f50b 0
4 12515729 0xbef991 0
3 12493939 0xbea473 0
6 12599253 0xc03fd5 0
5 12386010 0xbcfeda 0
2 12574000 0xbfdd30 0
0 12427658 0xbda18a 0
/esfs/jtacquaviva/ioperf/file_write.00000040
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12427655 0xbda187 0
7 12464352 0xbe30e0 0
1 12645641 0xc0f509 0
3 12493937 0xbea471 0
6 12599251 0xc03fd3 0
2 12573998 0xbfdd2e 0
5 12386008 0xbcfed8 0
4 12515730 0xbef992 0
/esfs/jtacquaviva/ioperf/file_write.00000002
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464394 0xbe310a 0
1 12645683 0xc0f533 0
3 12493979 0xbea49b 0
5 12386048 0xbcff00 0
6 12599292 0xc03ffc 0
0 12427697 0xbda1b1 0
2 12574039 0xbfdd57 0
4 12515768 0xbef9b8 0
/esfs/jtacquaviva/ioperf/file_write.00000026
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12515767 0xbef9b7 0
7 12464393 0xbe3109 0
1 12645682 0xc0f532 0
3 12493978 0xbea49a 0
5 12386047 0xbcfeff 0
6 12599291 0xc03ffb 0
0 12427696 0xbda1b0 0
2 12574038 0xbfdd56 0
/esfs/jtacquaviva/ioperf/file_write.00000033
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12386021 0xbcfee5 0
6 12599265 0xc03fe1 0
0 12427670 0xbda196 0
2 12574012 0xbfdd3c 0
4 12515741 0xbef99d 0
7 12464367 0xbe30ef 0
1 12645656 0xc0f518 0
3 12493952 0xbea480 0
/esfs/jtacquaviva/ioperf/file_write.00000000
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12386041 0xbcfef9 0
4 12515761 0xbef9b1 0
1 12645677 0xc0f52d 0
6 12599286 0xc03ff6 0
2 12574033 0xbfdd51 0
7 12464388 0xbe3104 0
0 12427692 0xbda1ac 0
3 12493974 0xbea496 0
/esfs/jtacquaviva/ioperf/file_write.00000020
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12493981 0xbea49d 0
5 12386050 0xbcff02 0
6 12599294 0xc03ffe 0
0 12427699 0xbda1b3 0
2 12574041 0xbfdd59 0
4 12515770 0xbef9ba 0
7 12464396 0xbe310c 0
1 12645685 0xc0f535 0
/esfs/jtacquaviva/ioperf/file_write.00000030
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12427686 0xbda1a6 0
2 12574028 0xbfdd4c 0
4 12515757 0xbef9ad 0
7 12464383 0xbe30ff 0
1 12645672 0xc0f528 0
3 12493968 0xbea490 0
5 12386037 0xbcfef5 0
6 12599281 0xc03ff1 0
/esfs/jtacquaviva/ioperf/file_write.00000032
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12493960 0xbea488 0
5 12386029 0xbcfeed 0
6 12599273 0xc03fe9 0
0 12427678 0xbda19e 0
2 12574020 0xbfdd44 0
4 12515749 0xbef9a5 0
7 12464375 0xbe30f7 0
1 12645664 0xc0f520 0
/esfs/jtacquaviva/ioperf/file_write.00000004
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12645651 0xc0f513 0
2 12574007 0xbfdd37 0
7 12464361 0xbe30e9 0
0 12427667 0xbda193 0
5 12386017 0xbcfee1 0
3 12493950 0xbea47e 0
4 12515739 0xbef99b 0
6 12599263 0xc03fdf 0
/esfs/jtacquaviva/ioperf/file_write.00000007
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12599275 0xc03feb 0
0 12427680 0xbda1a0 0
2 12574022 0xbfdd46 0
4 12515751 0xbef9a7 0
7 12464377 0xbe30f9 0
5 12386032 0xbcfef0 0
1 12645666 0xc0f522 0
3 12493963 0xbea48b 0
/esfs/jtacquaviva/ioperf/file_write.00000018
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12645663 0xc0f51f 0
3 12493959 0xbea487 0
5 12386028 0xbcfeec 0
6 12599272 0xc03fe8 0
0 12427677 0xbda19d 0
2 12574019 0xbfdd43 0
4 12515748 0xbef9a4 0
7 12464374 0xbe30f6 0
/esfs/jtacquaviva/ioperf/file_write.00000041
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12386033 0xbcfef1 0
2 12574024 0xbfdd48 0
4 12515753 0xbef9a9 0
7 12464378 0xbe30fa 0
3 12493964 0xbea48c 0
6 12599278 0xc03fee 0
0 12427684 0xbda1a4 0
1 12645669 0xc0f525 0
/esfs/jtacquaviva/ioperf/file_write.00000031
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12386004 0xbcfed4 0
6 12599248 0xc03fd0 0
2 12573995 0xbfdd2b 0
7 12464350 0xbe30de 0
0 12427653 0xbda185 0
1 12645639 0xc0f507 0
4 12515726 0xbef98e 0
3 12493935 0xbea46f 0
/esfs/jtacquaviva/ioperf/file_write.00000012
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12574004 0xbfdd34 0
1 12645649 0xc0f511 0
5 12386016 0xbcfee0 0
0 12427666 0xbda192 0
3 12493948 0xbea47c 0
4 12515737 0xbef999 0
6 12599262 0xc03fde 0
7 12464364 0xbe30ec 0
/esfs/jtacquaviva/ioperf/file_write.00000013
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12599266 0xc03fe2 0
0 12427671 0xbda197 0
2 12574013 0xbfdd3d 0
1 12645657 0xc0f519 0
3 12493953 0xbea481 0
5 12386022 0xbcfee6 0
4 12515742 0xbef99e 0
7 12464369 0xbe30f1 0
/esfs/jtacquaviva/ioperf/file_write.00000042
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12645658 0xc0f51a 0
6 12599268 0xc03fe4 0
0 12427674 0xbda19a 0
4 12515745 0xbef9a1 0
2 12574015 0xbfdd3f 0
7 12464372 0xbe30f4 0
3 12493957 0xbea485 0
5 12386025 0xbcfee9 0
/esfs/jtacquaviva/ioperf/file_write.00000025
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12427673 0xbda199 0
7 12464370 0xbe30f2 0
3 12493955 0xbea483 0
6 12599269 0xc03fe5 0
2 12574016 0xbfdd40 0
1 12645661 0xc0f51d 0
5 12386026 0xbcfeea 0
4 12515746 0xbef9a2 0
/esfs/jtacquaviva/ioperf/file_write.00000046
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12493962 0xbea48a 0
5 12386031 0xbcfeef 0
6 12599276 0xc03fec 0
0 12427681 0xbda1a1 0
2 12574023 0xbfdd47 0
4 12515752 0xbef9a8 0
7 12464379 0xbe30fb 0
1 12645668 0xc0f524 0
/esfs/jtacquaviva/ioperf/file_write.00000039
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12493973 0xbea495 0
5 12386043 0xbcfefb 0
6 12599287 0xc03ff7 0
2 12574034 0xbfdd52 0
7 12464389 0xbe3105 0
0 12427693 0xbda1ad 0
1 12645679 0xc0f52f 0
4 12515764 0xbef9b4 0
/esfs/jtacquaviva/ioperf/file_write.00000022
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573997 0xbfdd2d 0
0 12427657 0xbda189 0
1 12645644 0xc0f50c 0
5 12386009 0xbcfed9 0
3 12493940 0xbea474 0
4 12515731 0xbef993 0
6 12599254 0xc03fd6 0
7 12464355 0xbe30e3 0
/esfs/jtacquaviva/ioperf/file_write.00000034
lmm_stripe_count: 6
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464359 0xbe30e7 0
3 12493944 0xbea478 0
6 12599258 0xc03fda 0
2 12574005 0xbfdd35 0
1 12645648 0xc0f510 0
0 12427663 0xbda18f 0
/esfs/jtacquaviva/ioperf/file_write.00000003
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12493941 0xbea475 0
5 12386011 0xbcfedb 0
6 12599255 0xc03fd7 0
0 12427659 0xbda18b 0
2 12574001 0xbfdd31 0
4 12515732 0xbef994 0
7 12464356 0xbe30e4 0
1 12645645 0xc0f50d 0
/esfs/jtacquaviva/ioperf/file_write.00000008
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12515743 0xbef99f 0
1 12645660 0xc0f51c 0
5 12386024 0xbcfee8 0
7 12464371 0xbe30f3 0
3 12493956 0xbea484 0
6 12599270 0xc03fe6 0
0 12427675 0xbda19b 0
2 12574017 0xbfdd41 0
/esfs/jtacquaviva/ioperf/file_write.00000038
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464360 0xbe30e8 0
0 12427662 0xbda18e 0
5 12386015 0xbcfedf 0
6 12599261 0xc03fdd 0
3 12493947 0xbea47b 0
1 12645653 0xc0f515 0
2 12574010 0xbfdd3a 0
/esfs/jtacquaviva/ioperf/file_write.00000028
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12427682 0xbda1a2 0
1 12645667 0xc0f523 0
5 12386035 0xbcfef3 0
2 12574025 0xbfdd49 0
4 12515754 0xbef9aa 0
6 12599279 0xc03fef 0
7 12464380 0xbe30fc 0
3 12493965 0xbea48d 0
/esfs/jtacquaviva/ioperf/file_write.00000010
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12515736 0xbef998 0
3 12493946 0xbea47a 0
2 12574008 0xbfdd38 0
0 12427668 0xbda194 0
5 12386018 0xbcfee2 0
7 12464365 0xbe30ed 0
1 12645654 0xc0f516 0
/esfs/jtacquaviva/ioperf/file_write.00000043
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12515735 0xbef997 0
6 12599259 0xc03fdb 0
5 12386014 0xbcfede 0
1 12645650 0xc0f512 0
2 12574006 0xbfdd36 0
0 12427664 0xbda190 0
7 12464362 0xbe30ea 0
3 12493949 0xbea47d 0
/esfs/jtacquaviva/ioperf/file_write.00000036
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464368 0xbe30f0 0
6 12599267 0xc03fe3 0
0 12427672 0xbda198 0
3 12493954 0xbea482 0
1 12645659 0xc0f51b 0
5 12386023 0xbcfee7 0
2 12574014 0xbfdd3e 0
4 12515744 0xbef9a0 0
/esfs/jtacquaviva/ioperf/file_write.00000017
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12574029 0xbfdd4d 0
4 12515758 0xbef9ae 0
7 12464384 0xbe3100 0
1 12645673 0xc0f529 0
3 12493969 0xbea491 0
5 12386038 0xbcfef6 0
6 12599282 0xc03ff2 0
0 12427687 0xbda1a7 0
/esfs/jtacquaviva/ioperf/file_write.00000029
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12515728 0xbef990 0
5 12386007 0xbcfed7 0
0 12427656 0xbda188 0
1 12645642 0xc0f50a 0
6 12599252 0xc03fd4 0
3 12493938 0xbea472 0
7 12464354 0xbe30e2 0
2 12573999 0xbfdd2f 0
/esfs/jtacquaviva/ioperf/file_write.00000016
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12386030 0xbcfeee 0
6 12599274 0xc03fea 0
0 12427679 0xbda19f 0
2 12574021 0xbfdd45 0
4 12515750 0xbef9a6 0
7 12464376 0xbe30f8 0
1 12645665 0xc0f521 0
3 12493961 0xbea489 0
/esfs/jtacquaviva/ioperf/file_write.00000005
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464373 0xbe30f5 0
1 12645662 0xc0f51e 0
3 12493958 0xbea486 0
5 12386027 0xbcfeeb 0
6 12599271 0xc03fe7 0
0 12427676 0xbda19c 0
2 12574018 0xbfdd42 0
4 12515747 0xbef9a3 0
/esfs/jtacquaviva/ioperf/file_write.00000014
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12574002 0xbfdd32 0
3 12493942 0xbea476 0
0 12427660 0xbda18c 0
4 12515734 0xbef996 0
7 12464358 0xbe30e6 0
5 12386013 0xbcfedd 0
1 12645647 0xc0f50f 0
6 12599257 0xc03fd9 0
/esfs/jtacquaviva/ioperf/file_write.00000021
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12599280 0xc03ff0 0
0 12427685 0xbda1a5 0
2 12574027 0xbfdd4b 0
4 12515756 0xbef9ac 0
7 12464382 0xbe30fe 0
1 12645671 0xc0f527 0
3 12493967 0xbea48f 0
5 12386036 0xbcfef4 0
/esfs/jtacquaviva/ioperf/file_write.00000045
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12574037 0xbfdd55 0
4 12515766 0xbef9b6 0
7 12464392 0xbe3108 0
1 12645681 0xc0f531 0
3 12493977 0xbea499 0
5 12386046 0xbcfefe 0
6 12599290 0xc03ffa 0
0 12427695 0xbda1af 0
/esfs/jtacquaviva/ioperf/file_write.00000019
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12493951 0xbea47f 0
5 12386020 0xbcfee4 0
6 12599264 0xc03fe0 0
0 12427669 0xbda195 0
2 12574011 0xbfdd3b 0
4 12515740 0xbef99c 0
7 12464366 0xbe30ee 0
1 12645655 0xc0f517 0
/esfs/jtacquaviva/ioperf/file_write.00000047
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464386 0xbe3102 0
1 12645675 0xc0f52b 0
3 12493971 0xbea493 0
5 12386040 0xbcfef8 0
6 12599284 0xc03ff4 0
0 12427689 0xbda1a9 0
2 12574031 0xbfdd4f 0
4 12515760 0xbef9b0 0
/esfs/jtacquaviva/ioperf/file_write.00000035
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12386012 0xbcfedc 0
6 12599256 0xc03fd8 0
4 12515733 0xbef995 0
7 12464357 0xbe30e5 0
1 12645646 0xc0f50e 0
2 12574003 0xbfdd33 0
0 12427661 0xbda18d 0
3 12493943 0xbea477 0
/esfs/jtacquaviva/ioperf/file_write.00000009
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12515759 0xbef9af 0
7 12464385 0xbe3101 0
1 12645674 0xc0f52a 0
3 12493970 0xbea492 0
5 12386039 0xbcfef7 0
6 12599283 0xc03ff3 0
0 12427688 0xbda1a8 0
2 12574030 0xbfdd4e 0
/esfs/jtacquaviva/ioperf/file_write.00000015
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12645638 0xc0f506 0
0 12427652 0xbda184 0
4 12515725 0xbef98d 0
3 12493934 0xbea46e 0
2 12573994 0xbfdd2a 0
7 12464349 0xbe30dd 0
5 12386005 0xbcfed5 0
6 12599249 0xc03fd1 0
/esfs/jtacquaviva/ioperf/file_write.00000027
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12599260 0xc03fdc 0
3 12493945 0xbea479 0
1 12645652 0xc0f514 0
0 12427665 0xbda191 0
2 12574009 0xbfdd39 0
7 12464363 0xbe30eb 0
4 12515738 0xbef99a 0
5 12386019 0xbcfee3 0
/esfs/jtacquaviva/ioperf/file_write.00000024
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12645676 0xc0f52c 0
6 12599285 0xc03ff5 0
0 12427690 0xbda1aa 0
2 12574032 0xbfdd50 0
7 12464387 0xbe3103 0
3 12493972 0xbea494 0
5 12386042 0xbcfefa 0
4 12515762 0xbef9b2 0
/esfs/jtacquaviva/ioperf/file_write.00000001
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12386034 0xbcfef2 0
6 12599277 0xc03fed 0
0 12427683 0xbda1a3 0
2 12574026 0xbfdd4a 0
4 12515755 0xbef9ab 0
7 12464381 0xbe30fd 0
1 12645670 0xc0f526 0
3 12493966 0xbea48e 0
/esfs/jtacquaviva/ioperf/file_write.00000037
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12645684 0xc0f534 0
3 12493980 0xbea49c 0
5 12386049 0xbcff01 0
6 12599293 0xc03ffd 0
0 12427698 0xbda1b2 0
2 12574040 0xbfdd58 0
4 12515769 0xbef9b9 0
7 12464395 0xbe310b 0
/esfs/jtacquaviva/ioperf/file_write.00000011
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12599250 0xc03fd2 0
0 12427654 0xbda186 0
2 12573996 0xbfdd2c 0
4 12515727 0xbef98f 0
7 12464351 0xbe30df 0
1 12645640 0xc0f508 0
3 12493936 0xbea470 0
5 12386006 0xbcfed6 0

856
output/COUNT:1#NN:8#PPN:6#API:POSIX#T:10485760.txt Normal file
View File

@ -0,0 +1,856 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 48 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 26843545600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:6#API:POSIX#T:10485760.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 03:39:20 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 26843545600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 03:39:20 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 48 (6 per node)
repetitions = 3
xfersize = 10 MiB
blocksize = 25 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1283750625280.
WARNING: Using actual aggregate bytes moved = 394715463680.
write 5547 26214400 10240 0.002594 67.86 0.000553 67.87 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1287116554240.
WARNING: Using actual aggregate bytes moved = 394400890880.
write 5602 26214400 10240 0.007060 67.13 0.000728 67.14 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1287756185600.
WARNING: Using actual aggregate bytes moved = 391370506240.
write 5491 26214400 10240 0.003122 67.97 0.000932 67.98 2
Max Write: 5601.99 MiB/sec (5874.11 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 5601.99 5490.81 5546.45 45.39 67.66164 0 48 6 3 1 0 1 0 0 1 26843545600 10485760 394715463680 POSIX 0
Finished: Wed Oct 24 03:42:48 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 48 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 48 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 26843545600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
+ tee -a ./output/COUNT:1#NN:8#PPN:6#API:POSIX#T:10485760.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 03:43:12 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 26843545600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 03:43:12 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/indread8/file
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 48 (6 per node)
repetitions = 3
xfersize = 10 MiB
blocksize = 25 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 7730941132800.
WARNING: Using actual aggregate bytes moved = 241182965760.
read 3825 26214400 10240 0.005837 60.13 0.005007 60.13 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 7730941132800.
WARNING: Using actual aggregate bytes moved = 280158535680.
read 4443 26214400 10240 0.005055 60.13 0.004936 60.13 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 7730941132800.
WARNING: Using actual aggregate bytes moved = 293894881280.
read 4660 26214400 10240 0.005025 60.14 0.004984 60.14 2
Max Read: 4660.37 MiB/sec (4886.75 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 4660.37 3825.06 4309.60 353.89 60.13453 0 48 6 3 1 0 1 0 0 1 26843545600 10485760 241182965760 POSIX 0
Finished: Wed Oct 24 03:46:12 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write.00000006
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12463843 0xbe2ee3 0
6 12598742 0xc03dd6 0
1 12645133 0xc0f30d 0
4 12515223 0xbef797 0
5 12385496 0xbcfcd8 0
3 12493433 0xbea279 0
2 12573490 0xbfdb32 0
0 12427152 0xbd9f90 0
/esfs/jtacquaviva/ioperf/file_write.00000023
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12385529 0xbcfcf9 0
6 12598775 0xc03df7 0
0 12427183 0xbd9faf 0
2 12573521 0xbfdb51 0
4 12515255 0xbef7b7 0
7 12463877 0xbe2f05 0
1 12645166 0xc0f32e 0
3 12493465 0xbea299 0
/esfs/jtacquaviva/ioperf/file_write.00000044
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573494 0xbfdb36 0
7 12463852 0xbe2eec 0
3 12493441 0xbea281 0
1 12645141 0xc0f315 0
0 12427158 0xbd9f96 0
4 12515232 0xbef7a0 0
6 12598752 0xc03de0 0
5 12385506 0xbcfce2 0
/esfs/jtacquaviva/ioperf/file_write.00000040
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12598768 0xc03df0 0
0 12427176 0xbd9fa8 0
2 12573514 0xbfdb4a 0
4 12515247 0xbef7af 0
7 12463869 0xbe2efd 0
1 12645158 0xc0f326 0
3 12493457 0xbea291 0
5 12385522 0xbcfcf2 0
/esfs/jtacquaviva/ioperf/file_write.00000002
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12493455 0xbea28f 0
5 12385519 0xbcfcef 0
6 12598765 0xc03ded 0
0 12427174 0xbd9fa6 0
2 12573512 0xbfdb48 0
4 12515246 0xbef7ae 0
7 12463868 0xbe2efc 0
1 12645157 0xc0f325 0
/esfs/jtacquaviva/ioperf/file_write.00000026
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573510 0xbfdb46 0
4 12515244 0xbef7ac 0
3 12493454 0xbea28e 0
5 12385518 0xbcfcee 0
6 12598764 0xc03dec 0
7 12463866 0xbe2efa 0
0 12427173 0xbd9fa5 0
1 12645156 0xc0f324 0
/esfs/jtacquaviva/ioperf/file_write.00000033
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12427171 0xbd9fa3 0
2 12573509 0xbfdb45 0
4 12515243 0xbef7ab 0
7 12463865 0xbe2ef9 0
1 12645154 0xc0f322 0
3 12493453 0xbea28d 0
5 12385517 0xbcfced 0
6 12598763 0xc03deb 0
/esfs/jtacquaviva/ioperf/file_write.00000000
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573484 0xbfdb2c 0
7 12463840 0xbe2ee0 0
0 12427147 0xbd9f8b 0
1 12645130 0xc0f30a 0
4 12515221 0xbef795 0
5 12385494 0xbcfcd6 0
3 12493430 0xbea276 0
6 12598741 0xc03dd5 0
/esfs/jtacquaviva/ioperf/file_write.00000020
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12493437 0xbea27d 0
2 12573496 0xbfdb38 0
5 12385504 0xbcfce0 0
6 12598751 0xc03ddf 0
1 12645143 0xc0f317 0
7 12463854 0xbe2eee 0
0 12427160 0xbd9f98 0
/esfs/jtacquaviva/ioperf/file_write.00000030
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573501 0xbfdb3d 0
4 12515235 0xbef7a3 0
7 12463857 0xbe2ef1 0
5 12385509 0xbcfce5 0
6 12598755 0xc03de3 0
1 12645146 0xc0f31a 0
3 12493446 0xbea286 0
0 12427164 0xbd9f9c 0
/esfs/jtacquaviva/ioperf/file_write.00000032
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12515228 0xbef79c 0
5 12385501 0xbcfcdd 0
1 12645138 0xc0f312 0
6 12598749 0xc03ddd 0
0 12427157 0xbd9f95 0
3 12493438 0xbea27e 0
7 12463853 0xbe2eed 0
2 12573497 0xbfdb39 0
/esfs/jtacquaviva/ioperf/file_write.00000004
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573503 0xbfdb3f 0
1 12645149 0xc0f31d 0
5 12385511 0xbcfce7 0
0 12427165 0xbd9f9d 0
4 12515237 0xbef7a5 0
3 12493449 0xbea289 0
6 12598759 0xc03de7 0
7 12463860 0xbe2ef4 0
/esfs/jtacquaviva/ioperf/file_write.00000007
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12427161 0xbd9f99 0
2 12573499 0xbfdb3b 0
4 12515233 0xbef7a1 0
7 12463855 0xbe2eef 0
1 12645144 0xc0f318 0
3 12493443 0xbea283 0
5 12385508 0xbcfce4 0
6 12598754 0xc03de2 0
/esfs/jtacquaviva/ioperf/file_write.00000018
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12645148 0xc0f31c 0
3 12493448 0xbea288 0
5 12385512 0xbcfce8 0
6 12598758 0xc03de6 0
0 12427166 0xbd9f9e 0
2 12573505 0xbfdb41 0
4 12515239 0xbef7a7 0
7 12463861 0xbe2ef5 0
/esfs/jtacquaviva/ioperf/file_write.00000041
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12463851 0xbe2eeb 0
4 12515231 0xbef79f 0
1 12645142 0xc0f316 0
6 12598750 0xc03dde 0
5 12385505 0xbcfce1 0
2 12573498 0xbfdb3a 0
0 12427159 0xbd9f97 0
3 12493442 0xbea282 0
/esfs/jtacquaviva/ioperf/file_write.00000031
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573487 0xbfdb2f 0
1 12645132 0xc0f30c 0
3 12493431 0xbea277 0
5 12385495 0xbcfcd7 0
4 12515222 0xbef796 0
6 12598743 0xc03dd7 0
0 12427149 0xbd9f8d 0
7 12463845 0xbe2ee5 0
/esfs/jtacquaviva/ioperf/file_write.00000012
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12427177 0xbd9fa9 0
2 12573515 0xbfdb4b 0
4 12515249 0xbef7b1 0
7 12463871 0xbe2eff 0
1 12645160 0xc0f328 0
3 12493459 0xbea293 0
5 12385523 0xbcfcf3 0
6 12598769 0xc03df1 0
/esfs/jtacquaviva/ioperf/file_write.00000013
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12385515 0xbcfceb 0
6 12598761 0xc03de9 0
0 12427169 0xbd9fa1 0
2 12573507 0xbfdb43 0
4 12515241 0xbef7a9 0
7 12463863 0xbe2ef7 0
1 12645152 0xc0f320 0
3 12493451 0xbea28b 0
/esfs/jtacquaviva/ioperf/file_write.00000042
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12463874 0xbe2f02 0
1 12645163 0xc0f32b 0
3 12493462 0xbea296 0
5 12385526 0xbcfcf6 0
6 12598772 0xc03df4 0
0 12427180 0xbd9fac 0
2 12573518 0xbfdb4e 0
4 12515252 0xbef7b4 0
/esfs/jtacquaviva/ioperf/file_write.00000025
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12427145 0xbd9f89 0
5 12385493 0xbcfcd5 0
4 12515219 0xbef793 0
1 12645129 0xc0f309 0
2 12573486 0xbfdb2e 0
3 12493429 0xbea275 0
7 12463842 0xbe2ee2 0
6 12598740 0xc03dd4 0
/esfs/jtacquaviva/ioperf/file_write.00000046
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12645155 0xc0f323 0
0 12427172 0xbd9fa4 0
4 12515245 0xbef7ad 0
3 12493456 0xbea290 0
5 12385520 0xbcfcf0 0
2 12573511 0xbfdb47 0
6 12598767 0xc03def 0
7 12463867 0xbe2efb 0
/esfs/jtacquaviva/ioperf/file_write.00000039
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573483 0xbfdb2b 0
4 12515218 0xbef792 0
7 12463839 0xbe2edf 0
5 12385491 0xbcfcd3 0
1 12645128 0xc0f308 0
3 12493428 0xbea274 0
6 12598738 0xc03dd2 0
0 12427146 0xbd9f8a 0
/esfs/jtacquaviva/ioperf/file_write.00000022
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12493427 0xbea273 0
5 12385492 0xbcfcd4 0
6 12598739 0xc03dd3 0
2 12573485 0xbfdb2d 0
4 12515220 0xbef794 0
7 12463841 0xbe2ee1 0
0 12427148 0xbd9f8c 0
1 12645131 0xc0f30b 0
/esfs/jtacquaviva/ioperf/file_write.00000034
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12515251 0xbef7b3 0
7 12463873 0xbe2f01 0
1 12645162 0xc0f32a 0
3 12493461 0xbea295 0
5 12385525 0xbcfcf5 0
6 12598771 0xc03df3 0
0 12427179 0xbd9fab 0
2 12573517 0xbfdb4d 0
/esfs/jtacquaviva/ioperf/file_write.00000003
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12493464 0xbea298 0
5 12385528 0xbcfcf8 0
6 12598774 0xc03df6 0
0 12427182 0xbd9fae 0
2 12573520 0xbfdb50 0
4 12515254 0xbef7b6 0
7 12463876 0xbe2f04 0
1 12645165 0xc0f32d 0
/esfs/jtacquaviva/ioperf/file_write.00000008
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573479 0xbfdb27 0
3 12493424 0xbea270 0
4 12515215 0xbef78f 0
5 12385489 0xbcfcd1 0
7 12463836 0xbe2edc 0
1 12645126 0xc0f306 0
0 12427143 0xbd9f87 0
/esfs/jtacquaviva/ioperf/file_write.00000038
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12598776 0xc03df8 0
0 12427184 0xbd9fb0 0
2 12573522 0xbfdb52 0
4 12515256 0xbef7b8 0
7 12463878 0xbe2f06 0
1 12645167 0xc0f32f 0
3 12493466 0xbea29a 0
5 12385530 0xbcfcfa 0
/esfs/jtacquaviva/ioperf/file_write.00000028
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12645164 0xc0f32c 0
3 12493463 0xbea297 0
5 12385527 0xbcfcf7 0
6 12598773 0xc03df5 0
0 12427181 0xbd9fad 0
2 12573519 0xbfdb4f 0
4 12515253 0xbef7b5 0
7 12463875 0xbe2f03 0
/esfs/jtacquaviva/ioperf/file_write.00000010
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573516 0xbfdb4c 0
4 12515250 0xbef7b2 0
7 12463872 0xbe2f00 0
1 12645161 0xc0f329 0
3 12493460 0xbea294 0
5 12385524 0xbcfcf4 0
6 12598770 0xc03df2 0
0 12427178 0xbd9faa 0
/esfs/jtacquaviva/ioperf/file_write.00000043
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12493423 0xbea26f 0
7 12463835 0xbe2edb 0
5 12385488 0xbcfcd0 0
1 12645124 0xc0f304 0
6 12598736 0xc03dd0 0
2 12573480 0xbfdb28 0
0 12427142 0xbd9f86 0
4 12515216 0xbef790 0
/esfs/jtacquaviva/ioperf/file_write.00000036
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12515259 0xbef7bb 0
7 12463881 0xbe2f09 0
1 12645170 0xc0f332 0
3 12493469 0xbea29d 0
0 12427187 0xbd9fb3 0
2 12573525 0xbfdb55 0
6 12598780 0xc03dfc 0
5 12385534 0xbcfcfe 0
/esfs/jtacquaviva/ioperf/file_write.00000017
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12598753 0xc03de1 0
0 12427162 0xbd9f9a 0
2 12573500 0xbfdb3c 0
4 12515234 0xbef7a2 0
7 12463856 0xbe2ef0 0
1 12645145 0xc0f319 0
3 12493444 0xbea284 0
5 12385507 0xbcfce3 0
/esfs/jtacquaviva/ioperf/file_write.00000029
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573493 0xbfdb35 0
6 12598748 0xc03ddc 0
4 12515230 0xbef79e 0
0 12427156 0xbd9f94 0
7 12463850 0xbe2eea 0
5 12385503 0xbcfcdf 0
1 12645139 0xc0f313 0
3 12493439 0xbea27f 0
/esfs/jtacquaviva/ioperf/file_write.00000016
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12598735 0xc03dcf 0
4 12515214 0xbef78e 0
1 12645125 0xc0f305 0
0 12427141 0xbd9f85 0
3 12493425 0xbea271 0
2 12573481 0xbfdb29 0
7 12463837 0xbe2edd 0
/esfs/jtacquaviva/ioperf/file_write.00000005
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12598747 0xc03ddb 0
4 12515229 0xbef79d 0
5 12385502 0xbcfcde 0
0 12427155 0xbd9f93 0
7 12463849 0xbe2ee9 0
2 12573495 0xbfdb37 0
1 12645140 0xc0f314 0
3 12493440 0xbea280 0
/esfs/jtacquaviva/ioperf/file_write.00000014
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573489 0xbfdb31 0
7 12463844 0xbe2ee4 0
0 12427150 0xbd9f8e 0
5 12385497 0xbcfcd9 0
4 12515225 0xbef799 0
6 12598744 0xc03dd8 0
3 12493434 0xbea27a 0
1 12645135 0xc0f30f 0
/esfs/jtacquaviva/ioperf/file_write.00000021
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12385521 0xbcfcf1 0
6 12598766 0xc03dee 0
0 12427175 0xbd9fa7 0
2 12573513 0xbfdb49 0
4 12515248 0xbef7b0 0
7 12463870 0xbe2efe 0
1 12645159 0xc0f327 0
3 12493458 0xbea292 0
/esfs/jtacquaviva/ioperf/file_write.00000045
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573488 0xbfdb30 0
3 12493432 0xbea278 0
4 12515224 0xbef798 0
5 12385498 0xbcfcda 0
0 12427151 0xbd9f8f 0
7 12463846 0xbe2ee6 0
1 12645134 0xc0f30e 0
/esfs/jtacquaviva/ioperf/file_write.00000019
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12598779 0xc03dfb 0
5 12385533 0xbcfcfd 0
0 12427188 0xbd9fb4 0
2 12573526 0xbfdb56 0
4 12515260 0xbef7bc 0
7 12463882 0xbe2f0a 0
1 12645171 0xc0f333 0
3 12493470 0xbea29e 0
/esfs/jtacquaviva/ioperf/file_write.00000047
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12463859 0xbe2ef3 0
3 12493447 0xbea287 0
6 12598756 0xc03de4 0
2 12573504 0xbfdb40 0
1 12645150 0xc0f31e 0
5 12385513 0xbcfce9 0
0 12427167 0xbd9f9f 0
4 12515238 0xbef7a6 0
/esfs/jtacquaviva/ioperf/file_write.00000035
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12427144 0xbd9f88 0
2 12573482 0xbfdb2a 0
4 12515217 0xbef791 0
7 12463838 0xbe2ede 0
1 12645127 0xc0f307 0
3 12493426 0xbea272 0
5 12385490 0xbcfcd2 0
6 12598737 0xc03dd1 0
/esfs/jtacquaviva/ioperf/file_write.00000009
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12598762 0xc03dea 0
0 12427170 0xbd9fa2 0
2 12573508 0xbfdb44 0
4 12515242 0xbef7aa 0
7 12463864 0xbe2ef8 0
1 12645153 0xc0f321 0
3 12493452 0xbea28c 0
5 12385516 0xbcfcec 0
/esfs/jtacquaviva/ioperf/file_write.00000015
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12385499 0xbcfcdb 0
2 12573491 0xbfdb33 0
7 12463847 0xbe2ee7 0
3 12493435 0xbea27b 0
0 12427153 0xbd9f91 0
1 12645136 0xc0f310 0
4 12515226 0xbef79a 0
6 12598745 0xc03dd9 0
/esfs/jtacquaviva/ioperf/file_write.00000027
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12493445 0xbea285 0
0 12427163 0xbd9f9b 0
2 12573502 0xbfdb3e 0
4 12515236 0xbef7a4 0
7 12463858 0xbe2ef2 0
1 12645147 0xc0f31b 0
5 12385510 0xbcfce6 0
6 12598757 0xc03de5 0
/esfs/jtacquaviva/ioperf/file_write.00000024
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12427185 0xbd9fb1 0
2 12573523 0xbfdb53 0
4 12515257 0xbef7b9 0
7 12463879 0xbe2f07 0
1 12645168 0xc0f330 0
3 12493467 0xbea29b 0
5 12385531 0xbcfcfb 0
6 12598777 0xc03df9 0
/esfs/jtacquaviva/ioperf/file_write.00000001
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573524 0xbfdb54 0
4 12515258 0xbef7ba 0
7 12463880 0xbe2f08 0
1 12645169 0xc0f331 0
3 12493468 0xbea29c 0
5 12385532 0xbcfcfc 0
6 12598778 0xc03dfa 0
0 12427186 0xbd9fb2 0
/esfs/jtacquaviva/ioperf/file_write.00000037
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12493450 0xbea28a 0
5 12385514 0xbcfcea 0
6 12598760 0xc03de8 0
0 12427168 0xbd9fa0 0
2 12573506 0xbfdb42 0
4 12515240 0xbef7a8 0
7 12463862 0xbe2ef6 0
1 12645151 0xc0f31f 0
/esfs/jtacquaviva/ioperf/file_write.00000011
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12427154 0xbd9f92 0
4 12515227 0xbef79b 0
1 12645137 0xc0f311 0
5 12385500 0xbcfcdc 0
6 12598746 0xc03dda 0
3 12493436 0xbea27c 0
7 12463848 0xbe2ee8 0
2 12573492 0xbfdb34 0

855
output/COUNT:1#NN:8#PPN:6#API:POSIX#T:16384.txt Normal file
View File

@ -0,0 +1,855 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 48 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 26843545600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:6#API:POSIX#T:16384.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 07:10:23 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 26843545600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 07:10:23 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 48 (6 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 25 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288447180800.
WARNING: Using actual aggregate bytes moved = 31813091328.
write 274.83 26214400 16.00 0.002551 110.39 0.000442 110.39 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288427962368.
WARNING: Using actual aggregate bytes moved = 26790608896.
write 215.96 26214400 16.00 0.002872 118.30 0.000476 118.30 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288349925376.
WARNING: Using actual aggregate bytes moved = 27780775936.
write 214.21 26214400 16.00 0.003518 123.68 0.000549 123.68 2
Max Write: 274.83 MiB/sec (288.18 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 274.83 214.21 235.00 28.17 117.45843 0 48 6 3 1 0 1 0 0 1 26843545600 16384 31813091328 POSIX 0
Finished: Wed Oct 24 07:16:16 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 48 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ tee -a ./output/COUNT:1#NN:8#PPN:6#API:POSIX#T:16384.txt
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 48 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 26843545600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 07:16:22 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 26843545600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 07:16:22 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/indread8/file
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 48 (6 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 25 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 7730941132800.
WARNING: Using actual aggregate bytes moved = 6453608448.
read 102.39 26214400 16.00 0.001573 60.11 0.000848 60.11 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 7730941132800.
WARNING: Using actual aggregate bytes moved = 7719092224.
read 122.48 26214400 16.00 0.001499 60.10 0.000849 60.10 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 7730941132800.
WARNING: Using actual aggregate bytes moved = 7727251456.
read 122.60 26214400 16.00 0.000921 60.11 0.000827 60.11 2
Max Read: 122.60 MiB/sec (128.55 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 122.60 102.39 115.82 9.50 60.10760 0 48 6 3 1 0 1 0 0 1 26843545600 16384 6453608448 POSIX 0
Finished: Wed Oct 24 07:19:22 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write.00000006
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428710 0xbda5a6 0
6 12600305 0xc043f1 0
2 12575059 0xbfe153 0
4 12516784 0xbefdb0 0
7 12465419 0xbe350b 0
1 12646701 0xc0f92d 0
3 12495005 0xbea89d 0
5 12387064 0xbd02f8 0
/esfs/jtacquaviva/ioperf/file_write.00000023
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12600281 0xc043d9 0
1 12646675 0xc0f913 0
3 12494978 0xbea882 0
0 12428685 0xbda58d 0
5 12387038 0xbd02de 0
4 12516760 0xbefd98 0
7 12465393 0xbe34f1 0
2 12575034 0xbfe13a 0
/esfs/jtacquaviva/ioperf/file_write.00000044
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12387058 0xbd02f2 0
6 12600301 0xc043ed 0
0 12428706 0xbda5a2 0
2 12575055 0xbfe14f 0
4 12516780 0xbefdac 0
7 12465413 0xbe3505 0
1 12646696 0xc0f928 0
3 12494999 0xbea897 0
/esfs/jtacquaviva/ioperf/file_write.00000040
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428699 0xbda59b 0
7 12465406 0xbe34fe 0
3 12494992 0xbea890 0
2 12575049 0xbfe149 0
1 12646690 0xc0f922 0
4 12516774 0xbefda6 0
5 12387052 0xbd02ec 0
6 12600295 0xc043e7 0
/esfs/jtacquaviva/ioperf/file_write.00000002
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12575035 0xbfe13b 0
4 12516762 0xbefd9a 0
7 12465395 0xbe34f3 0
3 12494981 0xbea885 0
1 12646677 0xc0f915 0
0 12428688 0xbda590 0
5 12387041 0xbd02e1 0
/esfs/jtacquaviva/ioperf/file_write.00000026
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494979 0xbea883 0
7 12465394 0xbe34f2 0
6 12600283 0xc043db 0
0 12428687 0xbda58f 0
2 12575037 0xbfe13d 0
5 12387040 0xbd02e0 0
1 12646678 0xc0f916 0
/esfs/jtacquaviva/ioperf/file_write.00000033
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12575062 0xbfe156 0
1 12646703 0xc0f92f 0
5 12387065 0xbd02f9 0
0 12428713 0xbda5a9 0
4 12516786 0xbefdb2 0
3 12495007 0xbea89f 0
6 12600309 0xc043f5 0
7 12465421 0xbe350d 0
/esfs/jtacquaviva/ioperf/file_write.00000000
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12387031 0xbd02d7 0
7 12465385 0xbe34e9 0
1 12646668 0xc0f90c 0
3 12494971 0xbea87b 0
0 12428679 0xbda587 0
4 12516753 0xbefd91 0
6 12600274 0xc043d2 0
2 12575027 0xbfe133 0
/esfs/jtacquaviva/ioperf/file_write.00000020
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12465417 0xbe3509 0
5 12387062 0xbd02f6 0
1 12646700 0xc0f92c 0
3 12495003 0xbea89b 0
2 12575060 0xbfe154 0
0 12428711 0xbda5a7 0
6 12600306 0xc043f2 0
/esfs/jtacquaviva/ioperf/file_write.00000030
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12516794 0xbefdba 0
7 12465428 0xbe3514 0
1 12646711 0xc0f937 0
3 12495014 0xbea8a6 0
5 12387073 0xbd0301 0
6 12600316 0xc043fc 0
0 12428721 0xbda5b1 0
2 12575070 0xbfe15e 0
/esfs/jtacquaviva/ioperf/file_write.00000032
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12575023 0xbfe12f 0
1 12646664 0xc0f908 0
5 12387027 0xbd02d3 0
6 12600270 0xc043ce 0
4 12516749 0xbefd8d 0
3 12494968 0xbea878 0
0 12428676 0xbda584 0
7 12465382 0xbe34e6 0
/esfs/jtacquaviva/ioperf/file_write.00000004
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12600287 0xc043df 0
0 12428693 0xbda595 0
7 12465400 0xbe34f8 0
1 12646683 0xc0f91b 0
2 12575042 0xbfe142 0
4 12516768 0xbefda0 0
5 12387048 0xbd02e8 0
3 12494989 0xbea88d 0
/esfs/jtacquaviva/ioperf/file_write.00000007
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12516757 0xbefd95 0
7 12465389 0xbe34ed 0
1 12646672 0xc0f910 0
6 12600278 0xc043d6 0
0 12428682 0xbda58a 0
3 12494975 0xbea87f 0
5 12387036 0xbd02dc 0
2 12575031 0xbfe137 0
/esfs/jtacquaviva/ioperf/file_write.00000018
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12575050 0xbfe14a 0
4 12516775 0xbefda7 0
7 12465408 0xbe3500 0
1 12646691 0xc0f923 0
3 12494994 0xbea892 0
5 12387054 0xbd02ee 0
6 12600297 0xc043e9 0
0 12428701 0xbda59d 0
/esfs/jtacquaviva/ioperf/file_write.00000041
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12516754 0xbefd92 0
6 12600275 0xc043d3 0
2 12575028 0xbfe134 0
1 12646669 0xc0f90d 0
7 12465387 0xbe34eb 0
3 12494972 0xbea87c 0
5 12387033 0xbd02d9 0
/esfs/jtacquaviva/ioperf/file_write.00000031
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12646687 0xc0f91f 0
3 12494990 0xbea88e 0
5 12387049 0xbd02e9 0
6 12600292 0xc043e4 0
0 12428697 0xbda599 0
2 12575046 0xbfe146 0
4 12516771 0xbefda3 0
7 12465404 0xbe34fc 0
/esfs/jtacquaviva/ioperf/file_write.00000012
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494995 0xbea893 0
4 12516777 0xbefda9 0
6 12600298 0xc043ea 0
7 12465410 0xbe3502 0
0 12428703 0xbda59f 0
5 12387056 0xbd02f0 0
2 12575053 0xbfe14d 0
1 12646694 0xc0f926 0
/esfs/jtacquaviva/ioperf/file_write.00000013
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12575058 0xbfe152 0
1 12646699 0xc0f92b 0
3 12495002 0xbea89a 0
4 12516783 0xbefdaf 0
7 12465416 0xbe3508 0
5 12387061 0xbd02f5 0
0 12428709 0xbda5a5 0
6 12600304 0xc043f0 0
/esfs/jtacquaviva/ioperf/file_write.00000042
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12600284 0xc043dc 0
0 12428689 0xbda591 0
2 12575038 0xbfe13e 0
4 12516763 0xbefd9b 0
7 12465396 0xbe34f4 0
1 12646679 0xc0f917 0
3 12494982 0xbea886 0
5 12387042 0xbd02e2 0
/esfs/jtacquaviva/ioperf/file_write.00000025
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428708 0xbda5a4 0
2 12575057 0xbfe151 0
4 12516782 0xbefdae 0
7 12465415 0xbe3507 0
1 12646698 0xc0f92a 0
3 12495001 0xbea899 0
5 12387060 0xbd02f4 0
6 12600303 0xc043ef 0
/esfs/jtacquaviva/ioperf/file_write.00000046
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12600285 0xc043dd 0
0 12428690 0xbda592 0
2 12575039 0xbfe13f 0
4 12516764 0xbefd9c 0
7 12465397 0xbe34f5 0
1 12646680 0xc0f918 0
3 12494983 0xbea887 0
/esfs/jtacquaviva/ioperf/file_write.00000039
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12495008 0xbea8a0 0
0 12428715 0xbda5ab 0
7 12465422 0xbe350e 0
5 12387067 0xbd02fb 0
2 12575065 0xbfe159 0
1 12646706 0xc0f932 0
4 12516789 0xbefdb5 0
6 12600311 0xc043f7 0
/esfs/jtacquaviva/ioperf/file_write.00000022
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12575067 0xbfe15b 0
4 12516791 0xbefdb7 0
7 12465425 0xbe3511 0
1 12646708 0xc0f934 0
3 12495011 0xbea8a3 0
5 12387070 0xbd02fe 0
6 12600313 0xc043f9 0
0 12428719 0xbda5af 0
/esfs/jtacquaviva/ioperf/file_write.00000034
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494998 0xbea896 0
5 12387057 0xbd02f1 0
6 12600300 0xc043ec 0
0 12428705 0xbda5a1 0
2 12575054 0xbfe14e 0
4 12516779 0xbefdab 0
7 12465412 0xbe3504 0
1 12646695 0xc0f927 0
/esfs/jtacquaviva/ioperf/file_write.00000003
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12465418 0xbe350a 0
3 12495004 0xbea89c 0
2 12575061 0xbfe155 0
4 12516785 0xbefdb1 0
0 12428712 0xbda5a8 0
5 12387063 0xbd02f7 0
1 12646702 0xc0f92e 0
6 12600307 0xc043f3 0
/esfs/jtacquaviva/ioperf/file_write.00000008
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12646666 0xc0f90a 0
3 12494969 0xbea879 0
5 12387029 0xbd02d5 0
6 12600272 0xc043d0 0
0 12428677 0xbda585 0
2 12575025 0xbfe131 0
4 12516751 0xbefd8f 0
7 12465383 0xbe34e7 0
/esfs/jtacquaviva/ioperf/file_write.00000038
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12600302 0xc043ee 0
0 12428707 0xbda5a3 0
2 12575056 0xbfe150 0
4 12516781 0xbefdad 0
7 12465414 0xbe3506 0
1 12646697 0xc0f929 0
3 12495000 0xbea898 0
5 12387059 0xbd02f3 0
/esfs/jtacquaviva/ioperf/file_write.00000028
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494984 0xbea888 0
0 12428691 0xbda593 0
5 12387043 0xbd02e3 0
2 12575041 0xbfe141 0
7 12465398 0xbe34f6 0
6 12600288 0xc043e0 0
1 12646682 0xc0f91a 0
4 12516767 0xbefd9f 0
/esfs/jtacquaviva/ioperf/file_write.00000010
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428718 0xbda5ae 0
2 12575068 0xbfe15c 0
4 12516792 0xbefdb8 0
7 12465426 0xbe3512 0
1 12646709 0xc0f935 0
3 12495012 0xbea8a4 0
5 12387071 0xbd02ff 0
6 12600314 0xc043fa 0
/esfs/jtacquaviva/ioperf/file_write.00000043
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494970 0xbea87a 0
5 12387030 0xbd02d6 0
6 12600273 0xc043d1 0
0 12428678 0xbda586 0
2 12575026 0xbfe132 0
4 12516752 0xbefd90 0
7 12465384 0xbe34e8 0
1 12646667 0xc0f90b 0
/esfs/jtacquaviva/ioperf/file_write.00000036
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12600286 0xc043de 0
2 12575040 0xbfe140 0
0 12428692 0xbda594 0
5 12387044 0xbd02e4 0
4 12516765 0xbefd9d 0
1 12646681 0xc0f919 0
7 12465399 0xbe34f7 0
3 12494987 0xbea88b 0
/esfs/jtacquaviva/ioperf/file_write.00000017
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12387035 0xbd02db 0
6 12600279 0xc043d7 0
0 12428683 0xbda58b 0
1 12646673 0xc0f911 0
2 12575032 0xbfe138 0
3 12494976 0xbea880 0
4 12516759 0xbefd97 0
7 12465391 0xbe34ef 0
/esfs/jtacquaviva/ioperf/file_write.00000029
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12387051 0xbd02eb 0
2 12575048 0xbfe148 0
4 12516773 0xbefda5 0
1 12646689 0xc0f921 0
6 12600294 0xc043e6 0
7 12465407 0xbe34ff 0
0 12428700 0xbda59c 0
3 12494993 0xbea891 0
/esfs/jtacquaviva/ioperf/file_write.00000016
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12465381 0xbe34e5 0
3 12494967 0xbea877 0
0 12428675 0xbda583 0
1 12646665 0xc0f909 0
6 12600271 0xc043cf 0
2 12575024 0xbfe130 0
5 12387028 0xbd02d4 0
4 12516750 0xbefd8e 0
/esfs/jtacquaviva/ioperf/file_write.00000005
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12516776 0xbefda8 0
7 12465409 0xbe3501 0
5 12387053 0xbd02ed 0
6 12600296 0xc043e8 0
1 12646692 0xc0f924 0
0 12428702 0xbda59e 0
3 12494996 0xbea894 0
2 12575052 0xbfe14c 0
/esfs/jtacquaviva/ioperf/file_write.00000014
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428717 0xbda5ad 0
2 12575066 0xbfe15a 0
4 12516790 0xbefdb6 0
7 12465424 0xbe3510 0
1 12646707 0xc0f933 0
3 12495010 0xbea8a2 0
5 12387069 0xbd02fd 0
6 12600312 0xc043f8 0
/esfs/jtacquaviva/ioperf/file_write.00000021
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494985 0xbea889 0
5 12387046 0xbd02e6 0
7 12465401 0xbe34f9 0
4 12516766 0xbefd9e 0
6 12600290 0xc043e2 0
0 12428696 0xbda598 0
1 12646685 0xc0f91d 0
2 12575044 0xbfe144 0
/esfs/jtacquaviva/ioperf/file_write.00000045
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12465420 0xbe350c 0
3 12495006 0xbea89e 0
6 12600308 0xc043f4 0
2 12575063 0xbfe157 0
1 12646704 0xc0f930 0
5 12387066 0xbd02fa 0
0 12428714 0xbda5aa 0
4 12516787 0xbefdb3 0
/esfs/jtacquaviva/ioperf/file_write.00000019
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428680 0xbda588 0
5 12387032 0xbd02d8 0
7 12465386 0xbe34ea 0
4 12516755 0xbefd93 0
3 12494973 0xbea87d 0
2 12575029 0xbfe135 0
1 12646670 0xc0f90e 0
6 12600276 0xc043d4 0
/esfs/jtacquaviva/ioperf/file_write.00000047
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494991 0xbea88f 0
5 12387050 0xbd02ea 0
6 12600293 0xc043e5 0
0 12428698 0xbda59a 0
2 12575047 0xbfe147 0
4 12516772 0xbefda4 0
7 12465405 0xbe34fd 0
1 12646688 0xc0f920 0
/esfs/jtacquaviva/ioperf/file_write.00000035
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12516758 0xbefd96 0
7 12465390 0xbe34ee 0
3 12494977 0xbea881 0
1 12646674 0xc0f912 0
5 12387037 0xbd02dd 0
0 12428684 0xbda58c 0
6 12600280 0xc043d8 0
2 12575033 0xbfe139 0
/esfs/jtacquaviva/ioperf/file_write.00000009
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12575051 0xbfe14b 0
4 12516778 0xbefdaa 0
5 12387055 0xbd02ef 0
1 12646693 0xc0f925 0
6 12600299 0xc043eb 0
3 12494997 0xbea895 0
0 12428704 0xbda5a0 0
7 12465411 0xbe3503 0
/esfs/jtacquaviva/ioperf/file_write.00000015
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494986 0xbea88a 0
5 12387047 0xbd02e7 0
7 12465402 0xbe34fa 0
0 12428695 0xbda597 0
1 12646684 0xc0f91c 0
6 12600291 0xc043e3 0
2 12575045 0xbfe145 0
4 12516769 0xbefda1 0
/esfs/jtacquaviva/ioperf/file_write.00000027
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12600310 0xc043f6 0
2 12575064 0xbfe158 0
4 12516788 0xbefdb4 0
1 12646705 0xc0f931 0
0 12428716 0xbda5ac 0
7 12465423 0xbe350f 0
3 12495009 0xbea8a1 0
5 12387068 0xbd02fc 0
/esfs/jtacquaviva/ioperf/file_write.00000024
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12575030 0xbfe136 0
4 12516756 0xbefd94 0
7 12465388 0xbe34ec 0
1 12646671 0xc0f90f 0
3 12494974 0xbea87e 0
5 12387034 0xbd02da 0
6 12600277 0xc043d5 0
0 12428681 0xbda589 0
/esfs/jtacquaviva/ioperf/file_write.00000001
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12575069 0xbfe15d 0
4 12516793 0xbefdb9 0
7 12465427 0xbe3513 0
1 12646710 0xc0f936 0
3 12495013 0xbea8a5 0
5 12387072 0xbd0300 0
6 12600315 0xc043fb 0
0 12428720 0xbda5b0 0
/esfs/jtacquaviva/ioperf/file_write.00000037
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12465392 0xbe34f0 0
4 12516761 0xbefd99 0
5 12387039 0xbd02df 0
6 12600282 0xc043da 0
1 12646676 0xc0f914 0
0 12428686 0xbda58e 0
3 12494980 0xbea884 0
2 12575036 0xbfe13c 0
/esfs/jtacquaviva/ioperf/file_write.00000011
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12387045 0xbd02e5 0
6 12600289 0xc043e1 0
0 12428694 0xbda596 0
3 12494988 0xbea88c 0
2 12575043 0xbfe143 0
7 12465403 0xbe34fb 0
1 12646686 0xc0f91e 0
4 12516770 0xbefda2 0

110
output/COUNT:1#NN:8#PPN:8#API:MPIIO#T:102400.txt Normal file
View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 8 -np 64 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 20132659200 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:8#API:MPIIO#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 05:21:10 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 20132659200 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 05:21:10 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 64 (8 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 18.75 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288481689600.
WARNING: Using actual aggregate bytes moved = 27894272000.
write 436.81 19660800 100.00 0.014060 60.89 0.000730 60.90 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288481996800.
WARNING: Using actual aggregate bytes moved = 28205158400.
write 442.84 19660800 100.00 0.002115 60.74 0.000535 60.74 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288483020800.
WARNING: Using actual aggregate bytes moved = 28058112000.
write 440.55 19660800 100.00 0.007811 60.73 0.000577 60.74 2
Max Write: 442.84 MiB/sec (464.36 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 442.84 436.81 440.07 2.49 60.79296 0 64 8 3 0 0 1 0 0 1 20132659200 102400 27894272000 MPIIO 0
Finished: Wed Oct 24 05:24:13 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 8 -np 64 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 8 -np 64 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 20132659200 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
+ tee -a ./output/COUNT:1#NN:8#PPN:8#API:MPIIO#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 05:24:30 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 20132659200 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 05:24:30 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 64 (8 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 18.75 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 25346150400.
read 398.00 19660800 100.00 0.024084 60.71 0.003299 60.73 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 25875046400.
read 406.49 19660800 100.00 0.005116 60.70 0.005719 60.71 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 26402201600.
read 414.86 19660800 100.00 0.005228 60.68 0.011043 60.69 2
Max Read: 414.86 MiB/sec (435.01 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 414.86 398.00 406.45 6.88 60.71121 0 64 8 3 0 0 1 0 0 1 20132659200 102400 25346150400 MPIIO 0
Finished: Wed Oct 24 05:27:32 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12599539 0xc040f3 0
0 12427942 0xbda2a6 0
2 12574286 0xbfde4e 0
4 12516018 0xbefab2 0
7 12464645 0xbe3205 0
1 12645928 0xc0f628 0
3 12494227 0xbea593 0
5 12386296 0xbcfff8 0

110
output/COUNT:1#NN:8#PPN:8#API:MPIIO#T:1048576.txt Normal file
View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 8 -np 64 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 20132659200 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:8#API:MPIIO#T:1048576.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 04:13:18 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 20132659200 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 04:13:18 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 64 (8 per node)
repetitions = 3
xfersize = 1 MiB
blocksize = 18.75 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288490188800.
WARNING: Using actual aggregate bytes moved = 307188727808.
write 4872 19660800 1024.00 0.013533 60.11 0.002723 60.13 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288488091648.
WARNING: Using actual aggregate bytes moved = 302947237888.
write 4802 19660800 1024.00 0.048004 60.11 0.000935 60.16 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288487043072.
WARNING: Using actual aggregate bytes moved = 302345355264.
write 4782 19660800 1024.00 0.182298 60.12 0.000586 60.30 2
Max Write: 4872.12 MiB/sec (5108.78 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 4872.12 4781.84 4818.79 38.63 60.19604 0 64 8 3 0 0 1 0 0 1 20132659200 1048576 307188727808 MPIIO 0
Finished: Wed Oct 24 04:16:25 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 8 -np 64 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 8 -np 64 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 20132659200 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
+ tee -a ./output/COUNT:1#NN:8#PPN:8#API:MPIIO#T:1048576.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 04:16:54 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 20132659200 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 04:16:54 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 64 (8 per node)
repetitions = 3
xfersize = 1 MiB
blocksize = 18.75 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 100134813696.
read 1588.11 19660800 1024.00 0.011837 60.12 0.000871 60.13 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 103155761152.
read 1636.14 19660800 1024.00 0.002649 60.12 0.000955 60.13 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 103716749312.
read 1645.49 19660800 1024.00 0.002954 60.11 0.000986 60.11 2
Max Read: 1645.49 MiB/sec (1725.42 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 1645.49 1588.11 1623.25 25.14 60.12338 0 64 8 3 0 0 1 0 0 1 20132659200 1048576 100134813696 MPIIO 0
Finished: Wed Oct 24 04:19:54 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12599027 0xc03ef3 0
0 12427434 0xbda0aa 0
2 12573773 0xbfdc4d 0
4 12515505 0xbef8b1 0
7 12464129 0xbe3001 0
1 12645418 0xc0f42a 0
3 12493715 0xbea393 0
5 12385781 0xbcfdf5 0

110
output/COUNT:1#NN:8#PPN:8#API:MPIIO#T:10485760.txt Normal file
View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 8 -np 64 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 20132659200 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:8#API:MPIIO#T:10485760.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 03:06:20 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 20132659200 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 03:06:20 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 64 (8 per node)
repetitions = 3
xfersize = 10 MiB
blocksize = 18.75 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288395816960.
WARNING: Using actual aggregate bytes moved = 230256803840.
write 3605 19660800 10240 0.014762 60.90 0.004570 60.92 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288469217280.
WARNING: Using actual aggregate bytes moved = 221301964800.
write 3471.84 19660800 10240 0.021765 60.77 0.000796 60.79 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288448245760.
WARNING: Using actual aggregate bytes moved = 228243537920.
write 3593.63 19660800 10240 0.066377 60.50 0.000925 60.57 2
Max Write: 3604.83 MiB/sec (3779.94 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 3604.83 3471.84 3556.77 60.23 60.75853 0 64 8 3 0 0 1 0 0 1 20132659200 10485760 230256803840 MPIIO 0
Finished: Wed Oct 24 03:09:28 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 8 -np 64 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ tee -a ./output/COUNT:1#NN:8#PPN:8#API:MPIIO#T:10485760.txt
+ /opt/ddn/mvapich/bin/mpiexec -ppn 8 -np 64 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 20132659200 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 03:09:51 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 20132659200 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 03:09:51 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 64 (8 per node)
repetitions = 3
xfersize = 10 MiB
blocksize = 18.75 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 204126289920.
read 3234.82 19660800 10240 0.013093 60.17 0.005014 60.18 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 207230074880.
read 3281.32 19660800 10240 0.006754 60.22 0.005031 60.23 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 210973491200.
read 3342.84 19660800 10240 0.007098 60.18 0.004973 60.19 2
Max Read: 3342.84 MiB/sec (3505.22 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 3342.84 3234.82 3286.33 44.24 60.19892 0 64 8 3 0 0 1 0 0 1 20132659200 10485760 204126289920 MPIIO 0
Finished: Wed Oct 24 03:12:51 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573258 0xbfda4a 0
4 12514996 0xbef6b4 0
7 12463617 0xbe2e01 0
1 12644902 0xc0f226 0
3 12493203 0xbea193 0
5 12385267 0xbcfbf3 0
6 12598518 0xc03cf6 0
0 12426922 0xbd9eaa 0

110
output/COUNT:1#NN:8#PPN:8#API:MPIIO#T:16384.txt Normal file
View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 8 -np 64 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 20132659200 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:8#API:MPIIO#T:16384.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 06:32:43 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 20132659200 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 06:32:43 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 64 (8 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 18.75 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288485191680.
WARNING: Using actual aggregate bytes moved = 4896669696.
write 72.60 19660800 16.00 0.016341 64.30 0.000661 64.32 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288486846464.
WARNING: Using actual aggregate bytes moved = 4818714624.
write 71.58 19660800 16.00 0.002185 64.20 0.000805 64.20 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288489058304.
WARNING: Using actual aggregate bytes moved = 4859789312.
write 72.16 19660800 16.00 0.002538 64.23 0.000916 64.23 2
Max Write: 72.60 MiB/sec (76.13 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 72.60 71.58 72.11 0.42 64.24925 0 64 8 3 0 0 1 0 0 1 20132659200 16384 4896669696 MPIIO 0
Finished: Wed Oct 24 06:35:56 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 8 -np 64 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ tee -a ./output/COUNT:1#NN:8#PPN:8#API:MPIIO#T:16384.txt
+ /opt/ddn/mvapich/bin/mpiexec -ppn 8 -np 64 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 20132659200 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 06:36:04 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 20132659200 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 06:36:04 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 64 (8 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 18.75 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 5439520768.
read 80.81 19660800 16.00 0.023834 64.17 0.004831 64.19 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 5462327296.
read 81.18 19660800 16.00 0.006555 64.16 0.007248 64.17 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 5475713024.
read 81.36 19660800 16.00 0.006923 64.18 0.004780 64.18 2
Max Read: 81.36 MiB/sec (85.32 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 81.36 80.81 81.12 0.23 64.18069 0 64 8 3 0 0 1 0 0 1 20132659200 16384 5439520768 MPIIO 0
Finished: Wed Oct 24 06:39:16 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12574801 0xbfe051 0
4 12516528 0xbefcb0 0
7 12465158 0xbe3406 0
1 12646443 0xc0f82b 0
3 12494746 0xbea79a 0
5 12386810 0xbd01fa 0
6 12600050 0xc042f2 0
0 12428454 0xbda4a6 0

1105
output/COUNT:1#NN:8#PPN:8#API:POSIX#T:102400.txt Normal file
View File

File diff suppressed because it is too large Load Diff

1113
output/COUNT:1#NN:8#PPN:8#API:POSIX#T:1048576.txt Normal file
View File

File diff suppressed because it is too large Load Diff

1110
output/COUNT:1#NN:8#PPN:8#API:POSIX#T:10485760.txt Normal file
View File

File diff suppressed because it is too large Load Diff

1112
output/COUNT:1#NN:8#PPN:8#API:POSIX#T:16384.txt Normal file
View File

File diff suppressed because it is too large Load Diff