NN:8 experiment results

This commit is contained in:
Eugen Betke 2018-10-24 07:42:09 +02:00
parent 2060b9b5f6
commit a351cab075
40 changed files with 14710 additions and 0 deletions

View File

@ -0,0 +1,110 @@
+ tee -a ./output/COUNT:1#NN:8#PPN:1#API:MPIIO#T:102400.txt
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 161061273600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 05:49:36 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 161061273600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 05:49:36 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 8 (1 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 150 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288470323200.
WARNING: Using actual aggregate bytes moved = 15616716800.
write 245.15 157286400 100.00 0.009737 60.74 0.000690 60.75 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288488243200.
WARNING: Using actual aggregate bytes moved = 15217766400.
write 238.95 157286400 100.00 0.001339 60.74 0.000383 60.74 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288469913600.
WARNING: Using actual aggregate bytes moved = 15483904000.
write 242.97 157286400 100.00 0.001689 60.77 0.000406 60.77 2
Max Write: 245.15 MiB/sec (257.06 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 245.15 238.95 242.36 2.57 60.75423 0 8 1 3 0 0 1 0 0 1 161061273600 102400 15616716800 MPIIO 0
Finished: Wed Oct 24 05:52:39 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ tee -a ./output/COUNT:1#NN:8#PPN:1#API:MPIIO#T:102400.txt
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 161061273600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 05:52:42 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 161061273600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 05:52:42 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 8 (1 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 150 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 5620838400.
read 88.26 157286400 100.00 0.012851 60.72 0.000690 60.74 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 5649715200.
read 88.73 157286400 100.00 0.001083 60.72 0.000705 60.72 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 5670297600.
read 89.07 157286400 100.00 0.001049 60.71 0.000719 60.71 2
Max Read: 89.07 MiB/sec (93.39 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 89.07 88.26 88.69 0.33 60.72357 0 8 1 3 0 0 1 0 0 1 161061273600 102400 5620838400 MPIIO 0
Finished: Wed Oct 24 05:55:44 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494351 0xbea60f 0
5 12386422 0xbd0076 0
6 12599666 0xc04172 0
0 12428065 0xbda321 0
2 12574412 0xbfdecc 0
4 12516141 0xbefb2d 0
7 12464771 0xbe3283 0
1 12646052 0xc0f6a4 0

View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 161061273600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:1#API:MPIIO#T:1048576.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 04:40:14 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 161061273600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 04:40:14 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 8 (1 per node)
repetitions = 3
xfersize = 1 MiB
blocksize = 150 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288461877248.
WARNING: Using actual aggregate bytes moved = 162994847744.
write 2585.87 157286400 1024.00 0.013872 60.10 0.000374 60.11 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288479703040.
WARNING: Using actual aggregate bytes moved = 156644671488.
write 2482.69 157286400 1024.00 0.089696 60.08 0.000351 60.17 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288488091648.
WARNING: Using actual aggregate bytes moved = 159087853568.
write 2520.91 157286400 1024.00 0.087149 60.10 0.000404 60.18 2
Max Write: 2585.87 MiB/sec (2711.48 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 2585.87 2482.69 2529.82 42.59 60.15621 0 8 1 3 0 0 1 0 0 1 161061273600 1048576 162994847744 MPIIO 0
Finished: Wed Oct 24 04:43:18 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ tee -a ./output/COUNT:1#NN:8#PPN:1#API:MPIIO#T:1048576.txt
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 161061273600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 04:43:25 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 161061273600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 04:43:25 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 8 (1 per node)
repetitions = 3
xfersize = 1 MiB
blocksize = 150 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 31960596480.
read 506.65 157286400 1024.00 0.076376 60.08 0.000570 60.16 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 33031192576.
read 524.23 157286400 1024.00 0.001699 60.09 0.000530 60.09 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 33508294656.
read 531.82 157286400 1024.00 0.000879 60.09 0.000547 60.09 2
Max Read: 531.82 MiB/sec (557.66 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 531.82 506.65 520.90 10.54 60.11243 0 8 1 3 0 0 1 0 0 1 161061273600 1048576 31960596480 MPIIO 0
Finished: Wed Oct 24 04:46:26 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12599151 0xc03f6f 0
0 12427557 0xbda125 0
2 12573898 0xbfdcca 0
4 12515629 0xbef92d 0
7 12464253 0xbe307d 0
1 12645541 0xc0f4a5 0
3 12493839 0xbea40f 0
5 12385907 0xbcfe73 0

View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 161061273600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:1#API:MPIIO#T:10485760.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 03:33:06 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 161061273600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 03:33:06 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 8 (1 per node)
repetitions = 3
xfersize = 10 MiB
blocksize = 150 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288123187200.
WARNING: Using actual aggregate bytes moved = 105623060480.
write 1673.14 157286400 10240 0.014375 60.19 0.000682 60.20 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288228044800.
WARNING: Using actual aggregate bytes moved = 107091066880.
write 1690.88 157286400 10240 0.051064 60.35 0.000771 60.40 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288186101760.
WARNING: Using actual aggregate bytes moved = 107374182400.
write 1702.76 157286400 10240 0.054793 60.08 0.000918 60.14 2
Max Write: 1702.76 MiB/sec (1785.47 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 1702.76 1673.14 1688.92 12.17 60.24755 0 8 1 3 0 0 1 0 0 1 161061273600 10485760 105623060480 MPIIO 0
Finished: Wed Oct 24 03:36:10 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 161061273600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
+ tee -a ./output/COUNT:1#NN:8#PPN:1#API:MPIIO#T:10485760.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 03:36:17 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 161061273600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 03:36:17 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 8 (1 per node)
repetitions = 3
xfersize = 10 MiB
blocksize = 150 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 96563363840.
read 1531.88 157286400 10240 0.069546 60.05 0.004064 60.12 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 98964602880.
read 1570.90 157286400 10240 0.004699 60.08 0.004031 60.08 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 98744401920.
read 1567.39 157286400 10240 0.004338 60.08 0.004016 60.08 2
Max Read: 1570.90 MiB/sec (1647.20 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 1570.90 1531.88 1556.72 17.62 60.09226 0 8 1 3 0 0 1 0 0 1 161061273600 10485760 96563363840 MPIIO 0
Finished: Wed Oct 24 03:39:17 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12493327 0xbea20f 0
5 12385391 0xbcfc6f 0
6 12598641 0xc03d71 0
0 12427045 0xbd9f25 0
2 12573382 0xbfdac6 0
4 12515119 0xbef72f 0
7 12463739 0xbe2e7b 0
1 12645027 0xc0f2a3 0

View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 161061273600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:1#API:MPIIO#T:16384.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 07:03:49 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 161061273600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 07:03:49 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 8 (1 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 150 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288482652160.
WARNING: Using actual aggregate bytes moved = 2862219264.
write 42.16 157286400 16.00 0.012217 64.72 0.000924 64.74 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288479277056.
WARNING: Using actual aggregate bytes moved = 2864889856.
write 42.17 157286400 16.00 0.001418 64.78 0.000353 64.78 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288487223296.
WARNING: Using actual aggregate bytes moved = 2879717376.
write 42.40 157286400 16.00 0.001358 64.78 0.000395 64.78 2
Max Write: 42.40 MiB/sec (44.45 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 42.40 42.16 42.24 0.11 64.76696 0 8 1 3 0 0 1 0 0 1 161061273600 16384 2862219264 MPIIO 0
Finished: Wed Oct 24 07:07:04 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 161061273600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
+ tee -a ./output/COUNT:1#NN:8#PPN:1#API:MPIIO#T:16384.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 07:07:06 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 161061273600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 07:07:06 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 8 (1 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 150 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 864337920.
read 12.73 157286400 16.00 0.006994 64.72 0.000738 64.73 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 997228544.
read 14.70 157286400 16.00 0.001774 64.71 0.000756 64.71 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 1070743552.
read 15.78 157286400 16.00 0.001112 64.71 0.000835 64.71 2
Max Read: 15.78 MiB/sec (16.55 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 15.78 12.73 14.40 1.26 64.71666 0 8 1 3 0 0 1 0 0 1 161061273600 16384 864337920 MPIIO 0
Finished: Wed Oct 24 07:10:20 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12465283 0xbe3483 0
1 12646569 0xc0f8a9 0
3 12494870 0xbea816 0
5 12386930 0xbd0272 0
6 12600174 0xc0436e 0
0 12428580 0xbda524 0
2 12574926 0xbfe0ce 0
4 12516652 0xbefd2c 0

View File

@ -0,0 +1,219 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 161061273600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:1#API:POSIX#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 05:41:43 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 161061273600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 05:41:43 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 8 (1 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 150 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288465817600.
WARNING: Using actual aggregate bytes moved = 83866828800.
write 881.60 157286400 100.00 0.000598 90.72 0.000482 90.72 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288488550400.
WARNING: Using actual aggregate bytes moved = 80570163200.
write 808.36 157286400 100.00 0.000983 95.05 0.000788 95.05 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288469094400.
WARNING: Using actual aggregate bytes moved = 83156172800.
write 857.98 157286400 100.00 0.000992 92.43 0.000664 92.43 2
Max Write: 881.60 MiB/sec (924.43 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 881.60 808.36 849.31 30.52 92.73608 0 8 1 3 1 0 1 0 0 1 161061273600 102400 83866828800 POSIX 0
Finished: Wed Oct 24 05:46:25 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ tee -a ./output/COUNT:1#NN:8#PPN:1#API:POSIX#T:102400.txt
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 161061273600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 05:46:36 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 161061273600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 05:46:36 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/indread8/file
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 8 (1 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 150 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288490188800.
WARNING: Using actual aggregate bytes moved = 4965376000.
read 78.83 157286400 100.00 0.000876 60.07 0.000693 60.07 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288490188800.
WARNING: Using actual aggregate bytes moved = 5366579200.
read 85.21 157286400 100.00 0.000662 60.06 0.000757 60.06 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288490188800.
WARNING: Using actual aggregate bytes moved = 5337292800.
read 84.75 157286400 100.00 0.000726 60.06 0.000836 60.06 2
Max Read: 85.21 MiB/sec (89.35 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 85.21 78.83 82.93 2.90 60.06219 0 8 1 3 1 0 1 0 0 1 161061273600 102400 4965376000 POSIX 0
Finished: Wed Oct 24 05:49:36 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write.00000006
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12574408 0xbfdec8 0
4 12516137 0xbefb29 0
7 12464768 0xbe3280 0
1 12646049 0xc0f6a1 0
3 12494348 0xbea60c 0
5 12386419 0xbd0073 0
6 12599663 0xc0416f 0
0 12428062 0xbda31e 0
/esfs/jtacquaviva/ioperf/file_write.00000002
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12574405 0xbfdec5 0
1 12646046 0xc0f69e 0
7 12464765 0xbe327d 0
5 12386417 0xbd0071 0
4 12516135 0xbefb27 0
3 12494347 0xbea60b 0
0 12428060 0xbda31c 0
6 12599662 0xc0416e 0
/esfs/jtacquaviva/ioperf/file_write.00000000
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428058 0xbda31a 0
7 12464764 0xbe327c 0
2 12574406 0xbfdec6 0
3 12494345 0xbea609 0
4 12516134 0xbefb26 0
1 12646048 0xc0f6a0 0
5 12386418 0xbd0072 0
6 12599661 0xc0416d 0
/esfs/jtacquaviva/ioperf/file_write.00000004
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464763 0xbe327b 0
3 12494344 0xbea608 0
1 12646047 0xc0f69f 0
5 12386416 0xbd0070 0
6 12599660 0xc0416c 0
4 12516138 0xbefb2a 0
0 12428061 0xbda31d 0
/esfs/jtacquaviva/ioperf/file_write.00000007
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12646043 0xc0f69b 0
6 12599658 0xc0416a 0
0 12428057 0xbda319 0
2 12574404 0xbfdec4 0
3 12494342 0xbea606 0
5 12386415 0xbd006f 0
4 12516133 0xbefb25 0
7 12464766 0xbe327e 0
/esfs/jtacquaviva/ioperf/file_write.00000003
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12646042 0xc0f69a 0
6 12599656 0xc04168 0
2 12574402 0xbfdec2 0
4 12516131 0xbefb23 0
3 12494341 0xbea605 0
0 12428056 0xbda318 0
5 12386413 0xbd006d 0
7 12464762 0xbe327a 0
/esfs/jtacquaviva/ioperf/file_write.00000005
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12386412 0xbd006c 0
0 12428055 0xbda317 0
7 12464761 0xbe3279 0
6 12599657 0xc04169 0
2 12574403 0xbfdec3 0
1 12646044 0xc0f69c 0
3 12494343 0xbea607 0
4 12516132 0xbefb24 0
/esfs/jtacquaviva/ioperf/file_write.00000001
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12386414 0xbd006e 0
1 12646045 0xc0f69d 0
6 12599659 0xc0416b 0
2 12574407 0xbfdec7 0
3 12494346 0xbea60a 0
4 12516136 0xbefb28 0
0 12428059 0xbda31b 0
7 12464767 0xbe327f 0

View File

@ -0,0 +1,216 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 161061273600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:1#API:POSIX#T:1048576.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 04:33:14 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 161061273600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 04:33:14 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 8 (1 per node)
repetitions = 3
xfersize = 1 MiB
blocksize = 150 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288445100032.
WARNING: Using actual aggregate bytes moved = 337465311232.
write 4706 157286400 1024.00 0.000716 68.38 0.000429 68.38 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288446148608.
WARNING: Using actual aggregate bytes moved = 349623549952.
write 4889 157286400 1024.00 0.000895 68.20 0.000514 68.20 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288481800192.
WARNING: Using actual aggregate bytes moved = 338388058112.
write 4738 157286400 1024.00 0.001086 68.11 0.000644 68.11 2
Max Write: 4888.95 MiB/sec (5126.43 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 4888.95 4706.45 4777.83 79.63 68.23050 0 8 1 3 1 0 1 0 0 1 161061273600 1048576 337465311232 POSIX 0
Finished: Wed Oct 24 04:36:51 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 161061273600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
+ tee -a ./output/COUNT:1#NN:8#PPN:1#API:POSIX#T:1048576.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 04:37:13 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 161061273600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 04:37:13 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/indread8/file
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 8 (1 per node)
repetitions = 3
xfersize = 1 MiB
blocksize = 150 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288490188800.
WARNING: Using actual aggregate bytes moved = 39415971840.
read 626.28 157286400 1024.00 0.000842 60.02 0.000558 60.02 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288490188800.
WARNING: Using actual aggregate bytes moved = 40653291520.
read 645.99 157286400 1024.00 0.000575 60.02 0.000583 60.02 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288490188800.
WARNING: Using actual aggregate bytes moved = 41420849152.
read 658.16 157286400 1024.00 0.000618 60.02 0.000540 60.02 2
Max Read: 658.16 MiB/sec (690.14 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 658.16 626.28 643.48 13.14 60.01856 0 8 1 3 1 0 1 0 0 1 161061273600 1048576 39415971840 POSIX 0
Finished: Wed Oct 24 04:40:13 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write.00000006
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464243 0xbe3073 0
5 12385897 0xbcfe69 0
1 12645532 0xc0f49c 0
4 12515621 0xbef925 0
2 12573890 0xbfdcc2 0
0 12427550 0xbda11e 0
6 12599144 0xc03f68 0
3 12493831 0xbea407 0
/esfs/jtacquaviva/ioperf/file_write.00000002
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12427551 0xbda11f 0
5 12385900 0xbcfe6c 0
4 12515623 0xbef927 0
2 12573892 0xbfdcc4 0
1 12645535 0xc0f49f 0
6 12599147 0xc03f6b 0
3 12493835 0xbea40b 0
/esfs/jtacquaviva/ioperf/file_write.00000000
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12599142 0xc03f66 0
1 12645531 0xc0f49b 0
4 12515620 0xbef924 0
2 12573889 0xbfdcc1 0
0 12427549 0xbda11d 0
5 12385899 0xbcfe6b 0
7 12464248 0xbe3078 0
3 12493833 0xbea409 0
/esfs/jtacquaviva/ioperf/file_write.00000004
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12427552 0xbda120 0
3 12493832 0xbea408 0
7 12464249 0xbe3079 0
5 12385903 0xbcfe6f 0
6 12599146 0xc03f6a 0
2 12573895 0xbfdcc7 0
1 12645538 0xc0f4a2 0
4 12515626 0xbef92a 0
/esfs/jtacquaviva/ioperf/file_write.00000007
lmm_stripe_count: 6
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12645533 0xc0f49d 0
7 12464244 0xbe3074 0
3 12493830 0xbea406 0
4 12515622 0xbef926 0
5 12385901 0xbcfe6d 0
2 12573893 0xbfdcc5 0
/esfs/jtacquaviva/ioperf/file_write.00000003
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573888 0xbfdcc0 0
3 12493829 0xbea405 0
7 12464245 0xbe3075 0
5 12385902 0xbcfe6e 0
6 12599145 0xc03f69 0
1 12645537 0xc0f4a1 0
4 12515625 0xbef929 0
0 12427554 0xbda122 0
/esfs/jtacquaviva/ioperf/file_write.00000005
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464246 0xbe3076 0
2 12573891 0xbfdcc3 0
0 12427553 0xbda121 0
1 12645536 0xc0f4a0 0
6 12599148 0xc03f6c 0
5 12385904 0xbcfe70 0
3 12493836 0xbea40c 0
/esfs/jtacquaviva/ioperf/file_write.00000001
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12427548 0xbda11c 0
5 12385898 0xbcfe6a 0
1 12645534 0xc0f49e 0
6 12599143 0xc03f67 0
7 12464247 0xbe3077 0
3 12493834 0xbea40a 0
4 12515624 0xbef928 0
2 12573894 0xbfdcc6 0

View File

@ -0,0 +1,217 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 161061273600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:1#API:POSIX#T:10485760.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 03:26:07 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 161061273600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 03:26:07 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 8 (1 per node)
repetitions = 3
xfersize = 10 MiB
blocksize = 150 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288490188800.
WARNING: Using actual aggregate bytes moved = 384177274880.
write 5452 157286400 10240 0.000488 67.20 0.000300 67.20 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288269987840.
WARNING: Using actual aggregate bytes moved = 383233556480.
write 5361 157286400 10240 0.002442 68.17 0.000870 68.18 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288060272640.
WARNING: Using actual aggregate bytes moved = 382551982080.
write 5416 157286400 10240 0.001030 67.36 0.000696 67.37 2
Max Write: 5452.10 MiB/sec (5716.94 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 5452.10 5360.72 5409.48 37.56 67.58117 0 8 1 3 1 0 1 0 0 1 161061273600 10485760 384177274880 POSIX 0
Finished: Wed Oct 24 03:29:42 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ tee -a ./output/COUNT:1#NN:8#PPN:1#API:POSIX#T:10485760.txt
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 161061273600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 03:30:04 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 161061273600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 03:30:04 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/indread8/file
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 8 (1 per node)
repetitions = 3
xfersize = 10 MiB
blocksize = 150 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288490188800.
WARNING: Using actual aggregate bytes moved = 45329940480.
read 719.44 157286400 10240 0.000782 60.09 0.000533 60.09 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288490188800.
WARNING: Using actual aggregate bytes moved = 46472888320.
read 737.10 157286400 10240 0.000535 60.13 0.000640 60.13 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288490188800.
WARNING: Using actual aggregate bytes moved = 49377443840.
read 783.26 157286400 10240 0.000899 60.12 0.000746 60.12 2
Max Read: 783.26 MiB/sec (821.30 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 783.26 719.44 746.60 26.90 60.11220 0 8 1 3 1 0 1 0 0 1 161061273600 10485760 45329940480 POSIX 0
Finished: Wed Oct 24 03:33:05 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write.00000006
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12598633 0xc03d69 0
0 12427039 0xbd9f1f 0
2 12573378 0xbfdac2 0
7 12463736 0xbe2e78 0
4 12515115 0xbef72b 0
5 12385388 0xbcfc6c 0
1 12645024 0xc0f2a0 0
/esfs/jtacquaviva/ioperf/file_write.00000002
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12463730 0xbe2e72 0
0 12427035 0xbd9f1b 0
4 12515109 0xbef725 0
3 12493319 0xbea207 0
6 12598632 0xc03d68 0
1 12645018 0xc0f29a 0
5 12385383 0xbcfc67 0
2 12573374 0xbfdabe 0
/esfs/jtacquaviva/ioperf/file_write.00000000
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573376 0xbfdac0 0
4 12515112 0xbef728 0
1 12645022 0xc0f29e 0
5 12385387 0xbcfc6b 0
6 12598638 0xc03d6e 0
3 12493324 0xbea20c 0
0 12427041 0xbd9f21 0
/esfs/jtacquaviva/ioperf/file_write.00000004
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12463734 0xbe2e76 0
1 12645020 0xc0f29c 0
3 12493321 0xbea209 0
4 12515114 0xbef72a 0
5 12385386 0xbcfc6a 0
6 12598636 0xc03d6c 0
0 12427040 0xbd9f20 0
/esfs/jtacquaviva/ioperf/file_write.00000007
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12493318 0xbea206 0
2 12573372 0xbfdabc 0
1 12645017 0xc0f299 0
0 12427036 0xbd9f1c 0
4 12515110 0xbef726 0
5 12385384 0xbcfc68 0
7 12463733 0xbe2e75 0
6 12598634 0xc03d6a 0
/esfs/jtacquaviva/ioperf/file_write.00000003
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12493320 0xbea208 0
5 12385382 0xbcfc66 0
7 12463732 0xbe2e74 0
0 12427037 0xbd9f1d 0
2 12573375 0xbfdabf 0
6 12598635 0xc03d6b 0
1 12645021 0xc0f29d 0
4 12515113 0xbef729 0
/esfs/jtacquaviva/ioperf/file_write.00000005
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573377 0xbfdac1 0
5 12385385 0xbcfc69 0
7 12463735 0xbe2e77 0
3 12493323 0xbea20b 0
6 12598637 0xc03d6d 0
1 12645023 0xc0f29f 0
4 12515116 0xbef72c 0
0 12427042 0xbd9f22 0
/esfs/jtacquaviva/ioperf/file_write.00000001
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12598631 0xc03d67 0
7 12463731 0xbe2e73 0
5 12385381 0xbcfc65 0
2 12573373 0xbfdabd 0
4 12515111 0xbef727 0
0 12427038 0xbd9f1e 0
1 12645019 0xc0f29b 0
3 12493322 0xbea20a 0

View File

@ -0,0 +1,220 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 161061273600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:1#API:POSIX#T:16384.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 06:54:51 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 161061273600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 06:54:51 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 8 (1 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 150 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288487698432.
WARNING: Using actual aggregate bytes moved = 31234883584.
write 240.77 157286400 16.00 0.000595 123.72 0.000485 123.72 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288485863424.
WARNING: Using actual aggregate bytes moved = 26906968064.
write 224.12 157286400 16.00 0.000990 114.50 0.000522 114.50 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288469069824.
WARNING: Using actual aggregate bytes moved = 27960279040.
write 241.93 157286400 16.00 0.001059 110.22 0.000543 110.22 2
Max Write: 241.93 MiB/sec (253.68 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 241.93 224.12 235.60 8.14 116.14498 0 8 1 3 1 0 1 0 0 1 161061273600 16384 31234883584 POSIX 0
Finished: Wed Oct 24 07:00:41 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 161061273600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
+ tee -a ./output/COUNT:1#NN:8#PPN:1#API:POSIX#T:16384.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 07:00:47 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 161061273600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 07:00:47 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/indread8/file
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 8 (1 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 150 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288490188800.
WARNING: Using actual aggregate bytes moved = 709541888.
read 11.16 157286400 16.00 0.000829 60.62 0.000619 60.62 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288490188800.
WARNING: Using actual aggregate bytes moved = 1032896512.
read 16.25 157286400 16.00 0.000626 60.62 0.000688 60.62 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288490188800.
WARNING: Using actual aggregate bytes moved = 1021591552.
read 16.07 157286400 16.00 0.000473 60.62 0.000740 60.62 2
Max Read: 16.25 MiB/sec (17.04 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 16.25 11.16 14.49 2.36 60.62368 0 8 1 3 1 0 1 0 0 1 161061273600 16384 709541888 POSIX 0
Finished: Wed Oct 24 07:03:49 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write.00000006
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12516643 0xbefd23 0
0 12428571 0xbda51b 0
6 12600165 0xc04365 0
3 12494863 0xbea80f 0
1 12646562 0xc0f8a2 0
5 12386924 0xbd026c 0
2 12574921 0xbfe0c9 0
7 12465278 0xbe347e 0
/esfs/jtacquaviva/ioperf/file_write.00000002
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12574918 0xbfe0c6 0
6 12600167 0xc04367 0
7 12465277 0xbe347d 0
4 12516646 0xbefd26 0
1 12646563 0xc0f8a3 0
3 12494867 0xbea813 0
0 12428577 0xbda521 0
5 12386927 0xbd026f 0
/esfs/jtacquaviva/ioperf/file_write.00000000
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12465276 0xbe347c 0
2 12574920 0xbfe0c8 0
0 12428575 0xbda51f 0
3 12494866 0xbea812 0
5 12386926 0xbd026e 0
4 12516648 0xbefd28 0
1 12646566 0xc0f8a6 0
6 12600170 0xc0436a 0
/esfs/jtacquaviva/ioperf/file_write.00000004
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428570 0xbda51a 0
3 12494860 0xbea80c 0
2 12574916 0xbfe0c4 0
5 12386920 0xbd0268 0
4 12516642 0xbefd22 0
6 12600163 0xc04363 0
1 12646559 0xc0f89f 0
7 12465273 0xbe3479 0
/esfs/jtacquaviva/ioperf/file_write.00000007
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494861 0xbea80d 0
1 12646560 0xc0f8a0 0
7 12465274 0xbe347a 0
4 12516644 0xbefd24 0
2 12574917 0xbfe0c5 0
0 12428572 0xbda51c 0
6 12600166 0xc04366 0
5 12386923 0xbd026b 0
/esfs/jtacquaviva/ioperf/file_write.00000003
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12646561 0xc0f8a1 0
7 12465275 0xbe347b 0
5 12386921 0xbd0269 0
3 12494864 0xbea810 0
4 12516647 0xbefd27 0
2 12574922 0xbfe0ca 0
0 12428576 0xbda520 0
6 12600169 0xc04369 0
/esfs/jtacquaviva/ioperf/file_write.00000005
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12600164 0xc04364 0
5 12386922 0xbd026a 0
0 12428574 0xbda51e 0
3 12494865 0xbea811 0
1 12646564 0xc0f8a4 0
7 12465279 0xbe347f 0
2 12574923 0xbfe0cb 0
4 12516649 0xbefd29 0
/esfs/jtacquaviva/ioperf/file_write.00000001
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494862 0xbea80e 0
4 12516645 0xbefd25 0
0 12428573 0xbda51d 0
2 12574919 0xbfe0c7 0
6 12600168 0xc04368 0
5 12386925 0xbd026d 0
1 12646565 0xc0f8a5 0
7 12465280 0xbe3480 0

View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 80530636800 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:2#API:MPIIO#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 06:17:34 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 80530636800 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 06:17:34 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 16 (2 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 75 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288487628800.
WARNING: Using actual aggregate bytes moved = 18781184000.
write 295.02 78643200 100.00 0.019334 60.69 0.000530 60.71 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288482713600.
WARNING: Using actual aggregate bytes moved = 18567884800.
write 291.60 78643200 100.00 0.004748 60.72 0.000514 60.73 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288483635200.
WARNING: Using actual aggregate bytes moved = 18424524800.
write 289.30 78643200 100.00 0.001615 60.73 0.000613 60.74 2
Max Write: 295.02 MiB/sec (309.35 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 295.02 289.30 291.97 2.35 60.72489 0 16 2 3 0 0 1 0 0 1 80530636800 102400 18781184000 MPIIO 0
Finished: Wed Oct 24 06:20:37 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ tee -a ./output/COUNT:1#NN:8#PPN:2#API:MPIIO#T:102400.txt
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 80530636800 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 06:20:44 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 80530636800 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 06:20:44 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 16 (2 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 75 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 10075750400.
read 158.23 78643200 100.00 0.021584 60.70 0.007377 60.73 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 10156134400.
read 159.58 78643200 100.00 0.001522 60.69 0.000973 60.69 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 10192384000.
read 160.14 78643200 100.00 0.001574 60.70 0.001101 60.70 2
Max Read: 160.14 MiB/sec (167.92 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 160.14 158.23 159.32 0.80 60.70752 0 16 2 3 0 0 1 0 0 1 80530636800 102400 10075750400 MPIIO 0
Finished: Wed Oct 24 06:23:46 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494550 0xbea6d6 0
5 12386617 0xbd0139 0
6 12599857 0xc04231 0
0 12428261 0xbda3e5 0
2 12574609 0xbfdf91 0
4 12516335 0xbefbef 0
7 12464965 0xbe3345 0
1 12646248 0xc0f768 0

View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 80530636800 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:2#API:MPIIO#T:1048576.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 05:06:45 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 80530636800 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 05:06:45 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 16 (2 per node)
repetitions = 3
xfersize = 1 MiB
blocksize = 75 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288479703040.
WARNING: Using actual aggregate bytes moved = 210129387520.
write 3330.68 78643200 1024.00 0.021551 60.14 0.000686 60.17 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288484945920.
WARNING: Using actual aggregate bytes moved = 204985073664.
write 3242.99 78643200 1024.00 0.166736 60.11 0.000621 60.28 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288479703040.
WARNING: Using actual aggregate bytes moved = 207656845312.
write 3293.97 78643200 1024.00 0.023080 60.10 0.000735 60.12 2
Max Write: 3330.68 MiB/sec (3492.47 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 3330.68 3242.99 3289.21 35.96 60.18935 0 16 2 3 0 0 1 0 0 1 80530636800 1048576 210129387520 MPIIO 0
Finished: Wed Oct 24 05:09:51 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ tee -a ./output/COUNT:1#NN:8#PPN:2#API:MPIIO#T:1048576.txt
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 80530636800 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 05:10:07 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 80530636800 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 05:10:07 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 16 (2 per node)
repetitions = 3
xfersize = 1 MiB
blocksize = 75 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 49119494144.
read 777.58 78643200 1024.00 0.154625 60.09 0.001569 60.24 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 51002736640.
read 809.35 78643200 1024.00 0.001924 60.10 0.001515 60.10 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 51521781760.
read 817.73 78643200 1024.00 0.001927 60.09 0.001569 60.09 2
Max Read: 817.73 MiB/sec (857.45 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 817.73 777.58 801.55 17.29 60.14263 0 16 2 3 0 0 1 0 0 1 80530636800 1048576 49119494144 MPIIO 0
Finished: Wed Oct 24 05:13:08 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12386103 0xbcff37 0
6 12599346 0xc04032 0
0 12427750 0xbda1e6 0
2 12574096 0xbfdd90 0
4 12515825 0xbef9f1 0
7 12464450 0xbe3142 0
1 12645738 0xc0f56a 0
3 12494036 0xbea4d4 0

View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 80530636800 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:2#API:MPIIO#T:10485760.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 03:59:44 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 80530636800 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 03:59:44 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 16 (2 per node)
repetitions = 3
xfersize = 10 MiB
blocksize = 75 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288448245760.
WARNING: Using actual aggregate bytes moved = 131512401920.
write 2072.03 78643200 10240 0.014707 60.51 0.000592 60.53 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288353873920.
WARNING: Using actual aggregate bytes moved = 131082485760.
write 2073.23 78643200 10240 0.102562 60.19 0.000437 60.30 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288490188800.
WARNING: Using actual aggregate bytes moved = 125944463360.
write 1988.73 78643200 10240 0.141789 60.25 0.000477 60.40 2
Max Write: 2073.23 MiB/sec (2173.94 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 2073.23 1988.73 2044.66 39.55 60.40748 0 16 2 3 0 0 1 0 0 1 80530636800 10485760 131512401920 MPIIO 0
Finished: Wed Oct 24 04:02:50 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ tee -a ./output/COUNT:1#NN:8#PPN:2#API:MPIIO#T:10485760.txt
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 80530636800 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 04:03:05 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 80530636800 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 04:03:05 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 16 (2 per node)
repetitions = 3
xfersize = 10 MiB
blocksize = 75 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 134962216960.
read 2139.43 78643200 10240 0.081175 60.08 0.007885 60.16 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 138013573120.
read 2190.36 78643200 10240 0.003115 60.09 0.002665 60.09 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 139901009920.
read 2221.11 78643200 10240 0.003155 60.07 0.002717 60.07 2
Max Read: 2221.11 MiB/sec (2329.01 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 2221.11 2139.43 2183.63 33.69 60.10681 0 16 2 3 0 0 1 0 0 1 80530636800 10485760 134962216960 MPIIO 0
Finished: Wed Oct 24 04:06:06 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12598833 0xc03e31 0
0 12427240 0xbd9fe8 0
2 12573579 0xbfdb8b 0
4 12515313 0xbef7f1 0
7 12463935 0xbe2f3f 0
1 12645224 0xc0f368 0
3 12493522 0xbea2d2 0
5 12385588 0xbcfd34 0

View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 80530636800 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:2#API:MPIIO#T:16384.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 07:35:02 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 80530636800 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 07:35:02 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 16 (2 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 75 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288483028992.
WARNING: Using actual aggregate bytes moved = 3432415232.
write 50.85 78643200 16.00 0.017243 64.35 0.005945 64.37 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288482881536.
WARNING: Using actual aggregate bytes moved = 3391700992.
write 50.21 78643200 16.00 0.001709 64.42 0.000566 64.42 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288478015488.
WARNING: Using actual aggregate bytes moved = 3398025216.
write 50.14 78643200 16.00 0.001647 64.62 0.000643 64.62 2
Max Write: 50.85 MiB/sec (53.32 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 50.85 50.14 50.40 0.32 64.47118 0 16 2 3 0 0 1 0 0 1 80530636800 16384 3432415232 MPIIO 0
Finished: Wed Oct 24 07:38:16 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ tee -a ./output/COUNT:1#NN:8#PPN:2#API:MPIIO#T:16384.txt
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 80530636800 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 07:38:19 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 80530636800 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 07:38:19 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 16 (2 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 75 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 1962360832.
read 29.07 78643200 16.00 0.014638 64.36 0.001892 64.37 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 1974894592.
read 29.27 78643200 16.00 0.002467 64.35 0.001945 64.36 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 1981202432.
read 29.36 78643200 16.00 0.002380 64.36 0.002002 64.36 2
Max Read: 29.36 MiB/sec (30.78 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 29.36 29.07 29.23 0.12 64.36194 0 16 2 3 0 0 1 0 0 1 80530636800 16384 1962360832 MPIIO 0
Finished: Wed Oct 24 07:41:32 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12387130 0xbd033a 0
6 12600376 0xc04438 0
0 12428779 0xbda5eb 0
2 12575126 0xbfe196 0
4 12516852 0xbefdf4 0
7 12465488 0xbe3550 0
1 12646767 0xc0f96f 0
3 12495071 0xbea8df 0

View File

@ -0,0 +1,347 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 80530636800 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:2#API:POSIX#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 06:09:44 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 80530636800 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 06:09:44 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 16 (2 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 75 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288446668800.
WARNING: Using actual aggregate bytes moved = 84816793600.
write 906.70 78643200 100.00 0.000991 89.21 0.000457 89.21 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288423731200.
WARNING: Using actual aggregate bytes moved = 74972160000.
write 776.88 78643200 100.00 0.001311 92.03 0.001350 92.03 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288459468800.
WARNING: Using actual aggregate bytes moved = 77627289600.
write 798.18 78643200 100.00 0.001265 92.75 0.000567 92.75 2
Max Write: 906.70 MiB/sec (950.75 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 906.70 776.88 827.25 56.85 91.33173 0 16 2 3 1 0 1 0 0 1 80530636800 102400 84816793600 POSIX 0
Finished: Wed Oct 24 06:14:21 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ tee -a ./output/COUNT:1#NN:8#PPN:2#API:POSIX#T:102400.txt
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 80530636800 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 06:14:33 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 80530636800 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 06:14:33 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/indread8/file
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 16 (2 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 75 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 2576980377600.
WARNING: Using actual aggregate bytes moved = 11614617600.
read 184.49 78643200 100.00 0.001346 60.04 0.001002 60.04 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 2576980377600.
WARNING: Using actual aggregate bytes moved = 11714048000.
read 186.08 78643200 100.00 0.001052 60.04 0.001039 60.04 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 2576980377600.
WARNING: Using actual aggregate bytes moved = 11938304000.
read 189.62 78643200 100.00 0.001054 60.04 0.001043 60.04 2
Max Read: 189.62 MiB/sec (198.83 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 189.62 184.49 186.73 2.15 60.03849 0 16 2 3 1 0 1 0 0 1 80530636800 102400 11614617600 POSIX 0
Finished: Wed Oct 24 06:17:33 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write.00000006
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12646231 0xc0f757 0
3 12494534 0xbea6c6 0
6 12599842 0xc04222 0
0 12428245 0xbda3d5 0
2 12574594 0xbfdf82 0
5 12386604 0xbd012c 0
7 12464952 0xbe3338 0
4 12516324 0xbefbe4 0
/esfs/jtacquaviva/ioperf/file_write.00000002
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428243 0xbda3d3 0
4 12516317 0xbefbdd 0
6 12599840 0xc04220 0
1 12646230 0xc0f756 0
2 12574590 0xbfdf7e 0
3 12494532 0xbea6c4 0
5 12386599 0xbd0127 0
7 12464947 0xbe3333 0
/esfs/jtacquaviva/ioperf/file_write.00000000
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428244 0xbda3d4 0
3 12494533 0xbea6c5 0
5 12386600 0xbd0128 0
4 12516318 0xbefbde 0
7 12464948 0xbe3334 0
6 12599843 0xc04223 0
1 12646235 0xc0f75b 0
2 12574595 0xbfdf83 0
/esfs/jtacquaviva/ioperf/file_write.00000004
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12386602 0xbd012a 0
2 12574592 0xbfdf80 0
0 12428246 0xbda3d6 0
4 12516323 0xbefbe3 0
3 12494538 0xbea6ca 0
7 12464953 0xbe3339 0
6 12599844 0xc04224 0
1 12646237 0xc0f75d 0
/esfs/jtacquaviva/ioperf/file_write.00000007
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12386611 0xbd0133 0
6 12599851 0xc0422b 0
0 12428256 0xbda3e0 0
2 12574604 0xbfdf8c 0
4 12516330 0xbefbea 0
7 12464960 0xbe3340 0
1 12646244 0xc0f764 0
3 12494547 0xbea6d3 0
/esfs/jtacquaviva/ioperf/file_write.00000012
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12599853 0xc0422d 0
4 12516331 0xbefbeb 0
7 12464961 0xbe3341 0
3 12494546 0xbea6d2 0
0 12428257 0xbda3e1 0
1 12646245 0xc0f765 0
5 12386614 0xbd0136 0
2 12574605 0xbfdf8d 0
/esfs/jtacquaviva/ioperf/file_write.00000013
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12646233 0xc0f759 0
2 12574593 0xbfdf81 0
4 12516322 0xbefbe2 0
0 12428249 0xbda3d9 0
5 12386606 0xbd012e 0
7 12464954 0xbe333a 0
6 12599846 0xc04226 0
3 12494539 0xbea6cb 0
/esfs/jtacquaviva/ioperf/file_write.00000003
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12599847 0xc04227 0
0 12428251 0xbda3db 0
2 12574598 0xbfdf86 0
4 12516325 0xbefbe5 0
7 12464955 0xbe333b 0
1 12646238 0xc0f75e 0
3 12494540 0xbea6cc 0
5 12386607 0xbd012f 0
/esfs/jtacquaviva/ioperf/file_write.00000008
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12516326 0xbefbe6 0
7 12464957 0xbe333d 0
1 12646240 0xc0f760 0
3 12494542 0xbea6ce 0
5 12386609 0xbd0131 0
6 12599849 0xc04229 0
0 12428253 0xbda3dd 0
2 12574600 0xbfdf88 0
/esfs/jtacquaviva/ioperf/file_write.00000010
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428252 0xbda3dc 0
7 12464956 0xbe333c 0
1 12646239 0xc0f75f 0
3 12494541 0xbea6cd 0
5 12386608 0xbd0130 0
6 12599848 0xc04228 0
2 12574599 0xbfdf87 0
4 12516327 0xbefbe7 0
/esfs/jtacquaviva/ioperf/file_write.00000005
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12574603 0xbfdf8b 0
1 12646243 0xc0f763 0
5 12386613 0xbd0135 0
4 12516332 0xbefbec 0
3 12494545 0xbea6d1 0
7 12464962 0xbe3342 0
6 12599854 0xc0422e 0
0 12428258 0xbda3e2 0
/esfs/jtacquaviva/ioperf/file_write.00000014
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494543 0xbea6cf 0
6 12599850 0xc0422a 0
2 12574601 0xbfdf89 0
7 12464959 0xbe333f 0
5 12386612 0xbd0134 0
0 12428255 0xbda3df 0
4 12516329 0xbefbe9 0
1 12646242 0xc0f762 0
/esfs/jtacquaviva/ioperf/file_write.00000009
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12386601 0xbd0129 0
2 12574591 0xbfdf7f 0
6 12599841 0xc04221 0
1 12646234 0xc0f75a 0
7 12464950 0xbe3336 0
4 12516321 0xbefbe1 0
0 12428247 0xbda3d7 0
3 12494537 0xbea6c9 0
/esfs/jtacquaviva/ioperf/file_write.00000015
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12516319 0xbefbdf 0
3 12494535 0xbea6c7 0
7 12464949 0xbe3335 0
1 12646236 0xc0f75c 0
5 12386605 0xbd012d 0
6 12599845 0xc04225 0
2 12574597 0xbfdf85 0
0 12428250 0xbda3da 0
/esfs/jtacquaviva/ioperf/file_write.00000001
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464958 0xbe333e 0
5 12386610 0xbd0132 0
0 12428254 0xbda3de 0
4 12516328 0xbefbe8 0
1 12646241 0xc0f761 0
6 12599852 0xc0422c 0
2 12574602 0xbfdf8a 0
3 12494544 0xbea6d0 0
/esfs/jtacquaviva/ioperf/file_write.00000011
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12646232 0xc0f758 0
4 12516320 0xbefbe0 0
7 12464951 0xbe3337 0
3 12494536 0xbea6c8 0
5 12386603 0xbd012b 0
0 12428248 0xbda3d8 0
2 12574596 0xbfdf84 0

View File

@ -0,0 +1,347 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 80530636800 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:2#API:POSIX#T:1048576.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 04:59:49 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 80530636800 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 04:59:49 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 16 (2 per node)
repetitions = 3
xfersize = 1 MiB
blocksize = 75 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288456634368.
WARNING: Using actual aggregate bytes moved = 357283397632.
write 4918 78643200 1024.00 0.000937 69.28 0.000542 69.28 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288490188800.
WARNING: Using actual aggregate bytes moved = 343360405504.
write 4761 78643200 1024.00 0.001388 68.77 0.001163 68.78 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288473411584.
WARNING: Using actual aggregate bytes moved = 351454363648.
write 4870 78643200 1024.00 0.001704 68.82 0.000890 68.82 2
Max Write: 4917.97 MiB/sec (5156.86 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 4917.97 4761.16 4849.85 65.65 68.95912 0 16 2 3 1 0 1 0 0 1 80530636800 1048576 357283397632 POSIX 0
Finished: Wed Oct 24 05:03:25 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 80530636800 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
+ tee -a ./output/COUNT:1#NN:8#PPN:2#API:POSIX#T:1048576.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 05:03:44 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 80530636800 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 05:03:44 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/indread8/file
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 16 (2 per node)
repetitions = 3
xfersize = 1 MiB
blocksize = 75 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 2576980377600.
WARNING: Using actual aggregate bytes moved = 78327578624.
read 1244.50 78643200 1024.00 0.000621 60.02 0.000589 60.02 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 2576980377600.
WARNING: Using actual aggregate bytes moved = 83850428416.
read 1332.06 78643200 1024.00 0.000527 60.03 0.000671 60.03 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 2576980377600.
WARNING: Using actual aggregate bytes moved = 90405076992.
read 1436.48 78643200 1024.00 0.000686 60.02 0.000852 60.02 2
Max Read: 1436.48 MiB/sec (1506.26 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 1436.48 1244.50 1337.68 78.48 60.02489 0 16 2 3 1 0 1 0 0 1 80530636800 1048576 78327578624 POSIX 0
Finished: Wed Oct 24 05:06:44 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write.00000006
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12386085 0xbcff25 0
0 12427732 0xbda1d4 0
2 12574079 0xbfdd7f 0
3 12494019 0xbea4c3 0
6 12599330 0xc04022 0
4 12515808 0xbef9e0 0
7 12464434 0xbe3132 0
1 12645721 0xc0f559 0
/esfs/jtacquaviva/ioperf/file_write.00000002
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464433 0xbe3131 0
0 12427734 0xbda1d6 0
4 12515807 0xbef9df 0
2 12574080 0xbfdd80 0
3 12494021 0xbea4c5 0
5 12386087 0xbcff27 0
1 12645722 0xc0f55a 0
/esfs/jtacquaviva/ioperf/file_write.00000000
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12515821 0xbef9ed 0
7 12464447 0xbe313f 0
1 12645735 0xc0f567 0
3 12494033 0xbea4d1 0
5 12386100 0xbcff34 0
6 12599343 0xc0402f 0
0 12427747 0xbda1e3 0
2 12574093 0xbfdd8d 0
/esfs/jtacquaviva/ioperf/file_write.00000004
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12645729 0xc0f561 0
2 12574090 0xbfdd8a 0
3 12494028 0xbea4cc 0
5 12386094 0xbcff2e 0
4 12515819 0xbef9eb 0
6 12599339 0xc0402b 0
7 12464446 0xbe313e 0
0 12427745 0xbda1e1 0
/esfs/jtacquaviva/ioperf/file_write.00000007
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12427739 0xbda1db 0
1 12645727 0xc0f55f 0
6 12599335 0xc04027 0
7 12464440 0xbe3138 0
4 12515815 0xbef9e7 0
2 12574087 0xbfdd87 0
3 12494027 0xbea4cb 0
5 12386095 0xbcff2f 0
/esfs/jtacquaviva/ioperf/file_write.00000012
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12574091 0xbfdd8b 0
4 12515816 0xbef9e8 0
7 12464443 0xbe313b 0
1 12645733 0xc0f565 0
3 12494031 0xbea4cf 0
5 12386097 0xbcff31 0
6 12599341 0xc0402d 0
0 12427746 0xbda1e2 0
/esfs/jtacquaviva/ioperf/file_write.00000013
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12645720 0xc0f558 0
2 12574078 0xbfdd7e 0
3 12494018 0xbea4c2 0
4 12515806 0xbef9de 0
6 12599329 0xc04021 0
7 12464432 0xbe3130 0
0 12427733 0xbda1d5 0
5 12386086 0xbcff26 0
/esfs/jtacquaviva/ioperf/file_write.00000003
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12427741 0xbda1dd 0
2 12574088 0xbfdd88 0
1 12645731 0xc0f563 0
3 12494029 0xbea4cd 0
6 12599338 0xc0402a 0
7 12464445 0xbe313d 0
4 12515820 0xbef9ec 0
5 12386098 0xbcff32 0
/esfs/jtacquaviva/ioperf/file_write.00000008
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12427742 0xbda1de 0
2 12574092 0xbfdd8c 0
4 12515817 0xbef9e9 0
7 12464444 0xbe313c 0
1 12645734 0xc0f566 0
3 12494032 0xbea4d0 0
5 12386099 0xbcff33 0
6 12599342 0xc0402e 0
/esfs/jtacquaviva/ioperf/file_write.00000010
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12386092 0xbcff2c 0
7 12464439 0xbe3137 0
3 12494025 0xbea4c9 0
0 12427740 0xbda1dc 0
4 12515814 0xbef9e6 0
2 12574086 0xbfdd86 0
1 12645730 0xc0f562 0
6 12599337 0xc04029 0
/esfs/jtacquaviva/ioperf/file_write.00000005
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494020 0xbea4c4 0
6 12599332 0xc04024 0
4 12515810 0xbef9e2 0
5 12386089 0xbcff29 0
0 12427736 0xbda1d8 0
2 12574081 0xbfdd81 0
1 12645725 0xc0f55d 0
7 12464437 0xbe3135 0
/esfs/jtacquaviva/ioperf/file_write.00000014
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12515813 0xbef9e5 0
5 12386093 0xbcff2d 0
2 12574085 0xbfdd85 0
1 12645728 0xc0f560 0
7 12464441 0xbe3139 0
3 12494026 0xbea4ca 0
6 12599336 0xc04028 0
0 12427743 0xbda1df 0
/esfs/jtacquaviva/ioperf/file_write.00000009
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494022 0xbea4c6 0
5 12386090 0xbcff2a 0
6 12599334 0xc04026 0
0 12427738 0xbda1da 0
2 12574084 0xbfdd84 0
4 12515812 0xbef9e4 0
7 12464438 0xbe3136 0
1 12645726 0xc0f55e 0
/esfs/jtacquaviva/ioperf/file_write.00000015
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12427735 0xbda1d7 0
7 12464435 0xbe3133 0
4 12515809 0xbef9e1 0
5 12386088 0xbcff28 0
1 12645724 0xc0f55c 0
3 12494024 0xbea4c8 0
2 12574082 0xbfdd82 0
6 12599333 0xc04025 0
/esfs/jtacquaviva/ioperf/file_write.00000001
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464442 0xbe313a 0
2 12574089 0xbfdd89 0
1 12645732 0xc0f564 0
4 12515818 0xbef9ea 0
3 12494030 0xbea4ce 0
5 12386096 0xbcff30 0
6 12599340 0xc0402c 0
0 12427744 0xbda1e0 0
/esfs/jtacquaviva/ioperf/file_write.00000011
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12599331 0xc04023 0
7 12464436 0xbe3134 0
1 12645723 0xc0f55b 0
3 12494023 0xbea4c7 0
2 12574083 0xbfdd83 0
0 12427737 0xbda1d9 0
5 12386091 0xbcff2b 0
4 12515811 0xbef9e3 0

View File

@ -0,0 +1,343 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 80530636800 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:2#API:POSIX#T:10485760.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 03:52:43 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 80530636800 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 03:52:43 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 16 (2 per node)
repetitions = 3
xfersize = 10 MiB
blocksize = 75 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1287546470400.
WARNING: Using actual aggregate bytes moved = 390038814720.
write 5481 78643200 10240 0.000734 67.87 0.000542 67.87 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288490188800.
WARNING: Using actual aggregate bytes moved = 396298813440.
write 5558 78643200 10240 0.001981 68.00 0.000589 68.00 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288322416640.
WARNING: Using actual aggregate bytes moved = 391653621760.
write 5490 78643200 10240 0.001679 68.04 0.000678 68.04 2
Max Write: 5557.87 MiB/sec (5827.85 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 5557.87 5480.65 5509.41 34.47 67.96959 0 16 2 3 1 0 1 0 0 1 80530636800 10485760 390038814720 POSIX 0
Finished: Wed Oct 24 03:56:17 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ tee -a ./output/COUNT:1#NN:8#PPN:2#API:POSIX#T:10485760.txt
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 80530636800 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 03:56:42 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 80530636800 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 03:56:42 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/indread8/file
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 16 (2 per node)
repetitions = 3
xfersize = 10 MiB
blocksize = 75 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 2576980377600.
WARNING: Using actual aggregate bytes moved = 90177536000.
read 1430.03 78643200 10240 0.002533 60.14 0.002128 60.14 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 2576980377600.
WARNING: Using actual aggregate bytes moved = 97989427200.
read 1544.39 78643200 10240 0.002137 60.51 0.002124 60.51 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 2576980377600.
WARNING: Using actual aggregate bytes moved = 106472407040.
read 1688.81 78643200 10240 0.002131 60.12 0.002109 60.13 2
Max Read: 1688.81 MiB/sec (1770.85 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 1688.81 1430.03 1554.41 105.88 60.25765 0 16 2 3 1 0 1 0 0 1 80530636800 10485760 90177536000 POSIX 0
Finished: Wed Oct 24 03:59:43 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write.00000006
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12427227 0xbd9fdb 0
1 12645213 0xc0f35d 0
2 12573566 0xbfdb7e 0
4 12515301 0xbef7e5 0
6 12598822 0xc03e26 0
5 12385577 0xbcfd29 0
7 12463921 0xbe2f31 0
3 12493512 0xbea2c8 0
/esfs/jtacquaviva/ioperf/file_write.00000002
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12463928 0xbe2f38 0
1 12645218 0xc0f362 0
5 12385582 0xbcfd2e 0
0 12427235 0xbd9fe3 0
4 12515308 0xbef7ec 0
3 12493519 0xbea2cf 0
6 12598830 0xc03e2e 0
2 12573575 0xbfdb87 0
/esfs/jtacquaviva/ioperf/file_write.00000000
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12493513 0xbea2c9 0
5 12385579 0xbcfd2b 0
6 12598824 0xc03e28 0
0 12427232 0xbd9fe0 0
2 12573571 0xbfdb83 0
4 12515304 0xbef7e8 0
7 12463925 0xbe2f35 0
1 12645215 0xc0f35f 0
/esfs/jtacquaviva/ioperf/file_write.00000004
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573574 0xbfdb86 0
3 12493517 0xbea2cd 0
6 12598828 0xc03e2c 0
7 12463931 0xbe2f3b 0
5 12385584 0xbcfd30 0
0 12427236 0xbd9fe4 0
1 12645221 0xc0f365 0
4 12515309 0xbef7ed 0
/esfs/jtacquaviva/ioperf/file_write.00000007
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12463930 0xbe2f3a 0
1 12645220 0xc0f364 0
3 12493518 0xbea2ce 0
5 12385585 0xbcfd31 0
6 12598829 0xc03e2d 0
0 12427237 0xbd9fe5 0
2 12573576 0xbfdb88 0
4 12515310 0xbef7ee 0
/esfs/jtacquaviva/ioperf/file_write.00000012
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573573 0xbfdb85 0
7 12463927 0xbe2f37 0
3 12493515 0xbea2cb 0
6 12598827 0xc03e2b 0
4 12515307 0xbef7eb 0
1 12645219 0xc0f363 0
5 12385583 0xbcfd2f 0
/esfs/jtacquaviva/ioperf/file_write.00000013
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12645210 0xc0f35a 0
6 12598820 0xc03e24 0
2 12573568 0xbfdb80 0
5 12385575 0xbcfd27 0
3 12493510 0xbea2c6 0
0 12427231 0xbd9fdf 0
7 12463923 0xbe2f33 0
/esfs/jtacquaviva/ioperf/file_write.00000003
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12515300 0xbef7e4 0
6 12598821 0xc03e25 0
5 12385576 0xbcfd28 0
0 12427230 0xbd9fde 0
1 12645214 0xc0f35e 0
2 12573570 0xbfdb82 0
7 12463924 0xbe2f34 0
/esfs/jtacquaviva/ioperf/file_write.00000008
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12598826 0xc03e2a 0
4 12515306 0xbef7ea 0
1 12645217 0xc0f361 0
5 12385581 0xbcfd2d 0
0 12427234 0xbd9fe2 0
7 12463929 0xbe2f39 0
3 12493516 0xbea2cc 0
/esfs/jtacquaviva/ioperf/file_write.00000010
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12385580 0xbcfd2c 0
6 12598825 0xc03e29 0
0 12427233 0xbd9fe1 0
2 12573572 0xbfdb84 0
4 12515305 0xbef7e9 0
7 12463926 0xbe2f36 0
1 12645216 0xc0f360 0
3 12493514 0xbea2ca 0
/esfs/jtacquaviva/ioperf/file_write.00000005
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12645211 0xc0f35b 0
0 12427228 0xbd9fdc 0
5 12385574 0xbcfd26 0
4 12515303 0xbef7e7 0
2 12573569 0xbfdb81 0
3 12493511 0xbea2c7 0
7 12463922 0xbe2f32 0
6 12598823 0xc03e27 0
/esfs/jtacquaviva/ioperf/file_write.00000014
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12493508 0xbea2c4 0
2 12573564 0xbfdb7c 0
0 12427226 0xbd9fda 0
4 12515299 0xbef7e3 0
7 12463919 0xbe2f2f 0
5 12385573 0xbcfd25 0
6 12598819 0xc03e23 0
/esfs/jtacquaviva/ioperf/file_write.00000009
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12598818 0xc03e22 0
3 12493509 0xbea2c5 0
1 12645212 0xc0f35c 0
7 12463920 0xbe2f30 0
2 12573567 0xbfdb7f 0
4 12515302 0xbef7e6 0
0 12427229 0xbd9fdd 0
5 12385578 0xbcfd2a 0
/esfs/jtacquaviva/ioperf/file_write.00000015
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573562 0xbfdb7a 0
4 12515296 0xbef7e0 0
7 12463916 0xbe2f2c 0
1 12645207 0xc0f357 0
3 12493505 0xbea2c1 0
5 12385570 0xbcfd22 0
6 12598815 0xc03e1f 0
0 12427223 0xbd9fd7 0
/esfs/jtacquaviva/ioperf/file_write.00000001
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12515297 0xbef7e1 0
7 12463917 0xbe2f2d 0
5 12385571 0xbcfd23 0
6 12598816 0xc03e20 0
0 12427224 0xbd9fd8 0
1 12645208 0xc0f358 0
3 12493506 0xbea2c2 0
2 12573563 0xbfdb7b 0
/esfs/jtacquaviva/ioperf/file_write.00000011
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12598817 0xc03e21 0
3 12493507 0xbea2c3 0
0 12427225 0xbd9fd9 0
5 12385572 0xbcfd24 0
7 12463918 0xbe2f2e 0
4 12515298 0xbef7e2 0
1 12645209 0xc0f359 0
2 12573565 0xbfdb7d 0

View File

@ -0,0 +1,343 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 80530636800 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:2#API:POSIX#T:16384.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 07:25:56 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 80530636800 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 07:25:56 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 16 (2 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 75 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288470528000.
WARNING: Using actual aggregate bytes moved = 32117735424.
write 253.16 78643200 16.00 0.000766 120.99 0.000383 120.99 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288479817728.
WARNING: Using actual aggregate bytes moved = 26547306496.
write 211.75 78643200 16.00 0.001222 119.56 0.000635 119.57 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288482062336.
WARNING: Using actual aggregate bytes moved = 28408348672.
write 231.65 78643200 16.00 0.001139 116.95 0.000387 116.95 2
Max Write: 253.16 MiB/sec (265.45 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 253.16 211.75 232.19 16.91 119.16970 0 16 2 3 1 0 1 0 0 1 80530636800 16384 32117735424 POSIX 0
Finished: Wed Oct 24 07:31:55 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ tee -a ./output/COUNT:1#NN:8#PPN:2#API:POSIX#T:16384.txt
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 80530636800 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 07:32:00 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 80530636800 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 07:32:00 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/indread8/file
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 16 (2 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 75 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 2576980377600.
WARNING: Using actual aggregate bytes moved = 1782382592.
read 28.19 78643200 16.00 0.003095 60.29 0.002781 60.29 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 2576980377600.
WARNING: Using actual aggregate bytes moved = 2341502976.
read 37.04 78643200 16.00 0.002794 60.29 0.002829 60.29 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 2576980377600.
WARNING: Using actual aggregate bytes moved = 2328739840.
read 36.84 78643200 16.00 0.002892 60.28 0.002882 60.28 2
Max Read: 37.04 MiB/sec (38.84 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 37.04 28.19 34.03 4.12 60.28596 0 16 2 3 1 0 1 0 0 1 80530636800 16384 1782382592 POSIX 0
Finished: Wed Oct 24 07:35:01 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write.00000006
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12646748 0xc0f95c 0
4 12516833 0xbefde1 0
3 12495053 0xbea8cd 0
6 12600358 0xc04426 0
0 12428764 0xbda5dc 0
2 12575110 0xbfe186 0
7 12465472 0xbe3540 0
5 12387116 0xbd032c 0
/esfs/jtacquaviva/ioperf/file_write.00000002
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12387119 0xbd032f 0
6 12600364 0xc0442c 0
4 12516841 0xbefde9 0
7 12465476 0xbe3544 0
1 12646756 0xc0f964 0
0 12428768 0xbda5e0 0
2 12575115 0xbfe18b 0
3 12495062 0xbea8d6 0
/esfs/jtacquaviva/ioperf/file_write.00000000
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12646749 0xc0f95d 0
0 12428762 0xbda5da 0
4 12516836 0xbefde4 0
7 12465471 0xbe353f 0
5 12387115 0xbd032b 0
3 12495057 0xbea8d1 0
6 12600361 0xc04429 0
/esfs/jtacquaviva/ioperf/file_write.00000004
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428767 0xbda5df 0
1 12646757 0xc0f965 0
3 12495061 0xbea8d5 0
4 12516842 0xbefdea 0
5 12387121 0xbd0331 0
6 12600366 0xc0442e 0
2 12575116 0xbfe18c 0
7 12465478 0xbe3546 0
/esfs/jtacquaviva/ioperf/file_write.00000007
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12600369 0xc04431 0
0 12428772 0xbda5e4 0
2 12575119 0xbfe18f 0
4 12516846 0xbefdee 0
7 12465481 0xbe3549 0
1 12646761 0xc0f969 0
3 12495065 0xbea8d9 0
5 12387124 0xbd0334 0
/esfs/jtacquaviva/ioperf/file_write.00000012
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12575105 0xbfe181 0
1 12646747 0xc0f95b 0
5 12387110 0xbd0326 0
3 12495051 0xbea8cb 0
4 12516831 0xbefddf 0
6 12600354 0xc04422 0
7 12465466 0xbe353a 0
0 12428758 0xbda5d6 0
/esfs/jtacquaviva/ioperf/file_write.00000013
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12495058 0xbea8d2 0
0 12428766 0xbda5de 0
7 12465475 0xbe3543 0
2 12575113 0xbfe189 0
5 12387118 0xbd032e 0
6 12600363 0xc0442b 0
4 12516840 0xbefde8 0
1 12646755 0xc0f963 0
/esfs/jtacquaviva/ioperf/file_write.00000003
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12600357 0xc04425 0
7 12465469 0xbe353d 0
1 12646751 0xc0f95f 0
5 12387114 0xbd032a 0
2 12575109 0xbfe185 0
4 12516837 0xbefde5 0
0 12428765 0xbda5dd 0
/esfs/jtacquaviva/ioperf/file_write.00000008
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12575112 0xbfe188 0
4 12516839 0xbefde7 0
7 12465474 0xbe3542 0
5 12387117 0xbd032d 0
6 12600362 0xc0442a 0
1 12646754 0xc0f962 0
3 12495059 0xbea8d3 0
/esfs/jtacquaviva/ioperf/file_write.00000010
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12575114 0xbfe18a 0
3 12495060 0xbea8d4 0
0 12428769 0xbda5e1 0
6 12600365 0xc0442d 0
5 12387120 0xbd0330 0
4 12516843 0xbefdeb 0
7 12465477 0xbe3545 0
1 12646758 0xc0f966 0
/esfs/jtacquaviva/ioperf/file_write.00000005
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12516834 0xbefde2 0
5 12387113 0xbd0329 0
6 12600359 0xc04427 0
0 12428763 0xbda5db 0
2 12575108 0xbfe184 0
3 12495056 0xbea8d0 0
1 12646753 0xc0f961 0
7 12465473 0xbe3541 0
/esfs/jtacquaviva/ioperf/file_write.00000014
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428760 0xbda5d8 0
5 12387112 0xbd0328 0
6 12600356 0xc04424 0
1 12646750 0xc0f95e 0
3 12495054 0xbea8ce 0
7 12465470 0xbe353e 0
2 12575111 0xbfe187 0
4 12516838 0xbefde6 0
/esfs/jtacquaviva/ioperf/file_write.00000009
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12465467 0xbe353b 0
3 12495052 0xbea8cc 0
4 12516832 0xbefde0 0
0 12428759 0xbda5d7 0
5 12387111 0xbd0327 0
2 12575106 0xbfe182 0
6 12600355 0xc04423 0
/esfs/jtacquaviva/ioperf/file_write.00000015
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12495063 0xbea8d7 0
5 12387122 0xbd0332 0
6 12600367 0xc0442f 0
0 12428770 0xbda5e2 0
2 12575117 0xbfe18d 0
4 12516844 0xbefdec 0
7 12465479 0xbe3547 0
1 12646759 0xc0f967 0
/esfs/jtacquaviva/ioperf/file_write.00000001
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12387123 0xbd0333 0
6 12600368 0xc04430 0
0 12428771 0xbda5e3 0
2 12575118 0xbfe18e 0
4 12516845 0xbefded 0
7 12465480 0xbe3548 0
1 12646760 0xc0f968 0
3 12495064 0xbea8d8 0
/esfs/jtacquaviva/ioperf/file_write.00000011
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428761 0xbda5d9 0
2 12575107 0xbfe183 0
7 12465468 0xbe353c 0
4 12516835 0xbefde3 0
6 12600360 0xc04428 0
3 12495055 0xbea8cf 0
1 12646752 0xc0f960 0

View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 40265318400 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:4#API:MPIIO#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 05:35:28 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 40265318400 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 05:35:28 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 32 (4 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 37.50 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288488243200.
WARNING: Using actual aggregate bytes moved = 22185164800.
write 348.41 39321600 100.00 0.019215 60.71 0.000534 60.73 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288483942400.
WARNING: Using actual aggregate bytes moved = 22497484800.
write 353.21 39321600 100.00 0.001917 60.74 0.000539 60.74 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288487424000.
WARNING: Using actual aggregate bytes moved = 22288896000.
write 349.81 39321600 100.00 0.002036 60.76 0.000559 60.76 2
Max Write: 353.21 MiB/sec (370.37 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 353.21 348.41 350.48 2.01 60.74474 0 32 4 3 0 0 1 0 0 1 40265318400 102400 22185164800 MPIIO 0
Finished: Wed Oct 24 05:38:31 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 40265318400 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
+ tee -a ./output/COUNT:1#NN:8#PPN:4#API:MPIIO#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 05:38:40 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 40265318400 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 05:38:40 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 32 (4 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 37.50 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 16698368000.
read 262.32 39321600 100.00 0.014407 60.69 0.006799 60.71 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 16845721600.
read 264.70 39321600 100.00 0.005292 60.69 0.004282 60.69 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 16842854400.
read 264.62 39321600 100.00 0.005390 60.70 0.004318 60.70 2
Max Read: 264.70 MiB/sec (277.56 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 264.70 262.32 263.88 1.10 60.70035 0 32 4 3 0 0 1 0 0 1 40265318400 102400 16698368000 MPIIO 0
Finished: Wed Oct 24 05:41:43 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464743 0xbe3267 0
1 12646026 0xc0f68a 0
3 12494324 0xbea5f4 0
5 12386395 0xbd005b 0
6 12599639 0xc04157 0
0 12428038 0xbda306 0
2 12574385 0xbfdeb1 0
4 12516115 0xbefb13 0

View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 40265318400 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:4#API:MPIIO#T:1048576.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 04:26:49 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 40265318400 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 04:26:49 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 32 (4 per node)
repetitions = 3
xfersize = 1 MiB
blocksize = 37.50 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288488091648.
WARNING: Using actual aggregate bytes moved = 261622857728.
write 4151 39321600 1024.00 0.009951 60.09 0.000559 60.10 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288488091648.
WARNING: Using actual aggregate bytes moved = 258843082752.
write 4096 39321600 1024.00 0.180245 60.09 0.000508 60.27 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288484945920.
WARNING: Using actual aggregate bytes moved = 262620053504.
write 4156 39321600 1024.00 0.143859 60.12 0.000498 60.27 2
Max Write: 4155.83 MiB/sec (4357.70 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 4155.83 4095.85 4134.27 27.23 60.21318 0 32 4 3 0 0 1 0 0 1 40265318400 1048576 261622857728 MPIIO 0
Finished: Wed Oct 24 04:29:56 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 40265318400 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
+ tee -a ./output/COUNT:1#NN:8#PPN:4#API:MPIIO#T:1048576.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 04:30:13 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 40265318400 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 04:30:13 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 32 (4 per node)
repetitions = 3
xfersize = 1 MiB
blocksize = 37.50 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 73459040256.
read 1161.48 39321600 1024.00 0.183081 60.14 0.008206 60.32 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 75210162176.
read 1193.01 39321600 1024.00 0.009586 60.12 0.008296 60.12 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 76188483584.
read 1208.65 39321600 1024.00 0.009621 60.11 0.008259 60.12 2
Max Read: 1208.65 MiB/sec (1267.36 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 1208.65 1161.48 1187.71 19.62 60.18458 0 32 4 3 0 0 1 0 0 1 40265318400 1048576 73459040256 MPIIO 0
Finished: Wed Oct 24 04:33:13 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12515603 0xbef913 0
7 12464228 0xbe3064 0
1 12645515 0xc0f48b 0
3 12493813 0xbea3f5 0
5 12385880 0xbcfe58 0
6 12599126 0xc03f56 0
0 12427531 0xbda10b 0
2 12573871 0xbfdcaf 0

View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 40265318400 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:4#API:MPIIO#T:10485760.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 03:19:42 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 40265318400 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 03:19:42 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 32 (4 per node)
repetitions = 3
xfersize = 10 MiB
blocksize = 37.50 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288458731520.
WARNING: Using actual aggregate bytes moved = 174157987840.
write 2753.77 39321600 10240 0.010886 60.30 0.000964 60.31 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288437760000.
WARNING: Using actual aggregate bytes moved = 176737484800.
write 2792.94 39321600 10240 0.081348 60.27 0.000673 60.35 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288427274240.
WARNING: Using actual aggregate bytes moved = 171987435520.
write 2726.40 39321600 10240 0.026415 60.13 0.000841 60.16 2
Max Write: 2792.94 MiB/sec (2928.61 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 2792.94 2726.40 2757.70 27.31 60.27411 0 32 4 3 0 0 1 0 0 1 40265318400 10485760 174157987840 MPIIO 0
Finished: Wed Oct 24 03:22:49 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ tee -a ./output/COUNT:1#NN:8#PPN:4#API:MPIIO#T:10485760.txt
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 40265318400 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 03:23:06 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 40265318400 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 03:23:06 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 32 (4 per node)
repetitions = 3
xfersize = 10 MiB
blocksize = 37.50 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 174000701440.
read 2754.27 39321600 10240 0.143731 60.11 0.004690 60.25 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 177880432640.
read 2821.21 39321600 10240 0.005724 60.13 0.004749 60.13 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 180124385280.
read 2858.10 39321600 10240 0.006035 60.10 0.004731 60.10 2
Max Read: 2858.10 MiB/sec (2996.94 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 2858.10 2754.27 2811.19 42.98 60.16048 0 32 4 3 0 0 1 0 0 1 40265318400 10485760 174000701440 MPIIO 0
Finished: Wed Oct 24 03:26:07 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12515092 0xbef714 0
7 12463713 0xbe2e61 0
1 12645000 0xc0f288 0
3 12493301 0xbea1f5 0
5 12385364 0xbcfc54 0
6 12598615 0xc03d57 0
0 12427019 0xbd9f0b 0
2 12573356 0xbfdaac 0

View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 40265318400 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:4#API:MPIIO#T:16384.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 06:48:20 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 40265318400 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 06:48:20 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 32 (4 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 37.50 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288482373632.
WARNING: Using actual aggregate bytes moved = 4082876416.
write 60.46 39321600 16.00 0.012821 64.39 0.000421 64.41 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288484339712.
WARNING: Using actual aggregate bytes moved = 4046667776.
write 60.03 39321600 16.00 0.001576 64.29 0.000718 64.29 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288487600128.
WARNING: Using actual aggregate bytes moved = 4043341824.
write 59.99 39321600 16.00 0.002535 64.27 0.001013 64.27 2
Max Write: 60.46 MiB/sec (63.39 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 60.46 59.99 60.16 0.21 64.32336 0 32 4 3 0 0 1 0 0 1 40265318400 16384 4082876416 MPIIO 0
Finished: Wed Oct 24 06:51:33 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ tee -a ./output/COUNT:1#NN:8#PPN:4#API:MPIIO#T:16384.txt
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 40265318400 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 06:51:37 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 40265318400 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 06:51:37 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 32 (4 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 37.50 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 3402989568.
read 50.52 39321600 16.00 0.007093 64.23 0.000988 64.24 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 3412885504.
read 50.68 39321600 16.00 0.001842 64.22 0.000982 64.22 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 3433807872.
read 50.99 39321600 16.00 0.002303 64.22 0.000941 64.22 2
Max Read: 50.99 MiB/sec (53.47 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 50.99 50.52 50.73 0.20 64.22841 0 32 4 3 0 0 1 0 0 1 40265318400 16384 3402989568 MPIIO 0
Finished: Wed Oct 24 06:54:50 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428552 0xbda508 0
2 12574900 0xbfe0b4 0
4 12516626 0xbefd12 0
7 12465256 0xbe3468 0
1 12646543 0xc0f88f 0
3 12494843 0xbea7fb 0
5 12386905 0xbd0259 0
6 12600146 0xc04352 0

View File

@ -0,0 +1,600 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 40265318400 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:4#API:POSIX#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 05:27:34 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 40265318400 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 05:27:34 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 32 (4 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 37.50 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288460697600.
WARNING: Using actual aggregate bytes moved = 85053952000.
write 899.15 39321600 100.00 0.001971 90.21 0.000328 90.21 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288355840000.
WARNING: Using actual aggregate bytes moved = 75213209600.
write 759.79 39321600 100.00 0.002503 94.40 0.000411 94.41 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288453222400.
WARNING: Using actual aggregate bytes moved = 76511436800.
write 796.06 39321600 100.00 0.008759 91.65 0.000430 91.66 2
Max Write: 899.15 MiB/sec (942.83 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 899.15 759.79 818.33 59.04 92.09254 0 32 4 3 1 0 1 0 0 1 40265318400 102400 85053952000 POSIX 0
Finished: Wed Oct 24 05:32:12 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 40265318400 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
+ tee -a ./output/COUNT:1#NN:8#PPN:4#API:POSIX#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 05:32:26 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 40265318400 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 05:32:26 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/indread8/file
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 32 (4 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 37.50 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 5153960755200.
WARNING: Using actual aggregate bytes moved = 24710860800.
read 392.53 39321600 100.00 0.001527 60.03 0.000924 60.04 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 5153960755200.
WARNING: Using actual aggregate bytes moved = 25415168000.
read 403.74 39321600 100.00 0.001171 60.03 0.000952 60.03 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 5153960755200.
WARNING: Using actual aggregate bytes moved = 25638297600.
read 407.24 39321600 100.00 0.000996 60.04 0.000961 60.04 2
Max Read: 407.24 MiB/sec (427.02 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 407.24 392.53 401.17 6.27 60.03603 0 32 4 3 1 0 1 0 0 1 40265318400 102400 24710860800 POSIX 0
Finished: Wed Oct 24 05:35:26 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write.00000006
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428022 0xbda2f6 0
2 12574368 0xbfdea0 0
4 12516100 0xbefb04 0
5 12386380 0xbd004c 0
7 12464727 0xbe3257 0
1 12646011 0xc0f67b 0
3 12494309 0xbea5e5 0
6 12599623 0xc04147 0
/esfs/jtacquaviva/ioperf/file_write.00000023
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12574370 0xbfdea2 0
4 12516101 0xbefb05 0
7 12464728 0xbe3258 0
1 12646012 0xc0f67c 0
3 12494310 0xbea5e6 0
5 12386381 0xbd004d 0
6 12599624 0xc04148 0
0 12428025 0xbda2f9 0
/esfs/jtacquaviva/ioperf/file_write.00000002
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464714 0xbe324a 0
1 12645997 0xc0f66d 0
3 12494297 0xbea5d9 0
5 12386367 0xbd003f 0
2 12574358 0xbfde96 0
0 12428012 0xbda2ec 0
6 12599610 0xc0413a 0
4 12516089 0xbefaf9 0
/esfs/jtacquaviva/ioperf/file_write.00000026
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494317 0xbea5ed 0
5 12386390 0xbd0056 0
7 12464737 0xbe3261 0
6 12599633 0xc04151 0
0 12428034 0xbda302 0
2 12574380 0xbfdeac 0
4 12516111 0xbefb0f 0
1 12646020 0xc0f684 0
/esfs/jtacquaviva/ioperf/file_write.00000000
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12646000 0xc0f670 0
2 12574359 0xbfde97 0
3 12494299 0xbea5db 0
0 12428014 0xbda2ee 0
5 12386369 0xbd0041 0
7 12464717 0xbe324d 0
4 12516090 0xbefafa 0
6 12599612 0xc0413c 0
/esfs/jtacquaviva/ioperf/file_write.00000020
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12386388 0xbd0054 0
0 12428031 0xbda2ff 0
3 12494318 0xbea5ee 0
2 12574378 0xbfdeaa 0
7 12464738 0xbe3262 0
6 12599634 0xc04152 0
4 12516110 0xbefb0e 0
1 12646021 0xc0f685 0
/esfs/jtacquaviva/ioperf/file_write.00000030
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12574353 0xbfde91 0
1 12645994 0xc0f66a 0
5 12386363 0xbd003b 0
0 12428008 0xbda2e8 0
4 12516084 0xbefaf4 0
3 12494294 0xbea5d6 0
6 12599607 0xc04137 0
7 12464712 0xbe3248 0
/esfs/jtacquaviva/ioperf/file_write.00000004
lmm_stripe_count: 6
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464719 0xbe324f 0
6 12599614 0xc0413e 0
5 12386372 0xbd0044 0
0 12428017 0xbda2f1 0
1 12646003 0xc0f673 0
2 12574362 0xbfde9a 0
/esfs/jtacquaviva/ioperf/file_write.00000007
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464711 0xbe3247 0
3 12494293 0xbea5d5 0
6 12599606 0xc04136 0
2 12574354 0xbfde92 0
1 12645995 0xc0f66b 0
5 12386364 0xbd003c 0
0 12428009 0xbda2e9 0
4 12516085 0xbefaf5 0
/esfs/jtacquaviva/ioperf/file_write.00000018
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12646017 0xc0f681 0
3 12494315 0xbea5eb 0
5 12386386 0xbd0052 0
6 12599629 0xc0414d 0
0 12428029 0xbda2fd 0
2 12574376 0xbfdea8 0
4 12516106 0xbefb0a 0
7 12464735 0xbe325f 0
/esfs/jtacquaviva/ioperf/file_write.00000031
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428024 0xbda2f8 0
2 12574372 0xbfdea4 0
4 12516102 0xbefb06 0
7 12464730 0xbe325a 0
1 12646013 0xc0f67d 0
3 12494311 0xbea5e7 0
5 12386382 0xbd004e 0
6 12599625 0xc04149 0
/esfs/jtacquaviva/ioperf/file_write.00000012
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12574356 0xbfde94 0
4 12516087 0xbefaf7 0
5 12386368 0xbd0040 0
1 12645999 0xc0f66f 0
3 12494298 0xbea5da 0
0 12428013 0xbda2ed 0
7 12464716 0xbe324c 0
6 12599611 0xc0413b 0
/esfs/jtacquaviva/ioperf/file_write.00000013
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12599635 0xc04153 0
0 12428035 0xbda303 0
2 12574382 0xbfdeae 0
4 12516112 0xbefb10 0
7 12464740 0xbe3264 0
1 12646023 0xc0f687 0
3 12494321 0xbea5f1 0
5 12386392 0xbd0058 0
/esfs/jtacquaviva/ioperf/file_write.00000025
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12645996 0xc0f66c 0
3 12494295 0xbea5d7 0
5 12386365 0xbd003d 0
6 12599609 0xc04139 0
0 12428010 0xbda2ea 0
7 12464713 0xbe3249 0
4 12516086 0xbefaf6 0
2 12574357 0xbfde95 0
/esfs/jtacquaviva/ioperf/file_write.00000022
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12516093 0xbefafd 0
3 12494302 0xbea5de 0
7 12464722 0xbe3252 0
6 12599616 0xc04140 0
2 12574363 0xbfde9b 0
5 12386375 0xbd0047 0
1 12646006 0xc0f676 0
/esfs/jtacquaviva/ioperf/file_write.00000003
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464733 0xbe325d 0
1 12646016 0xc0f680 0
3 12494314 0xbea5ea 0
5 12386385 0xbd0051 0
6 12599628 0xc0414c 0
0 12428028 0xbda2fc 0
2 12574375 0xbfdea7 0
4 12516105 0xbefb09 0
/esfs/jtacquaviva/ioperf/file_write.00000008
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12574373 0xbfdea5 0
4 12516103 0xbefb07 0
7 12464731 0xbe325b 0
1 12646014 0xc0f67e 0
3 12494312 0xbea5e8 0
5 12386383 0xbd004f 0
6 12599626 0xc0414a 0
0 12428026 0xbda2fa 0
/esfs/jtacquaviva/ioperf/file_write.00000028
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12516107 0xbefb0b 0
7 12464736 0xbe3260 0
6 12599631 0xc0414f 0
1 12646019 0xc0f683 0
5 12386391 0xbd0057 0
0 12428033 0xbda301 0
3 12494319 0xbea5ef 0
2 12574381 0xbfdead 0
/esfs/jtacquaviva/ioperf/file_write.00000010
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428006 0xbda2e6 0
4 12516082 0xbefaf2 0
3 12494291 0xbea5d3 0
2 12574352 0xbfde90 0
7 12464710 0xbe3246 0
1 12645993 0xc0f669 0
5 12386362 0xbd003a 0
6 12599605 0xc04135 0
/esfs/jtacquaviva/ioperf/file_write.00000017
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428015 0xbda2ef 0
5 12386371 0xbd0043 0
7 12464718 0xbe324e 0
1 12646002 0xc0f672 0
2 12574361 0xbfde99 0
4 12516091 0xbefafb 0
3 12494301 0xbea5dd 0
6 12599615 0xc0413f 0
/esfs/jtacquaviva/ioperf/file_write.00000029
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12516094 0xbefafe 0
5 12386373 0xbd0045 0
6 12599617 0xc04141 0
1 12646005 0xc0f675 0
2 12574364 0xbfde9c 0
0 12428018 0xbda2f2 0
3 12494304 0xbea5e0 0
7 12464723 0xbe3253 0
/esfs/jtacquaviva/ioperf/file_write.00000016
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12516098 0xbefb02 0
7 12464726 0xbe3256 0
5 12386378 0xbd004a 0
2 12574369 0xbfdea1 0
1 12646009 0xc0f679 0
6 12599621 0xc04145 0
3 12494308 0xbea5e4 0
/esfs/jtacquaviva/ioperf/file_write.00000005
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464734 0xbe325e 0
1 12646018 0xc0f682 0
3 12494316 0xbea5ec 0
5 12386387 0xbd0053 0
6 12599630 0xc0414e 0
0 12428030 0xbda2fe 0
2 12574377 0xbfdea9 0
4 12516108 0xbefb0c 0
/esfs/jtacquaviva/ioperf/file_write.00000014
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428020 0xbda2f4 0
2 12574366 0xbfde9e 0
4 12516096 0xbefb00 0
7 12464724 0xbe3254 0
1 12646007 0xc0f677 0
3 12494305 0xbea5e1 0
5 12386376 0xbd0048 0
6 12599619 0xc04143 0
/esfs/jtacquaviva/ioperf/file_write.00000021
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12386361 0xbd0039 0
2 12574351 0xbfde8f 0
7 12464709 0xbe3245 0
1 12645992 0xc0f668 0
6 12599604 0xc04134 0
0 12428007 0xbda2e7 0
4 12516083 0xbefaf3 0
3 12494292 0xbea5d4 0
/esfs/jtacquaviva/ioperf/file_write.00000019
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12599608 0xc04138 0
2 12574355 0xbfde93 0
3 12494296 0xbea5d8 0
7 12464715 0xbe324b 0
5 12386366 0xbd003e 0
1 12645998 0xc0f66e 0
4 12516088 0xbefaf8 0
0 12428011 0xbda2eb 0
/esfs/jtacquaviva/ioperf/file_write.00000009
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464721 0xbe3251 0
1 12646004 0xc0f674 0
5 12386374 0xbd0046 0
6 12599618 0xc04142 0
4 12516095 0xbefaff 0
3 12494303 0xbea5df 0
0 12428019 0xbda2f3 0
2 12574365 0xbfde9d 0
/esfs/jtacquaviva/ioperf/file_write.00000015
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12516104 0xbefb08 0
7 12464732 0xbe325c 0
1 12646015 0xc0f67f 0
3 12494313 0xbea5e9 0
5 12386384 0xbd0050 0
6 12599627 0xc0414b 0
0 12428027 0xbda2fb 0
2 12574374 0xbfdea6 0
/esfs/jtacquaviva/ioperf/file_write.00000027
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12386370 0xbd0042 0
1 12646001 0xc0f671 0
6 12599613 0xc0413d 0
3 12494300 0xbea5dc 0
2 12574360 0xbfde98 0
7 12464720 0xbe3250 0
0 12428016 0xbda2f0 0
4 12516092 0xbefafc 0
/esfs/jtacquaviva/ioperf/file_write.00000024
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494307 0xbea5e3 0
4 12516099 0xbefb03 0
5 12386379 0xbd004b 0
0 12428023 0xbda2f7 0
1 12646010 0xc0f67a 0
2 12574371 0xbfdea3 0
7 12464729 0xbe3259 0
6 12599622 0xc04146 0
/esfs/jtacquaviva/ioperf/file_write.00000001
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12574367 0xbfde9f 0
4 12516097 0xbefb01 0
7 12464725 0xbe3255 0
1 12646008 0xc0f678 0
3 12494306 0xbea5e2 0
5 12386377 0xbd0049 0
6 12599620 0xc04144 0
0 12428021 0xbda2f5 0
/esfs/jtacquaviva/ioperf/file_write.00000011
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12386389 0xbd0055 0
6 12599632 0xc04150 0
0 12428032 0xbda300 0
2 12574379 0xbfdeab 0
4 12516109 0xbefb0d 0
7 12464739 0xbe3263 0
1 12646022 0xc0f686 0
3 12494320 0xbea5f0 0

View File

@ -0,0 +1,602 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 40265318400 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:4#API:POSIX#T:1048576.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 04:19:56 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 40265318400 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 04:19:56 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 32 (4 per node)
repetitions = 3
xfersize = 1 MiB
blocksize = 37.50 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288410497024.
WARNING: Using actual aggregate bytes moved = 351248842752.
write 4842 39321600 1024.00 0.002099 69.18 0.000774 69.18 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288187150336.
WARNING: Using actual aggregate bytes moved = 352358236160.
write 4855 39321600 1024.00 0.008233 69.21 0.000361 69.22 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288462925824.
WARNING: Using actual aggregate bytes moved = 346617282560.
write 4787 39321600 1024.00 0.001639 69.06 0.000374 69.06 2
Max Write: 4854.87 MiB/sec (5090.70 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 4854.87 4786.51 4827.75 29.64 69.15340 0 32 4 3 1 0 1 0 0 1 40265318400 1048576 351248842752 POSIX 0
Finished: Wed Oct 24 04:23:30 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ tee -a ./output/COUNT:1#NN:8#PPN:4#API:POSIX#T:1048576.txt
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 40265318400 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 04:23:47 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 40265318400 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 04:23:47 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/indread8/file
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 32 (4 per node)
repetitions = 3
xfersize = 1 MiB
blocksize = 37.50 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 5153960755200.
WARNING: Using actual aggregate bytes moved = 151530766336.
read 2407.47 39321600 1024.00 0.001461 60.02 0.000884 60.03 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 5153960755200.
WARNING: Using actual aggregate bytes moved = 172089147392.
read 2734.60 39321600 1024.00 0.000905 60.01 0.001178 60.01 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 5153960755200.
WARNING: Using actual aggregate bytes moved = 189633921024.
read 3013.02 39321600 1024.00 0.000915 60.02 0.000907 60.02 2
Max Read: 3013.02 MiB/sec (3159.38 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 3013.02 2407.47 2718.37 247.48 60.02112 0 32 4 3 1 0 1 0 0 1 40265318400 1048576 151530766336 POSIX 0
Finished: Wed Oct 24 04:26:47 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write.00000006
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573838 0xbfdc8e 0
1 12645483 0xc0f46b 0
6 12599095 0xc03f37 0
5 12385849 0xbcfe39 0
4 12515573 0xbef8f5 0
3 12493784 0xbea3d8 0
7 12464200 0xbe3048 0
/esfs/jtacquaviva/ioperf/file_write.00000023
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12515582 0xbef8fe 0
1 12645495 0xc0f477 0
5 12385858 0xbcfe42 0
0 12427512 0xbda0f8 0
7 12464209 0xbe3051 0
3 12493794 0xbea3e2 0
6 12599107 0xc03f43 0
2 12573852 0xbfdc9c 0
/esfs/jtacquaviva/ioperf/file_write.00000002
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12645498 0xc0f47a 0
3 12493796 0xbea3e4 0
5 12385862 0xbcfe46 0
6 12599109 0xc03f45 0
0 12427514 0xbda0fa 0
2 12573854 0xbfdc9e 0
4 12515586 0xbef902 0
7 12464211 0xbe3053 0
/esfs/jtacquaviva/ioperf/file_write.00000026
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573845 0xbfdc95 0
4 12515577 0xbef8f9 0
7 12464202 0xbe304a 0
1 12645489 0xc0f471 0
3 12493787 0xbea3db 0
5 12385853 0xbcfe3d 0
6 12599100 0xc03f3c 0
0 12427505 0xbda0f1 0
/esfs/jtacquaviva/ioperf/file_write.00000000
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12427522 0xbda102 0
2 12573862 0xbfdca6 0
4 12515594 0xbef90a 0
7 12464219 0xbe305b 0
1 12645506 0xc0f482 0
3 12493804 0xbea3ec 0
5 12385870 0xbcfe4e 0
6 12599117 0xc03f4d 0
/esfs/jtacquaviva/ioperf/file_write.00000020
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464223 0xbe305f 0
1 12645510 0xc0f486 0
5 12385875 0xbcfe53 0
0 12427526 0xbda106 0
4 12515598 0xbef90e 0
3 12493809 0xbea3f1 0
6 12599122 0xc03f52 0
2 12573867 0xbfdcab 0
/esfs/jtacquaviva/ioperf/file_write.00000030
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464196 0xbe3044 0
5 12385847 0xbcfe37 0
3 12493781 0xbea3d5 0
2 12573843 0xbfdc93 0
1 12645485 0xc0f46d 0
6 12599097 0xc03f39 0
4 12515574 0xbef8f6 0
0 12427504 0xbda0f0 0
/esfs/jtacquaviva/ioperf/file_write.00000004
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464204 0xbe304c 0
1 12645491 0xc0f473 0
3 12493789 0xbea3dd 0
5 12385855 0xbcfe3f 0
6 12599102 0xc03f3e 0
0 12427507 0xbda0f3 0
2 12573847 0xbfdc97 0
4 12515579 0xbef8fb 0
/esfs/jtacquaviva/ioperf/file_write.00000007
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12493797 0xbea3e5 0
5 12385863 0xbcfe47 0
6 12599110 0xc03f46 0
0 12427515 0xbda0fb 0
2 12573855 0xbfdc9f 0
4 12515587 0xbef903 0
7 12464212 0xbe3054 0
1 12645499 0xc0f47b 0
/esfs/jtacquaviva/ioperf/file_write.00000018
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12599104 0xc03f40 0
4 12515580 0xbef8fc 0
3 12493792 0xbea3e0 0
0 12427511 0xbda0f7 0
1 12645494 0xc0f476 0
2 12573850 0xbfdc9a 0
7 12464208 0xbe3050 0
5 12385859 0xbcfe43 0
/esfs/jtacquaviva/ioperf/file_write.00000031
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12385873 0xbcfe51 0
3 12493808 0xbea3f0 0
6 12599121 0xc03f51 0
2 12573866 0xbfdcaa 0
7 12464224 0xbe3060 0
0 12427527 0xbda107 0
4 12515599 0xbef90f 0
1 12645511 0xc0f487 0
/esfs/jtacquaviva/ioperf/file_write.00000012
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12385864 0xbcfe48 0
6 12599111 0xc03f47 0
0 12427516 0xbda0fc 0
2 12573856 0xbfdca0 0
4 12515588 0xbef904 0
7 12464213 0xbe3055 0
1 12645500 0xc0f47c 0
3 12493798 0xbea3e6 0
/esfs/jtacquaviva/ioperf/file_write.00000013
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12385846 0xbcfe36 0
6 12599094 0xc03f36 0
4 12515571 0xbef8f3 0
2 12573840 0xbfdc90 0
3 12493782 0xbea3d6 0
1 12645484 0xc0f46c 0
0 12427502 0xbda0ee 0
7 12464199 0xbe3047 0
/esfs/jtacquaviva/ioperf/file_write.00000025
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12599116 0xc03f4c 0
0 12427521 0xbda101 0
2 12573861 0xbfdca5 0
4 12515593 0xbef909 0
7 12464218 0xbe305a 0
1 12645505 0xc0f481 0
3 12493803 0xbea3eb 0
5 12385869 0xbcfe4d 0
/esfs/jtacquaviva/ioperf/file_write.00000022
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12427518 0xbda0fe 0
2 12573858 0xbfdca2 0
4 12515590 0xbef906 0
7 12464215 0xbe3057 0
1 12645502 0xc0f47e 0
3 12493800 0xbea3e8 0
5 12385866 0xbcfe4a 0
6 12599113 0xc03f49 0
/esfs/jtacquaviva/ioperf/file_write.00000003
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464194 0xbe3042 0
6 12599093 0xc03f35 0
3 12493780 0xbea3d4 0
1 12645482 0xc0f46a 0
0 12427498 0xbda0ea 0
2 12573839 0xbfdc8f 0
5 12385848 0xbcfe38 0
4 12515572 0xbef8f4 0
/esfs/jtacquaviva/ioperf/file_write.00000008
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12493791 0xbea3df 0
0 12427509 0xbda0f5 0
2 12573849 0xbfdc99 0
7 12464206 0xbe304e 0
1 12645493 0xc0f475 0
5 12385857 0xbcfe41 0
6 12599105 0xc03f41 0
4 12515583 0xbef8ff 0
/esfs/jtacquaviva/ioperf/file_write.00000028
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573859 0xbfdca3 0
4 12515591 0xbef907 0
7 12464216 0xbe3058 0
1 12645503 0xc0f47f 0
6 12599114 0xc03f4a 0
0 12427519 0xbda0ff 0
3 12493801 0xbea3e9 0
5 12385868 0xbcfe4c 0
/esfs/jtacquaviva/ioperf/file_write.00000010
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573841 0xbfdc91 0
6 12599096 0xc03f38 0
0 12427501 0xbda0ed 0
7 12464198 0xbe3046 0
5 12385850 0xbcfe3a 0
1 12645488 0xc0f470 0
3 12493786 0xbea3da 0
/esfs/jtacquaviva/ioperf/file_write.00000017
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12599112 0xc03f48 0
0 12427517 0xbda0fd 0
2 12573857 0xbfdca1 0
4 12515589 0xbef905 0
7 12464214 0xbe3056 0
1 12645501 0xc0f47d 0
3 12493799 0xbea3e7 0
5 12385865 0xbcfe49 0
/esfs/jtacquaviva/ioperf/file_write.00000029
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12427510 0xbda0f6 0
7 12464207 0xbe304f 0
3 12493793 0xbea3e1 0
6 12599106 0xc03f42 0
2 12573851 0xbfdc9b 0
5 12385860 0xbcfe44 0
1 12645496 0xc0f478 0
4 12515584 0xbef900 0
/esfs/jtacquaviva/ioperf/file_write.00000016
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573842 0xbfdc92 0
7 12464197 0xbe3045 0
1 12645486 0xc0f46e 0
0 12427503 0xbda0ef 0
3 12493785 0xbea3d9 0
5 12385851 0xbcfe3b 0
6 12599098 0xc03f3a 0
4 12515575 0xbef8f7 0
/esfs/jtacquaviva/ioperf/file_write.00000005
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573863 0xbfdca7 0
4 12515595 0xbef90b 0
7 12464220 0xbe305c 0
1 12645507 0xc0f483 0
3 12493805 0xbea3ed 0
5 12385871 0xbcfe4f 0
6 12599118 0xc03f4e 0
0 12427523 0xbda103 0
/esfs/jtacquaviva/ioperf/file_write.00000014
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12515596 0xbef90c 0
7 12464221 0xbe305d 0
1 12645508 0xc0f484 0
3 12493806 0xbea3ee 0
5 12385872 0xbcfe50 0
6 12599119 0xc03f4f 0
0 12427524 0xbda104 0
2 12573864 0xbfdca8 0
/esfs/jtacquaviva/ioperf/file_write.00000021
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12493783 0xbea3d7 0
0 12427500 0xbda0ec 0
1 12645487 0xc0f46f 0
7 12464201 0xbe3049 0
2 12573844 0xbfdc94 0
5 12385852 0xbcfe3c 0
4 12515576 0xbef8f8 0
6 12599099 0xc03f3b 0
/esfs/jtacquaviva/ioperf/file_write.00000019
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464222 0xbe305e 0
1 12645509 0xc0f485 0
3 12493807 0xbea3ef 0
5 12385874 0xbcfe52 0
6 12599120 0xc03f50 0
0 12427525 0xbda105 0
2 12573865 0xbfdca9 0
4 12515597 0xbef90d 0
/esfs/jtacquaviva/ioperf/file_write.00000009
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12385876 0xbcfe54 0
6 12599123 0xc03f53 0
0 12427528 0xbda108 0
2 12573868 0xbfdcac 0
4 12515600 0xbef910 0
7 12464225 0xbe3061 0
1 12645512 0xc0f488 0
3 12493810 0xbea3f2 0
/esfs/jtacquaviva/ioperf/file_write.00000015
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12645492 0xc0f474 0
3 12493790 0xbea3de 0
5 12385856 0xbcfe40 0
6 12599103 0xc03f3f 0
0 12427508 0xbda0f4 0
2 12573848 0xbfdc98 0
4 12515581 0xbef8fd 0
7 12464205 0xbe304d 0
/esfs/jtacquaviva/ioperf/file_write.00000027
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464210 0xbe3052 0
1 12645497 0xc0f479 0
3 12493795 0xbea3e3 0
5 12385861 0xbcfe45 0
6 12599108 0xc03f44 0
0 12427513 0xbda0f9 0
2 12573853 0xbfdc9d 0
4 12515585 0xbef901 0
/esfs/jtacquaviva/ioperf/file_write.00000024
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12645481 0xc0f469 0
6 12599092 0xc03f34 0
5 12385845 0xbcfe35 0
3 12493779 0xbea3d3 0
2 12573837 0xbfdc8d 0
7 12464195 0xbe3043 0
4 12515570 0xbef8f2 0
0 12427499 0xbda0eb 0
/esfs/jtacquaviva/ioperf/file_write.00000001
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12515578 0xbef8fa 0
7 12464203 0xbe304b 0
1 12645490 0xc0f472 0
3 12493788 0xbea3dc 0
5 12385854 0xbcfe3e 0
6 12599101 0xc03f3d 0
0 12427506 0xbda0f2 0
2 12573846 0xbfdc96 0
/esfs/jtacquaviva/ioperf/file_write.00000011
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12385867 0xbcfe4b 0
6 12599115 0xc03f4b 0
7 12464217 0xbe3059 0
1 12645504 0xc0f480 0
0 12427520 0xbda100 0
2 12573860 0xbfdca4 0
4 12515592 0xbef908 0
3 12493802 0xbea3ea 0

View File

@ -0,0 +1,599 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 40265318400 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:4#API:POSIX#T:10485760.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 03:12:53 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 40265318400 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 03:12:53 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 32 (4 per node)
repetitions = 3
xfersize = 10 MiB
blocksize = 37.50 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288154644480.
WARNING: Using actual aggregate bytes moved = 396697272320.
write 5553 39321600 10240 0.001622 68.12 0.000543 68.13 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1284966973440.
WARNING: Using actual aggregate bytes moved = 391297105920.
write 5525 39321600 10240 0.009448 67.54 0.000672 67.55 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1286602752000.
WARNING: Using actual aggregate bytes moved = 391611678720.
write 5503 39321600 10240 0.015847 67.86 0.000832 67.87 2
Max Write: 5553.20 MiB/sec (5822.95 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 5553.20 5502.54 5526.77 20.74 67.84871 0 32 4 3 1 0 1 0 0 1 40265318400 10485760 396697272320 POSIX 0
Finished: Wed Oct 24 03:16:22 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 40265318400 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
+ tee -a ./output/COUNT:1#NN:8#PPN:4#API:POSIX#T:10485760.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 03:16:39 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 40265318400 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 03:16:39 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/indread8/file
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 32 (4 per node)
repetitions = 3
xfersize = 10 MiB
blocksize = 37.50 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 5153960755200.
WARNING: Using actual aggregate bytes moved = 169198223360.
read 2681.48 39321600 10240 0.001316 60.17 0.000816 60.18 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 5153960755200.
WARNING: Using actual aggregate bytes moved = 191574835200.
read 3033.32 39321600 10240 0.000779 60.23 0.001073 60.23 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 5153960755200.
WARNING: Using actual aggregate bytes moved = 211235635200.
read 3351.82 39321600 10240 0.000796 60.10 0.000797 60.10 2
Max Read: 3351.82 MiB/sec (3514.63 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 3351.82 2681.48 3022.21 273.78 60.16949 0 32 4 3 1 0 1 0 0 1 40265318400 10485760 169198223360 POSIX 0
Finished: Wed Oct 24 03:19:40 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write.00000006
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12493276 0xbea1dc 0
4 12515068 0xbef6fc 0
5 12385340 0xbcfc3c 0
7 12463689 0xbe2e49 0
1 12644976 0xc0f270 0
6 12598591 0xc03d3f 0
0 12426995 0xbd9ef3 0
2 12573333 0xbfda95 0
/esfs/jtacquaviva/ioperf/file_write.00000023
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12515059 0xbef6f3 0
3 12493267 0xbea1d3 0
2 12573324 0xbfda8c 0
6 12598585 0xc03d39 0
5 12385335 0xbcfc37 0
7 12463684 0xbe2e44 0
1 12644972 0xc0f26c 0
0 12426991 0xbd9eef 0
/esfs/jtacquaviva/ioperf/file_write.00000002
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12463704 0xbe2e58 0
1 12644991 0xc0f27f 0
3 12493293 0xbea1ed 0
5 12385355 0xbcfc4b 0
6 12598607 0xc03d4f 0
0 12427011 0xbd9f03 0
2 12573348 0xbfdaa4 0
4 12515084 0xbef70c 0
/esfs/jtacquaviva/ioperf/file_write.00000026
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12426986 0xbd9eea 0
4 12515061 0xbef6f5 0
5 12385334 0xbcfc36 0
1 12644969 0xc0f269 0
3 12493272 0xbea1d8 0
7 12463682 0xbe2e42 0
2 12573328 0xbfda90 0
/esfs/jtacquaviva/ioperf/file_write.00000000
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12463695 0xbe2e4f 0
4 12515073 0xbef701 0
6 12598598 0xc03d46 0
2 12573339 0xbfda9b 0
1 12644983 0xc0f277 0
3 12493287 0xbea1e7 0
5 12385350 0xbcfc46 0
0 12427006 0xbd9efe 0
/esfs/jtacquaviva/ioperf/file_write.00000020
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12598589 0xc03d3d 0
7 12463687 0xbe2e47 0
0 12426993 0xbd9ef1 0
3 12493275 0xbea1db 0
5 12385338 0xbcfc3a 0
2 12573330 0xbfda92 0
1 12644974 0xc0f26e 0
/esfs/jtacquaviva/ioperf/file_write.00000030
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12493297 0xbea1f1 0
5 12385359 0xbcfc4f 0
6 12598611 0xc03d53 0
0 12427015 0xbd9f07 0
2 12573352 0xbfdaa8 0
4 12515088 0xbef710 0
7 12463709 0xbe2e5d 0
1 12644996 0xc0f284 0
/esfs/jtacquaviva/ioperf/file_write.00000004
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12385332 0xbcfc34 0
6 12598583 0xc03d37 0
7 12463680 0xbe2e40 0
1 12644968 0xc0f268 0
3 12493269 0xbea1d5 0
2 12573327 0xbfda8f 0
4 12515064 0xbef6f8 0
0 12426990 0xbd9eee 0
/esfs/jtacquaviva/ioperf/file_write.00000007
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12644993 0xc0f281 0
3 12493294 0xbea1ee 0
0 12427012 0xbd9f04 0
2 12573349 0xbfdaa5 0
4 12515085 0xbef70d 0
5 12385356 0xbcfc4c 0
6 12598609 0xc03d51 0
7 12463707 0xbe2e5b 0
/esfs/jtacquaviva/ioperf/file_write.00000018
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12515069 0xbef6fd 0
5 12385342 0xbcfc3e 0
2 12573334 0xbfda96 0
3 12493280 0xbea1e0 0
7 12463693 0xbe2e4d 0
6 12598594 0xc03d42 0
0 12426999 0xbd9ef7 0
1 12644980 0xc0f274 0
/esfs/jtacquaviva/ioperf/file_write.00000031
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12644995 0xc0f283 0
3 12493296 0xbea1f0 0
5 12385358 0xbcfc4e 0
6 12598610 0xc03d52 0
0 12427014 0xbd9f06 0
2 12573351 0xbfdaa7 0
4 12515087 0xbef70f 0
7 12463708 0xbe2e5c 0
/esfs/jtacquaviva/ioperf/file_write.00000012
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12598590 0xc03d3e 0
4 12515067 0xbef6fb 0
5 12385339 0xbcfc3b 0
1 12644975 0xc0f26f 0
7 12463688 0xbe2e48 0
0 12426994 0xbd9ef2 0
3 12493277 0xbea1dd 0
2 12573331 0xbfda93 0
/esfs/jtacquaviva/ioperf/file_write.00000013
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12515082 0xbef70a 0
7 12463705 0xbe2e59 0
1 12644992 0xc0f280 0
3 12493292 0xbea1ec 0
5 12385354 0xbcfc4a 0
6 12598606 0xc03d4e 0
0 12427010 0xbd9f02 0
2 12573347 0xbfdaa3 0
/esfs/jtacquaviva/ioperf/file_write.00000025
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12493285 0xbea1e5 0
5 12385348 0xbcfc44 0
6 12598602 0xc03d4a 0
0 12427005 0xbd9efd 0
2 12573343 0xbfda9f 0
4 12515079 0xbef707 0
7 12463700 0xbe2e54 0
1 12644987 0xc0f27b 0
/esfs/jtacquaviva/ioperf/file_write.00000022
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12463694 0xbe2e4e 0
5 12385344 0xbcfc40 0
1 12644982 0xc0f276 0
6 12598597 0xc03d45 0
3 12493286 0xbea1e6 0
0 12427002 0xbd9efa 0
2 12573340 0xbfda9c 0
4 12515077 0xbef705 0
/esfs/jtacquaviva/ioperf/file_write.00000003
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573332 0xbfda94 0
1 12644977 0xc0f271 0
4 12515070 0xbef6fe 0
0 12426997 0xbd9ef5 0
3 12493279 0xbea1df 0
6 12598592 0xc03d40 0
7 12463692 0xbe2e4c 0
/esfs/jtacquaviva/ioperf/file_write.00000008
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573345 0xbfdaa1 0
4 12515081 0xbef709 0
7 12463703 0xbe2e57 0
1 12644989 0xc0f27d 0
3 12493291 0xbea1eb 0
5 12385353 0xbcfc49 0
6 12598604 0xc03d4c 0
0 12427009 0xbd9f01 0
/esfs/jtacquaviva/ioperf/file_write.00000028
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12385331 0xbcfc33 0
2 12573322 0xbfda8a 0
7 12463679 0xbe2e3f 0
4 12515060 0xbef6f4 0
3 12493268 0xbea1d4 0
0 12426989 0xbd9eed 0
6 12598586 0xc03d3a 0
1 12644971 0xc0f26b 0
/esfs/jtacquaviva/ioperf/file_write.00000010
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12427000 0xbd9ef8 0
3 12493284 0xbea1e4 0
5 12385347 0xbcfc43 0
6 12598600 0xc03d48 0
2 12573341 0xbfda9d 0
4 12515076 0xbef704 0
7 12463698 0xbe2e52 0
1 12644985 0xc0f279 0
/esfs/jtacquaviva/ioperf/file_write.00000017
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573337 0xbfda99 0
1 12644981 0xc0f275 0
3 12493282 0xbea1e2 0
6 12598596 0xc03d44 0
5 12385345 0xbcfc41 0
0 12427001 0xbd9ef9 0
4 12515075 0xbef703 0
7 12463697 0xbe2e51 0
/esfs/jtacquaviva/ioperf/file_write.00000029
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12385360 0xbcfc50 0
6 12598612 0xc03d54 0
0 12427016 0xbd9f08 0
2 12573353 0xbfdaa9 0
4 12515089 0xbef711 0
7 12463710 0xbe2e5e 0
1 12644997 0xc0f285 0
3 12493298 0xbea1f2 0
/esfs/jtacquaviva/ioperf/file_write.00000016
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12598584 0xc03d38 0
7 12463681 0xbe2e41 0
0 12426988 0xbd9eec 0
2 12573326 0xbfda8e 0
3 12493273 0xbea1d9 0
5 12385336 0xbcfc38 0
4 12515065 0xbef6f9 0
/esfs/jtacquaviva/ioperf/file_write.00000005
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12385346 0xbcfc42 0
6 12598599 0xc03d47 0
0 12427003 0xbd9efb 0
2 12573342 0xbfda9e 0
4 12515078 0xbef706 0
7 12463699 0xbe2e53 0
1 12644986 0xc0f27a 0
3 12493288 0xbea1e8 0
/esfs/jtacquaviva/ioperf/file_write.00000014
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12644967 0xc0f267 0
6 12598582 0xc03d36 0
0 12426985 0xbd9ee9 0
5 12385333 0xbcfc35 0
4 12515062 0xbef6f6 0
2 12573325 0xbfda8d 0
3 12493271 0xbea1d7 0
7 12463683 0xbe2e43 0
/esfs/jtacquaviva/ioperf/file_write.00000021
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12385351 0xbcfc47 0
2 12573344 0xbfdaa0 0
4 12515080 0xbef708 0
1 12644988 0xc0f27c 0
6 12598603 0xc03d4b 0
7 12463702 0xbe2e56 0
3 12493290 0xbea1ea 0
0 12427008 0xbd9f00 0
/esfs/jtacquaviva/ioperf/file_write.00000019
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12427007 0xbd9eff 0
7 12463701 0xbe2e55 0
3 12493289 0xbea1e9 0
5 12385352 0xbcfc48 0
2 12573346 0xbfdaa2 0
1 12644990 0xc0f27e 0
4 12515083 0xbef70b 0
6 12598605 0xc03d4d 0
/esfs/jtacquaviva/ioperf/file_write.00000009
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12463690 0xbe2e4a 0
5 12385341 0xbcfc3d 0
4 12515071 0xbef6ff 0
2 12573335 0xbfda97 0
0 12426998 0xbd9ef6 0
1 12644979 0xc0f273 0
3 12493281 0xbea1e1 0
6 12598595 0xc03d43 0
/esfs/jtacquaviva/ioperf/file_write.00000015
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573338 0xbfda9a 0
7 12463696 0xbe2e50 0
3 12493283 0xbea1e3 0
4 12515074 0xbef702 0
5 12385349 0xbcfc45 0
6 12598601 0xc03d49 0
0 12427004 0xbd9efc 0
1 12644984 0xc0f278 0
/esfs/jtacquaviva/ioperf/file_write.00000027
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12598608 0xc03d50 0
7 12463706 0xbe2e5a 0
1 12644994 0xc0f282 0
0 12427013 0xbd9f05 0
2 12573350 0xbfdaa6 0
3 12493295 0xbea1ef 0
5 12385357 0xbcfc4d 0
4 12515086 0xbef70e 0
/esfs/jtacquaviva/ioperf/file_write.00000024
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12493278 0xbea1de 0
0 12426996 0xbd9ef4 0
1 12644978 0xc0f272 0
7 12463691 0xbe2e4b 0
6 12598593 0xc03d41 0
4 12515072 0xbef700 0
5 12385343 0xbcfc3f 0
2 12573336 0xbfda98 0
/esfs/jtacquaviva/ioperf/file_write.00000001
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573323 0xbfda8b 0
0 12426987 0xbd9eeb 0
4 12515063 0xbef6f7 0
3 12493270 0xbea1d6 0
1 12644970 0xc0f26a 0
7 12463685 0xbe2e45 0
6 12598587 0xc03d3b 0
/esfs/jtacquaviva/ioperf/file_write.00000011
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12644973 0xc0f26d 0
3 12493274 0xbea1da 0
5 12385337 0xbcfc39 0
6 12598588 0xc03d3c 0
0 12426992 0xbd9ef0 0
2 12573329 0xbfda91 0
4 12515066 0xbef6fa 0
7 12463686 0xbe2e46 0

View File

@ -0,0 +1,600 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 40265318400 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:4#API:POSIX#T:16384.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 06:39:18 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 40265318400 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 06:39:18 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 32 (4 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 37.50 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288488091648.
WARNING: Using actual aggregate bytes moved = 32587415552.
write 267.29 39321600 16.00 0.002044 116.27 0.000366 116.27 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288455798784.
WARNING: Using actual aggregate bytes moved = 27252424704.
write 221.19 39321600 16.00 0.014096 117.49 0.000384 117.50 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288337129472.
WARNING: Using actual aggregate bytes moved = 27642986496.
write 220.63 39321600 16.00 0.002227 119.49 0.000433 119.49 2
Max Write: 267.29 MiB/sec (280.27 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 267.29 220.63 236.37 21.87 117.75372 0 32 4 3 1 0 1 0 0 1 40265318400 16384 32587415552 POSIX 0
Finished: Wed Oct 24 06:45:12 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ tee -a ./output/COUNT:1#NN:8#PPN:4#API:POSIX#T:16384.txt
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 40265318400 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 06:45:17 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 40265318400 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 06:45:17 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/indread8/file
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 32 (4 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 37.50 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 5153960755200.
WARNING: Using actual aggregate bytes moved = 3915972608.
read 62.08 39321600 16.00 0.001261 60.15 0.000709 60.15 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 5153960755200.
WARNING: Using actual aggregate bytes moved = 5149704192.
read 81.65 39321600 16.00 0.000742 60.15 0.000776 60.15 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 5153960755200.
WARNING: Using actual aggregate bytes moved = 5127798784.
read 81.31 39321600 16.00 0.001570 60.14 0.000759 60.15 2
Max Read: 81.65 MiB/sec (85.61 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 81.65 62.08 75.01 9.14 60.15091 0 32 4 3 1 0 1 0 0 1 40265318400 16384 3915972608 POSIX 0
Finished: Wed Oct 24 06:48:18 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write.00000006
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12386874 0xbd023a 0
6 12600115 0xc04333 0
4 12516594 0xbefcf2 0
7 12465225 0xbe3449 0
1 12646510 0xc0f86e 0
0 12428520 0xbda4e8 0
2 12574869 0xbfe095 0
3 12494813 0xbea7dd 0
/esfs/jtacquaviva/ioperf/file_write.00000023
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494814 0xbea7de 0
5 12386877 0xbd023d 0
6 12600118 0xc04336 0
1 12646513 0xc0f871 0
7 12465229 0xbe344d 0
4 12516598 0xbefcf6 0
2 12574872 0xbfe098 0
0 12428525 0xbda4ed 0
/esfs/jtacquaviva/ioperf/file_write.00000002
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12386883 0xbd0243 0
6 12600124 0xc0433c 0
0 12428531 0xbda4f3 0
4 12516605 0xbefcfd 0
1 12646522 0xc0f87a 0
7 12465236 0xbe3454 0
2 12574881 0xbfe0a1 0
3 12494826 0xbea7ea 0
/esfs/jtacquaviva/ioperf/file_write.00000026
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12574892 0xbfe0ac 0
4 12516618 0xbefd0a 0
7 12465248 0xbe3460 0
1 12646534 0xc0f886 0
3 12494835 0xbea7f3 0
5 12386897 0xbd0251 0
6 12600138 0xc0434a 0
0 12428544 0xbda500 0
/esfs/jtacquaviva/ioperf/file_write.00000000
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12465228 0xbe344c 0
1 12646515 0xc0f873 0
3 12494816 0xbea7e0 0
2 12574874 0xbfe09a 0
4 12516600 0xbefcf8 0
6 12600121 0xc04339 0
5 12386879 0xbd023f 0
0 12428527 0xbda4ef 0
/esfs/jtacquaviva/ioperf/file_write.00000020
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12465240 0xbe3458 0
1 12646526 0xc0f87e 0
3 12494827 0xbea7eb 0
5 12386889 0xbd0249 0
6 12600130 0xc04342 0
0 12428536 0xbda4f8 0
2 12574884 0xbfe0a4 0
4 12516610 0xbefd02 0
/esfs/jtacquaviva/ioperf/file_write.00000030
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12600133 0xc04345 0
0 12428539 0xbda4fb 0
2 12574887 0xbfe0a7 0
1 12646529 0xc0f881 0
3 12494830 0xbea7ee 0
4 12516613 0xbefd05 0
5 12386893 0xbd024d 0
7 12465244 0xbe345c 0
/esfs/jtacquaviva/ioperf/file_write.00000004
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12646523 0xc0f87b 0
6 12600126 0xc0433e 0
5 12386887 0xbd0247 0
7 12465238 0xbe3456 0
0 12428534 0xbda4f6 0
4 12516609 0xbefd01 0
2 12574883 0xbfe0a3 0
/esfs/jtacquaviva/ioperf/file_write.00000007
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12465243 0xbe345b 0
5 12386892 0xbd024c 0
2 12574888 0xbfe0a8 0
1 12646530 0xc0f882 0
6 12600134 0xc04346 0
0 12428540 0xbda4fc 0
3 12494832 0xbea7f0 0
4 12516614 0xbefd06 0
/esfs/jtacquaviva/ioperf/file_write.00000018
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12574879 0xbfe09f 0
3 12494825 0xbea7e9 0
6 12600127 0xc0433f 0
0 12428533 0xbda4f5 0
1 12646524 0xc0f87c 0
4 12516608 0xbefd00 0
7 12465239 0xbe3457 0
/esfs/jtacquaviva/ioperf/file_write.00000031
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12386898 0xbd0252 0
1 12646536 0xc0f888 0
6 12600140 0xc0434c 0
0 12428546 0xbda502 0
2 12574894 0xbfe0ae 0
4 12516620 0xbefd0c 0
7 12465250 0xbe3462 0
3 12494837 0xbea7f5 0
/esfs/jtacquaviva/ioperf/file_write.00000012
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428542 0xbda4fe 0
7 12465246 0xbe345e 0
3 12494834 0xbea7f2 0
2 12574890 0xbfe0aa 0
4 12516616 0xbefd08 0
1 12646532 0xc0f884 0
5 12386896 0xbd0250 0
6 12600137 0xc04349 0
/esfs/jtacquaviva/ioperf/file_write.00000013
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494824 0xbea7e8 0
4 12516606 0xbefcfe 0
2 12574880 0xbfe0a0 0
7 12465237 0xbe3455 0
1 12646525 0xc0f87d 0
6 12600129 0xc04341 0
0 12428535 0xbda4f7 0
5 12386888 0xbd0248 0
/esfs/jtacquaviva/ioperf/file_write.00000025
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12516597 0xbefcf5 0
3 12494817 0xbea7e1 0
7 12465230 0xbe344e 0
2 12574875 0xbfe09b 0
5 12386878 0xbd023e 0
1 12646517 0xc0f875 0
0 12428526 0xbda4ee 0
/esfs/jtacquaviva/ioperf/file_write.00000022
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12646518 0xc0f876 0
2 12574877 0xbfe09d 0
7 12465233 0xbe3451 0
4 12516603 0xbefcfb 0
0 12428530 0xbda4f2 0
6 12600123 0xc0433b 0
5 12386885 0xbd0245 0
3 12494821 0xbea7e5 0
/esfs/jtacquaviva/ioperf/file_write.00000003
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12600141 0xc0434d 0
0 12428547 0xbda503 0
2 12574895 0xbfe0af 0
4 12516621 0xbefd0d 0
7 12465251 0xbe3463 0
1 12646537 0xc0f889 0
3 12494838 0xbea7f6 0
5 12386900 0xbd0254 0
/esfs/jtacquaviva/ioperf/file_write.00000008
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12646527 0xc0f87f 0
3 12494828 0xbea7ec 0
0 12428537 0xbda4f9 0
2 12574885 0xbfe0a5 0
7 12465241 0xbe3459 0
5 12386891 0xbd024b 0
6 12600132 0xc04344 0
4 12516612 0xbefd04 0
/esfs/jtacquaviva/ioperf/file_write.00000028
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12465235 0xbe3453 0
3 12494822 0xbea7e6 0
1 12646521 0xc0f879 0
5 12386886 0xbd0246 0
0 12428532 0xbda4f4 0
4 12516607 0xbefcff 0
2 12574882 0xbfe0a2 0
6 12600128 0xc04340 0
/esfs/jtacquaviva/ioperf/file_write.00000010
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428523 0xbda4eb 0
1 12646514 0xc0f872 0
2 12574871 0xbfe097 0
3 12494818 0xbea7e2 0
4 12516599 0xbefcf7 0
6 12600120 0xc04338 0
7 12465232 0xbe3450 0
5 12386881 0xbd0241 0
/esfs/jtacquaviva/ioperf/file_write.00000017
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494833 0xbea7f1 0
5 12386895 0xbd024f 0
2 12574891 0xbfe0ab 0
4 12516617 0xbefd09 0
1 12646533 0xc0f885 0
6 12600136 0xc04348 0
0 12428543 0xbda4ff 0
7 12465247 0xbe345f 0
/esfs/jtacquaviva/ioperf/file_write.00000029
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12574868 0xbfe094 0
3 12494812 0xbea7dc 0
4 12516595 0xbefcf3 0
7 12465226 0xbe344a 0
5 12386875 0xbd023b 0
6 12600116 0xc04334 0
1 12646511 0xc0f86f 0
0 12428521 0xbda4e9 0
/esfs/jtacquaviva/ioperf/file_write.00000016
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12516596 0xbefcf4 0
7 12465227 0xbe344b 0
5 12386876 0xbd023c 0
1 12646512 0xc0f870 0
2 12574870 0xbfe096 0
0 12428522 0xbda4ea 0
6 12600117 0xc04335 0
3 12494815 0xbea7df 0
/esfs/jtacquaviva/ioperf/file_write.00000005
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12516619 0xbefd0b 0
7 12465249 0xbe3461 0
1 12646535 0xc0f887 0
6 12600139 0xc0434b 0
0 12428545 0xbda501 0
2 12574893 0xbfe0ad 0
3 12494836 0xbea7f4 0
5 12386899 0xbd0253 0
/esfs/jtacquaviva/ioperf/file_write.00000014
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12646538 0xc0f88a 0
6 12600142 0xc0434e 0
3 12494839 0xbea7f7 0
0 12428549 0xbda505 0
5 12386902 0xbd0256 0
2 12574897 0xbfe0b1 0
4 12516623 0xbefd0f 0
7 12465253 0xbe3465 0
/esfs/jtacquaviva/ioperf/file_write.00000021
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12465223 0xbe3447 0
5 12386872 0xbd0238 0
0 12428518 0xbda4e6 0
2 12574866 0xbfe092 0
1 12646508 0xc0f86c 0
3 12494811 0xbea7db 0
6 12600114 0xc04332 0
4 12516593 0xbefcf1 0
/esfs/jtacquaviva/ioperf/file_write.00000019
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428548 0xbda504 0
2 12574896 0xbfe0b0 0
4 12516622 0xbefd0e 0
3 12494840 0xbea7f8 0
5 12386901 0xbd0255 0
7 12465252 0xbe3464 0
6 12600143 0xc0434f 0
1 12646539 0xc0f88b 0
/esfs/jtacquaviva/ioperf/file_write.00000009
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12646520 0xc0f878 0
0 12428529 0xbda4f1 0
4 12516604 0xbefcfc 0
5 12386884 0xbd0244 0
6 12600125 0xc0433d 0
2 12574878 0xbfe09e 0
3 12494823 0xbea7e7 0
/esfs/jtacquaviva/ioperf/file_write.00000015
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428524 0xbda4ec 0
6 12600119 0xc04337 0
2 12574873 0xbfe099 0
1 12646516 0xc0f874 0
3 12494819 0xbea7e3 0
7 12465231 0xbe344f 0
5 12386880 0xbd0240 0
4 12516601 0xbefcf9 0
/esfs/jtacquaviva/ioperf/file_write.00000027
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12516602 0xbefcfa 0
6 12600122 0xc0433a 0
2 12574876 0xbfe09c 0
3 12494820 0xbea7e4 0
5 12386882 0xbd0242 0
0 12428528 0xbda4f0 0
1 12646519 0xc0f877 0
7 12465234 0xbe3452 0
/esfs/jtacquaviva/ioperf/file_write.00000024
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12600131 0xc04343 0
4 12516611 0xbefd03 0
1 12646528 0xc0f880 0
0 12428538 0xbda4fa 0
5 12386890 0xbd024a 0
2 12574886 0xbfe0a6 0
7 12465242 0xbe345a 0
3 12494829 0xbea7ed 0
/esfs/jtacquaviva/ioperf/file_write.00000001
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12646531 0xc0f883 0
3 12494831 0xbea7ef 0
5 12386894 0xbd024e 0
6 12600135 0xc04347 0
0 12428541 0xbda4fd 0
2 12574889 0xbfe0a9 0
4 12516615 0xbefd07 0
7 12465245 0xbe345d 0
/esfs/jtacquaviva/ioperf/file_write.00000011
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494810 0xbea7da 0
6 12600113 0xc04331 0
4 12516592 0xbefcf0 0
5 12386873 0xbd0239 0
0 12428519 0xbda4e7 0
2 12574867 0xbfe093 0
7 12465224 0xbe3448 0
1 12646509 0xc0f86d 0

View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 48 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 26843545600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:6#API:MPIIO#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 06:03:30 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 26843545600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 06:03:30 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 48 (6 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 25 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288489574400.
WARNING: Using actual aggregate bytes moved = 24692736000.
write 387.51 26214400 100.00 0.010085 60.76 0.000485 60.77 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288487116800.
WARNING: Using actual aggregate bytes moved = 24790630400.
write 389.21 26214400 100.00 0.001933 60.74 0.000494 60.74 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288489881600.
WARNING: Using actual aggregate bytes moved = 24570265600.
write 385.76 26214400 100.00 0.002338 60.74 0.000528 60.74 2
Max Write: 389.21 MiB/sec (408.11 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 389.21 385.76 387.49 1.41 60.75249 0 48 6 3 0 0 1 0 0 1 26843545600 102400 24692736000 MPIIO 0
Finished: Wed Oct 24 06:06:33 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 48 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 48 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 26843545600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
+ tee -a ./output/COUNT:1#NN:8#PPN:6#API:MPIIO#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 06:06:41 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 26843545600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 06:06:41 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 48 (6 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 25 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 21360230400.
read 335.58 26214400 100.00 0.018368 60.68 0.000767 60.70 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 21834342400.
read 343.08 26214400 100.00 0.002198 60.69 0.000792 60.69 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 22139699200.
read 347.93 26214400 100.00 0.002467 60.68 0.000849 60.69 2
Max Read: 347.93 MiB/sec (364.83 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 347.93 335.58 342.19 5.08 60.69428 0 48 6 3 0 0 1 0 0 1 26843545600 102400 21360230400 MPIIO 0
Finished: Wed Oct 24 06:09:43 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494498 0xbea6a2 0
5 12386568 0xbd0108 0
6 12599809 0xc04201 0
0 12428211 0xbda3b3 0
2 12574557 0xbfdf5d 0
4 12516284 0xbefbbc 0
7 12464914 0xbe3312 0
1 12646197 0xc0f735 0

View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 48 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 26843545600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:6#API:MPIIO#T:1048576.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 04:53:21 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 26843545600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 04:53:21 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 48 (6 per node)
repetitions = 3
xfersize = 1 MiB
blocksize = 25 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288490188800.
WARNING: Using actual aggregate bytes moved = 294345768960.
write 4669 26214400 1024.00 0.010255 60.10 0.007472 60.12 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288490188800.
WARNING: Using actual aggregate bytes moved = 286901927936.
write 4535 26214400 1024.00 0.224817 60.11 0.000744 60.33 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288470265856.
WARNING: Using actual aggregate bytes moved = 287090671616.
write 4544 26214400 1024.00 0.143342 60.11 0.000853 60.25 2
Max Write: 4669.30 MiB/sec (4896.11 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 4669.30 4535.12 4582.85 61.24 60.23384 0 48 6 3 0 0 1 0 0 1 26843545600 1048576 294345768960 MPIIO 0
Finished: Wed Oct 24 04:56:28 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 48 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 48 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 26843545600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
+ tee -a ./output/COUNT:1#NN:8#PPN:6#API:MPIIO#T:1048576.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 04:56:47 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 26843545600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 04:56:47 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 48 (6 per node)
repetitions = 3
xfersize = 1 MiB
blocksize = 25 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 89326092288.
read 1414.48 26214400 1024.00 0.116877 60.11 0.002345 60.23 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 91959066624.
read 1458.33 26214400 1024.00 0.003724 60.13 0.002358 60.14 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 92708798464.
read 1470.67 26214400 1024.00 0.004265 60.11 0.002330 60.12 2
Max Read: 1470.67 MiB/sec (1542.10 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 1470.67 1414.48 1447.83 24.11 60.16018 0 48 6 3 0 0 1 0 0 1 26843545600 1048576 89326092288 MPIIO 0
Finished: Wed Oct 24 04:59:48 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12574044 0xbfdd5c 0
4 12515773 0xbef9bd 0
7 12464399 0xbe310f 0
1 12645688 0xc0f538 0
3 12493985 0xbea4a1 0
5 12386053 0xbcff05 0
6 12599297 0xc04001 0
0 12427702 0xbda1b6 0

View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 48 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 26843545600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:6#API:MPIIO#T:10485760.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 03:46:15 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 26843545600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 03:46:15 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 48 (6 per node)
repetitions = 3
xfersize = 10 MiB
blocksize = 25 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288427274240.
WARNING: Using actual aggregate bytes moved = 203077713920.
write 3215.70 26214400 10240 0.015478 60.21 0.000596 60.23 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288217559040.
WARNING: Using actual aggregate bytes moved = 195790110720.
write 3081.74 26214400 10240 0.176834 60.41 0.000714 60.59 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288469217280.
WARNING: Using actual aggregate bytes moved = 202343710720.
write 3169.02 26214400 10240 0.021270 60.87 0.000804 60.89 2
Max Write: 3215.70 MiB/sec (3371.90 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 3215.70 3081.74 3155.49 55.52 60.56940 0 48 6 3 0 0 1 0 0 1 26843545600 10485760 203077713920 MPIIO 0
Finished: Wed Oct 24 03:49:22 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 48 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 48 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 26843545600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
+ tee -a ./output/COUNT:1#NN:8#PPN:6#API:MPIIO#T:10485760.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 03:49:41 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 26843545600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 03:49:41 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 48 (6 per node)
repetitions = 3
xfersize = 10 MiB
blocksize = 25 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 195171450880.
read 3089.98 26214400 10240 0.062367 60.17 0.008497 60.24 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 199470612480.
read 3163.62 26214400 10240 0.006616 60.13 0.005305 60.13 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 200057815040.
read 3171.48 26214400 10240 0.007063 60.16 0.005277 60.16 2
Max Read: 3171.48 MiB/sec (3325.53 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 3171.48 3089.98 3141.69 36.70 60.17508 0 48 6 3 0 0 1 0 0 1 26843545600 10485760 195171450880 MPIIO 0
Finished: Wed Oct 24 03:52:42 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12515263 0xbef7bf 0
7 12463885 0xbe2f0d 0
1 12645174 0xc0f336 0
3 12493473 0xbea2a1 0
5 12385538 0xbcfd02 0
6 12598783 0xc03dff 0
0 12427191 0xbd9fb7 0
2 12573529 0xbfdb59 0

View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 48 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 26843545600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:6#API:MPIIO#T:16384.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 07:19:24 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 26843545600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 07:19:24 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 48 (6 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 25 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288483831808.
WARNING: Using actual aggregate bytes moved = 4463050752.
write 66.13 26214400 16.00 0.015352 64.34 0.006864 64.36 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288486223872.
WARNING: Using actual aggregate bytes moved = 4473749504.
write 66.08 26214400 16.00 0.001987 64.56 0.000559 64.57 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288484454400.
WARNING: Using actual aggregate bytes moved = 4464164864.
write 66.30 26214400 16.00 0.002209 64.21 0.000578 64.22 2
Max Write: 66.30 MiB/sec (69.52 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 66.30 66.08 66.17 0.09 64.38060 0 48 6 3 0 0 1 0 0 1 26843545600 16384 4463050752 MPIIO 0
Finished: Wed Oct 24 07:22:38 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 48 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ tee -a ./output/COUNT:1#NN:8#PPN:6#API:MPIIO#T:16384.txt
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 48 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 26843545600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 07:22:43 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 26843545600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 07:22:43 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 48 (6 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 25 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 4522311680.
read 67.19 26214400 16.00 0.015215 64.18 0.000954 64.19 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 4550836224.
read 67.61 26214400 16.00 0.002446 64.19 0.001371 64.19 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 4560273408.
read 67.76 26214400 16.00 0.003257 64.18 0.001555 64.18 2
Max Read: 67.76 MiB/sec (71.05 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 67.76 67.19 67.52 0.24 64.18861 0 48 6 3 0 0 1 0 0 1 26843545600 16384 4522311680 MPIIO 0
Finished: Wed Oct 24 07:25:55 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12387076 0xbd0304 0
6 12600319 0xc043ff 0
0 12428724 0xbda5b4 0
2 12575073 0xbfe161 0
4 12516798 0xbefdbe 0
7 12465431 0xbe3517 0
1 12646714 0xc0f93a 0
3 12495017 0xbea8a9 0

View File

@ -0,0 +1,850 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 48 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 26843545600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:6#API:POSIX#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 05:55:47 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 26843545600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 05:55:47 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 48 (6 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 25 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288431206400.
WARNING: Using actual aggregate bytes moved = 86527283200.
write 918.39 26214400 100.00 0.003204 89.85 0.000397 89.85 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288349696000.
WARNING: Using actual aggregate bytes moved = 75418931200.
write 799.92 26214400 100.00 0.005326 89.91 0.000442 89.92 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288346009600.
WARNING: Using actual aggregate bytes moved = 77376921600.
write 807.16 26214400 100.00 0.006983 91.42 0.000547 91.42 2
Max Write: 918.39 MiB/sec (963.00 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 918.39 799.92 841.82 54.22 90.39670 0 48 6 3 1 0 1 0 0 1 26843545600 102400 86527283200 POSIX 0
Finished: Wed Oct 24 06:00:19 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 48 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 48 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 26843545600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
+ tee -a ./output/COUNT:1#NN:8#PPN:6#API:POSIX#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 06:00:28 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 26843545600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 06:00:28 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/indread8/file
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 48 (6 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 25 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 7730941132800.
WARNING: Using actual aggregate bytes moved = 35547340800.
read 564.70 26214400 100.00 0.004925 60.03 0.004252 60.03 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 7730941132800.
WARNING: Using actual aggregate bytes moved = 37631078400.
read 597.80 26214400 100.00 0.004236 60.03 0.004165 60.03 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 7730941132800.
WARNING: Using actual aggregate bytes moved = 38628454400.
read 613.67 26214400 100.00 0.004246 60.03 0.004178 60.03 2
Max Read: 613.67 MiB/sec (643.48 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 613.67 564.70 592.06 20.40 60.03218 0 48 6 3 1 0 1 0 0 1 26843545600 102400 35547340800 POSIX 0
Finished: Wed Oct 24 06:03:28 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write.00000006
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428180 0xbda394 0
2 12574527 0xbfdf3f 0
3 12494470 0xbea686 0
1 12646170 0xc0f71a 0
5 12386541 0xbd00ed 0
6 12599782 0xc041e6 0
4 12516257 0xbefba1 0
7 12464886 0xbe32f6 0
/esfs/jtacquaviva/ioperf/file_write.00000023
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428167 0xbda387 0
7 12464871 0xbe32e7 0
1 12646155 0xc0f70b 0
5 12386525 0xbd00dd 0
2 12574513 0xbfdf31 0
3 12494456 0xbea678 0
4 12516242 0xbefb92 0
6 12599769 0xc041d9 0
/esfs/jtacquaviva/ioperf/file_write.00000044
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12574515 0xbfdf33 0
4 12516243 0xbefb93 0
5 12386529 0xbd00e1 0
7 12464875 0xbe32eb 0
6 12599772 0xc041dc 0
0 12428171 0xbda38b 0
1 12646160 0xc0f710 0
3 12494460 0xbea67c 0
/esfs/jtacquaviva/ioperf/file_write.00000040
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12599786 0xc041ea 0
0 12428187 0xbda39b 0
2 12574533 0xbfdf45 0
4 12516261 0xbefba5 0
7 12464890 0xbe32fa 0
1 12646174 0xc0f71e 0
3 12494475 0xbea68b 0
5 12386545 0xbd00f1 0
/esfs/jtacquaviva/ioperf/file_write.00000002
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12599778 0xc041e2 0
4 12516252 0xbefb9c 0
5 12386536 0xbd00e8 0
7 12464883 0xbe32f3 0
1 12646166 0xc0f716 0
2 12574526 0xbfdf3e 0
3 12494468 0xbea684 0
0 12428182 0xbda396 0
/esfs/jtacquaviva/ioperf/file_write.00000026
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428174 0xbda38e 0
3 12494462 0xbea67e 0
2 12574521 0xbfdf39 0
5 12386533 0xbd00e5 0
4 12516250 0xbefb9a 0
1 12646164 0xc0f714 0
7 12464879 0xbe32ef 0
6 12599776 0xc041e0 0
/esfs/jtacquaviva/ioperf/file_write.00000033
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464894 0xbe32fe 0
5 12386549 0xbd00f5 0
0 12428191 0xbda39f 0
4 12516265 0xbefba9 0
1 12646178 0xc0f722 0
6 12599791 0xc041ef 0
2 12574538 0xbfdf4a 0
/esfs/jtacquaviva/ioperf/file_write.00000000
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494453 0xbea675 0
0 12428165 0xbda385 0
2 12574511 0xbfdf2f 0
7 12464869 0xbe32e5 0
5 12386523 0xbd00db 0
6 12599767 0xc041d7 0
1 12646154 0xc0f70a 0
4 12516240 0xbefb90 0
/esfs/jtacquaviva/ioperf/file_write.00000020
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494486 0xbea696 0
5 12386556 0xbd00fc 0
6 12599797 0xc041f5 0
0 12428199 0xbda3a7 0
2 12574544 0xbfdf50 0
4 12516272 0xbefbb0 0
7 12464902 0xbe3306 0
1 12646185 0xc0f729 0
/esfs/jtacquaviva/ioperf/file_write.00000030
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12646184 0xc0f728 0
3 12494485 0xbea695 0
5 12386555 0xbd00fb 0
6 12599796 0xc041f4 0
0 12428198 0xbda3a6 0
2 12574543 0xbfdf4f 0
4 12516271 0xbefbaf 0
7 12464901 0xbe3305 0
/esfs/jtacquaviva/ioperf/file_write.00000032
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12646152 0xc0f708 0
3 12494452 0xbea674 0
5 12386522 0xbd00da 0
6 12599765 0xc041d5 0
0 12428164 0xbda384 0
2 12574510 0xbfdf2e 0
4 12516238 0xbefb8e 0
7 12464868 0xbe32e4 0
/esfs/jtacquaviva/ioperf/file_write.00000004
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464878 0xbe32ee 0
1 12646163 0xc0f713 0
0 12428175 0xbda38f 0
3 12494463 0xbea67f 0
4 12516248 0xbefb98 0
2 12574522 0xbfdf3a 0
5 12386534 0xbd00e6 0
/esfs/jtacquaviva/ioperf/file_write.00000007
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12574535 0xbfdf47 0
4 12516263 0xbefba7 0
7 12464892 0xbe32fc 0
1 12646176 0xc0f720 0
3 12494477 0xbea68d 0
5 12386547 0xbd00f3 0
6 12599788 0xc041ec 0
0 12428189 0xbda39d 0
/esfs/jtacquaviva/ioperf/file_write.00000018
lmm_stripe_count: 6
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494479 0xbea68f 0
6 12599790 0xc041ee 0
2 12574537 0xbfdf49 0
7 12464895 0xbe32ff 0
5 12386550 0xbd00f6 0
0 12428192 0xbda3a0 0
/esfs/jtacquaviva/ioperf/file_write.00000041
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12516241 0xbefb91 0
3 12494455 0xbea677 0
6 12599768 0xc041d8 0
7 12464872 0xbe32e8 0
5 12386526 0xbd00de 0
1 12646156 0xc0f70c 0
0 12428168 0xbda388 0
2 12574514 0xbfdf32 0
/esfs/jtacquaviva/ioperf/file_write.00000031
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464873 0xbe32e9 0
5 12386528 0xbd00e0 0
4 12516245 0xbefb95 0
6 12599771 0xc041db 0
1 12646159 0xc0f70f 0
3 12494459 0xbea67b 0
0 12428172 0xbda38c 0
2 12574518 0xbfdf36 0
/esfs/jtacquaviva/ioperf/file_write.00000012
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428179 0xbda393 0
1 12646169 0xc0f719 0
5 12386538 0xbd00ea 0
4 12516255 0xbefb9f 0
2 12574529 0xbfdf41 0
3 12494471 0xbea687 0
6 12599781 0xc041e5 0
/esfs/jtacquaviva/ioperf/file_write.00000013
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428188 0xbda39c 0
2 12574534 0xbfdf46 0
4 12516262 0xbefba6 0
7 12464891 0xbe32fb 0
1 12646175 0xc0f71f 0
3 12494476 0xbea68c 0
5 12386546 0xbd00f2 0
6 12599787 0xc041eb 0
/esfs/jtacquaviva/ioperf/file_write.00000042
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12516276 0xbefbb4 0
7 12464906 0xbe330a 0
5 12386560 0xbd0100 0
6 12599801 0xc041f9 0
2 12574548 0xbfdf54 0
1 12646189 0xc0f72d 0
0 12428204 0xbda3ac 0
3 12494491 0xbea69b 0
/esfs/jtacquaviva/ioperf/file_write.00000025
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12599804 0xc041fc 0
0 12428206 0xbda3ae 0
2 12574551 0xbfdf57 0
4 12516279 0xbefbb7 0
7 12464909 0xbe330d 0
1 12646192 0xc0f730 0
3 12494493 0xbea69d 0
5 12386563 0xbd0103 0
/esfs/jtacquaviva/ioperf/file_write.00000046
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464899 0xbe3303 0
1 12646183 0xc0f727 0
3 12494483 0xbea693 0
5 12386554 0xbd00fa 0
6 12599795 0xc041f3 0
0 12428197 0xbda3a5 0
2 12574542 0xbfdf4e 0
4 12516270 0xbefbae 0
/esfs/jtacquaviva/ioperf/file_write.00000039
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12386531 0xbd00e3 0
1 12646161 0xc0f711 0
6 12599774 0xc041de 0
3 12494461 0xbea67d 0
2 12574519 0xbfdf37 0
7 12464877 0xbe32ed 0
0 12428173 0xbda38d 0
4 12516247 0xbefb97 0
/esfs/jtacquaviva/ioperf/file_write.00000022
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12574523 0xbfdf3b 0
4 12516251 0xbefb9b 0
7 12464881 0xbe32f1 0
1 12646165 0xc0f715 0
3 12494465 0xbea681 0
0 12428177 0xbda391 0
5 12386535 0xbd00e7 0
6 12599777 0xc041e1 0
/esfs/jtacquaviva/ioperf/file_write.00000034
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494473 0xbea689 0
5 12386543 0xbd00ef 0
6 12599784 0xc041e8 0
0 12428185 0xbda399 0
2 12574531 0xbfdf43 0
4 12516259 0xbefba3 0
7 12464888 0xbe32f8 0
1 12646172 0xc0f71c 0
/esfs/jtacquaviva/ioperf/file_write.00000003
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12386544 0xbd00f0 0
6 12599785 0xc041e9 0
0 12428186 0xbda39a 0
2 12574532 0xbfdf44 0
4 12516260 0xbefba4 0
7 12464889 0xbe32f9 0
1 12646173 0xc0f71d 0
3 12494474 0xbea68a 0
/esfs/jtacquaviva/ioperf/file_write.00000008
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12646157 0xc0f70d 0
3 12494458 0xbea67a 0
0 12428170 0xbda38a 0
5 12386530 0xbd00e2 0
2 12574517 0xbfdf35 0
6 12599773 0xc041dd 0
7 12464876 0xbe32ec 0
4 12516246 0xbefb96 0
/esfs/jtacquaviva/ioperf/file_write.00000038
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12386552 0xbd00f8 0
7 12464897 0xbe3301 0
0 12428195 0xbda3a3 0
2 12574541 0xbfdf4d 0
1 12646181 0xc0f725 0
4 12516269 0xbefbad 0
3 12494482 0xbea692 0
/esfs/jtacquaviva/ioperf/file_write.00000028
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428207 0xbda3af 0
2 12574552 0xbfdf58 0
4 12516280 0xbefbb8 0
7 12464910 0xbe330e 0
1 12646193 0xc0f731 0
3 12494494 0xbea69e 0
5 12386564 0xbd0104 0
6 12599805 0xc041fd 0
/esfs/jtacquaviva/ioperf/file_write.00000010
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12574508 0xbfdf2c 0
1 12646150 0xc0f706 0
3 12494450 0xbea672 0
6 12599763 0xc041d3 0
4 12516236 0xbefb8c 0
5 12386521 0xbd00d9 0
0 12428163 0xbda383 0
7 12464867 0xbe32e3 0
/esfs/jtacquaviva/ioperf/file_write.00000043
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494467 0xbea683 0
6 12599780 0xc041e4 0
2 12574528 0xbfdf40 0
5 12386539 0xbd00eb 0
4 12516256 0xbefba0 0
7 12464885 0xbe32f5 0
0 12428183 0xbda397 0
/esfs/jtacquaviva/ioperf/file_write.00000036
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12574546 0xbfdf52 0
1 12646187 0xc0f72b 0
5 12386558 0xbd00fe 0
4 12516275 0xbefbb3 0
3 12494489 0xbea699 0
7 12464905 0xbe3309 0
6 12599800 0xc041f8 0
0 12428202 0xbda3aa 0
/esfs/jtacquaviva/ioperf/file_write.00000017
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428194 0xbda3a2 0
3 12494481 0xbea691 0
2 12574540 0xbfdf4c 0
7 12464898 0xbe3302 0
6 12599793 0xc041f1 0
4 12516268 0xbefbac 0
1 12646182 0xc0f726 0
/esfs/jtacquaviva/ioperf/file_write.00000029
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12599766 0xc041d6 0
4 12516239 0xbefb8f 0
1 12646153 0xc0f709 0
0 12428166 0xbda386 0
3 12494454 0xbea676 0
2 12574512 0xbfdf30 0
7 12464870 0xbe32e6 0
5 12386524 0xbd00dc 0
/esfs/jtacquaviva/ioperf/file_write.00000016
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464866 0xbe32e2 0
5 12386520 0xbd00d8 0
0 12428162 0xbda382 0
1 12646151 0xc0f707 0
3 12494451 0xbea673 0
6 12599764 0xc041d4 0
2 12574509 0xbfdf2d 0
4 12516237 0xbefb8d 0
/esfs/jtacquaviva/ioperf/file_write.00000005
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494480 0xbea690 0
5 12386551 0xbd00f7 0
6 12599792 0xc041f0 0
0 12428193 0xbda3a1 0
2 12574539 0xbfdf4b 0
4 12516266 0xbefbaa 0
7 12464896 0xbe3300 0
1 12646179 0xc0f723 0
/esfs/jtacquaviva/ioperf/file_write.00000014
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494457 0xbea679 0
6 12599770 0xc041da 0
5 12386527 0xbd00df 0
0 12428169 0xbda389 0
7 12464874 0xbe32ea 0
2 12574516 0xbfdf34 0
4 12516244 0xbefb94 0
1 12646158 0xc0f70e 0
/esfs/jtacquaviva/ioperf/file_write.00000021
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12646171 0xc0f71b 0
3 12494472 0xbea688 0
5 12386542 0xbd00ee 0
6 12599783 0xc041e7 0
0 12428184 0xbda398 0
2 12574530 0xbfdf42 0
4 12516258 0xbefba2 0
7 12464887 0xbe32f7 0
/esfs/jtacquaviva/ioperf/file_write.00000045
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428161 0xbda381 0
2 12574507 0xbfdf2b 0
4 12516235 0xbefb8b 0
7 12464865 0xbe32e1 0
1 12646149 0xc0f705 0
3 12494449 0xbea671 0
5 12386519 0xbd00d7 0
6 12599762 0xc041d2 0
/esfs/jtacquaviva/ioperf/file_write.00000019
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12646162 0xc0f712 0
2 12574520 0xbfdf38 0
5 12386532 0xbd00e4 0
4 12516249 0xbefb99 0
6 12599775 0xc041df 0
7 12464880 0xbe32f0 0
0 12428176 0xbda390 0
3 12494464 0xbea680 0
/esfs/jtacquaviva/ioperf/file_write.00000047
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12516264 0xbefba8 0
7 12464893 0xbe32fd 0
1 12646177 0xc0f721 0
3 12494478 0xbea68e 0
5 12386548 0xbd00f4 0
6 12599789 0xc041ed 0
0 12428190 0xbda39e 0
2 12574536 0xbfdf48 0
/esfs/jtacquaviva/ioperf/file_write.00000035
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494466 0xbea682 0
7 12464882 0xbe32f2 0
2 12574524 0xbfdf3c 0
5 12386537 0xbd00e9 0
6 12599779 0xc041e3 0
0 12428178 0xbda392 0
1 12646168 0xc0f718 0
4 12516253 0xbefb9d 0
/esfs/jtacquaviva/ioperf/file_write.00000009
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494490 0xbea69a 0
0 12428203 0xbda3ab 0
4 12516277 0xbefbb5 0
7 12464907 0xbe330b 0
5 12386561 0xbd0101 0
2 12574549 0xbfdf55 0
6 12599802 0xc041fa 0
1 12646190 0xc0f72e 0
/esfs/jtacquaviva/ioperf/file_write.00000015
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12599799 0xc041f7 0
4 12516274 0xbefbb2 0
7 12464904 0xbe3308 0
3 12494488 0xbea698 0
0 12428201 0xbda3a9 0
1 12646188 0xc0f72c 0
2 12574547 0xbfdf53 0
5 12386559 0xbd00ff 0
/esfs/jtacquaviva/ioperf/file_write.00000027
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12386562 0xbd0102 0
6 12599803 0xc041fb 0
0 12428205 0xbda3ad 0
2 12574550 0xbfdf56 0
4 12516278 0xbefbb6 0
7 12464908 0xbe330c 0
1 12646191 0xc0f72f 0
3 12494492 0xbea69c 0
/esfs/jtacquaviva/ioperf/file_write.00000024
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12574553 0xbfdf59 0
4 12516281 0xbefbb9 0
7 12464911 0xbe330f 0
1 12646194 0xc0f732 0
3 12494495 0xbea69f 0
5 12386565 0xbd0105 0
6 12599806 0xc041fe 0
0 12428208 0xbda3b0 0
/esfs/jtacquaviva/ioperf/file_write.00000001
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12386557 0xbd00fd 0
6 12599798 0xc041f6 0
0 12428200 0xbda3a8 0
2 12574545 0xbfdf51 0
4 12516273 0xbefbb1 0
7 12464903 0xbe3307 0
1 12646186 0xc0f72a 0
3 12494487 0xbea697 0
/esfs/jtacquaviva/ioperf/file_write.00000037
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12574525 0xbfdf3d 0
1 12646167 0xc0f717 0
0 12428181 0xbda395 0
4 12516254 0xbefb9e 0
3 12494469 0xbea685 0
5 12386540 0xbd00ec 0
7 12464884 0xbe32f4 0
/esfs/jtacquaviva/ioperf/file_write.00000011
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12516267 0xbefbab 0
1 12646180 0xc0f724 0
6 12599794 0xc041f2 0
7 12464900 0xbe3304 0
5 12386553 0xbd00f9 0
3 12494484 0xbea694 0
0 12428196 0xbda3a4 0

View File

@ -0,0 +1,856 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 48 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 26843545600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:6#API:POSIX#T:1048576.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 04:46:28 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 26843545600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 04:46:28 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 48 (6 per node)
repetitions = 3
xfersize = 1 MiB
blocksize = 25 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288490188800.
WARNING: Using actual aggregate bytes moved = 354606383104.
write 4887 26214400 1024.00 0.003007 69.19 0.000488 69.19 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288490188800.
WARNING: Using actual aggregate bytes moved = 336111599616.
write 4583 26214400 1024.00 0.004174 69.94 0.000649 69.95 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288199733248.
WARNING: Using actual aggregate bytes moved = 347011547136.
write 4804 26214400 1024.00 0.003389 68.88 0.000886 68.89 2
Max Write: 4887.50 MiB/sec (5124.91 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 4887.50 4582.56 4758.03 128.67 69.34262 0 48 6 3 1 0 1 0 0 1 26843545600 1048576 354606383104 POSIX 0
Finished: Wed Oct 24 04:50:01 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 48 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 48 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 26843545600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
+ tee -a ./output/COUNT:1#NN:8#PPN:6#API:POSIX#T:1048576.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 04:50:18 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 26843545600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 04:50:18 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/indread8/file
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 48 (6 per node)
repetitions = 3
xfersize = 1 MiB
blocksize = 25 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 7730941132800.
WARNING: Using actual aggregate bytes moved = 216919965696.
read 3446.39 26214400 1024.00 0.001722 60.02 0.000834 60.03 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 7730941132800.
WARNING: Using actual aggregate bytes moved = 255836815360.
read 4065 26214400 1024.00 0.000895 60.02 0.000866 60.02 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 7730941132800.
WARNING: Using actual aggregate bytes moved = 276677263360.
read 4396 26214400 1024.00 0.000878 60.02 0.000887 60.02 2
Max Read: 4395.92 MiB/sec (4609.45 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 4395.92 3446.39 3969.01 393.51 60.02471 0 48 6 3 1 0 1 0 0 1 26843545600 1048576 216919965696 POSIX 0
Finished: Wed Oct 24 04:53:18 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write.00000006
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12427694 0xbda1ae 0
2 12574036 0xbfdd54 0
4 12515765 0xbef9b5 0
7 12464391 0xbe3107 0
1 12645680 0xc0f530 0
3 12493976 0xbea498 0
5 12386045 0xbcfefd 0
6 12599289 0xc03ff9 0
/esfs/jtacquaviva/ioperf/file_write.00000023
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12645678 0xc0f52e 0
0 12427691 0xbda1ab 0
4 12515763 0xbef9b3 0
3 12493975 0xbea497 0
2 12574035 0xbfdd53 0
7 12464390 0xbe3106 0
6 12599288 0xc03ff8 0
5 12386044 0xbcfefc 0
/esfs/jtacquaviva/ioperf/file_write.00000044
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464353 0xbe30e1 0
1 12645643 0xc0f50b 0
4 12515729 0xbef991 0
3 12493939 0xbea473 0
6 12599253 0xc03fd5 0
5 12386010 0xbcfeda 0
2 12574000 0xbfdd30 0
0 12427658 0xbda18a 0
/esfs/jtacquaviva/ioperf/file_write.00000040
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12427655 0xbda187 0
7 12464352 0xbe30e0 0
1 12645641 0xc0f509 0
3 12493937 0xbea471 0
6 12599251 0xc03fd3 0
2 12573998 0xbfdd2e 0
5 12386008 0xbcfed8 0
4 12515730 0xbef992 0
/esfs/jtacquaviva/ioperf/file_write.00000002
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464394 0xbe310a 0
1 12645683 0xc0f533 0
3 12493979 0xbea49b 0
5 12386048 0xbcff00 0
6 12599292 0xc03ffc 0
0 12427697 0xbda1b1 0
2 12574039 0xbfdd57 0
4 12515768 0xbef9b8 0
/esfs/jtacquaviva/ioperf/file_write.00000026
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12515767 0xbef9b7 0
7 12464393 0xbe3109 0
1 12645682 0xc0f532 0
3 12493978 0xbea49a 0
5 12386047 0xbcfeff 0
6 12599291 0xc03ffb 0
0 12427696 0xbda1b0 0
2 12574038 0xbfdd56 0
/esfs/jtacquaviva/ioperf/file_write.00000033
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12386021 0xbcfee5 0
6 12599265 0xc03fe1 0
0 12427670 0xbda196 0
2 12574012 0xbfdd3c 0
4 12515741 0xbef99d 0
7 12464367 0xbe30ef 0
1 12645656 0xc0f518 0
3 12493952 0xbea480 0
/esfs/jtacquaviva/ioperf/file_write.00000000
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12386041 0xbcfef9 0
4 12515761 0xbef9b1 0
1 12645677 0xc0f52d 0
6 12599286 0xc03ff6 0
2 12574033 0xbfdd51 0
7 12464388 0xbe3104 0
0 12427692 0xbda1ac 0
3 12493974 0xbea496 0
/esfs/jtacquaviva/ioperf/file_write.00000020
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12493981 0xbea49d 0
5 12386050 0xbcff02 0
6 12599294 0xc03ffe 0
0 12427699 0xbda1b3 0
2 12574041 0xbfdd59 0
4 12515770 0xbef9ba 0
7 12464396 0xbe310c 0
1 12645685 0xc0f535 0
/esfs/jtacquaviva/ioperf/file_write.00000030
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12427686 0xbda1a6 0
2 12574028 0xbfdd4c 0
4 12515757 0xbef9ad 0
7 12464383 0xbe30ff 0
1 12645672 0xc0f528 0
3 12493968 0xbea490 0
5 12386037 0xbcfef5 0
6 12599281 0xc03ff1 0
/esfs/jtacquaviva/ioperf/file_write.00000032
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12493960 0xbea488 0
5 12386029 0xbcfeed 0
6 12599273 0xc03fe9 0
0 12427678 0xbda19e 0
2 12574020 0xbfdd44 0
4 12515749 0xbef9a5 0
7 12464375 0xbe30f7 0
1 12645664 0xc0f520 0
/esfs/jtacquaviva/ioperf/file_write.00000004
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12645651 0xc0f513 0
2 12574007 0xbfdd37 0
7 12464361 0xbe30e9 0
0 12427667 0xbda193 0
5 12386017 0xbcfee1 0
3 12493950 0xbea47e 0
4 12515739 0xbef99b 0
6 12599263 0xc03fdf 0
/esfs/jtacquaviva/ioperf/file_write.00000007
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12599275 0xc03feb 0
0 12427680 0xbda1a0 0
2 12574022 0xbfdd46 0
4 12515751 0xbef9a7 0
7 12464377 0xbe30f9 0
5 12386032 0xbcfef0 0
1 12645666 0xc0f522 0
3 12493963 0xbea48b 0
/esfs/jtacquaviva/ioperf/file_write.00000018
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12645663 0xc0f51f 0
3 12493959 0xbea487 0
5 12386028 0xbcfeec 0
6 12599272 0xc03fe8 0
0 12427677 0xbda19d 0
2 12574019 0xbfdd43 0
4 12515748 0xbef9a4 0
7 12464374 0xbe30f6 0
/esfs/jtacquaviva/ioperf/file_write.00000041
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12386033 0xbcfef1 0
2 12574024 0xbfdd48 0
4 12515753 0xbef9a9 0
7 12464378 0xbe30fa 0
3 12493964 0xbea48c 0
6 12599278 0xc03fee 0
0 12427684 0xbda1a4 0
1 12645669 0xc0f525 0
/esfs/jtacquaviva/ioperf/file_write.00000031
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12386004 0xbcfed4 0
6 12599248 0xc03fd0 0
2 12573995 0xbfdd2b 0
7 12464350 0xbe30de 0
0 12427653 0xbda185 0
1 12645639 0xc0f507 0
4 12515726 0xbef98e 0
3 12493935 0xbea46f 0
/esfs/jtacquaviva/ioperf/file_write.00000012
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12574004 0xbfdd34 0
1 12645649 0xc0f511 0
5 12386016 0xbcfee0 0
0 12427666 0xbda192 0
3 12493948 0xbea47c 0
4 12515737 0xbef999 0
6 12599262 0xc03fde 0
7 12464364 0xbe30ec 0
/esfs/jtacquaviva/ioperf/file_write.00000013
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12599266 0xc03fe2 0
0 12427671 0xbda197 0
2 12574013 0xbfdd3d 0
1 12645657 0xc0f519 0
3 12493953 0xbea481 0
5 12386022 0xbcfee6 0
4 12515742 0xbef99e 0
7 12464369 0xbe30f1 0
/esfs/jtacquaviva/ioperf/file_write.00000042
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12645658 0xc0f51a 0
6 12599268 0xc03fe4 0
0 12427674 0xbda19a 0
4 12515745 0xbef9a1 0
2 12574015 0xbfdd3f 0
7 12464372 0xbe30f4 0
3 12493957 0xbea485 0
5 12386025 0xbcfee9 0
/esfs/jtacquaviva/ioperf/file_write.00000025
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12427673 0xbda199 0
7 12464370 0xbe30f2 0
3 12493955 0xbea483 0
6 12599269 0xc03fe5 0
2 12574016 0xbfdd40 0
1 12645661 0xc0f51d 0
5 12386026 0xbcfeea 0
4 12515746 0xbef9a2 0
/esfs/jtacquaviva/ioperf/file_write.00000046
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12493962 0xbea48a 0
5 12386031 0xbcfeef 0
6 12599276 0xc03fec 0
0 12427681 0xbda1a1 0
2 12574023 0xbfdd47 0
4 12515752 0xbef9a8 0
7 12464379 0xbe30fb 0
1 12645668 0xc0f524 0
/esfs/jtacquaviva/ioperf/file_write.00000039
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12493973 0xbea495 0
5 12386043 0xbcfefb 0
6 12599287 0xc03ff7 0
2 12574034 0xbfdd52 0
7 12464389 0xbe3105 0
0 12427693 0xbda1ad 0
1 12645679 0xc0f52f 0
4 12515764 0xbef9b4 0
/esfs/jtacquaviva/ioperf/file_write.00000022
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573997 0xbfdd2d 0
0 12427657 0xbda189 0
1 12645644 0xc0f50c 0
5 12386009 0xbcfed9 0
3 12493940 0xbea474 0
4 12515731 0xbef993 0
6 12599254 0xc03fd6 0
7 12464355 0xbe30e3 0
/esfs/jtacquaviva/ioperf/file_write.00000034
lmm_stripe_count: 6
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464359 0xbe30e7 0
3 12493944 0xbea478 0
6 12599258 0xc03fda 0
2 12574005 0xbfdd35 0
1 12645648 0xc0f510 0
0 12427663 0xbda18f 0
/esfs/jtacquaviva/ioperf/file_write.00000003
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12493941 0xbea475 0
5 12386011 0xbcfedb 0
6 12599255 0xc03fd7 0
0 12427659 0xbda18b 0
2 12574001 0xbfdd31 0
4 12515732 0xbef994 0
7 12464356 0xbe30e4 0
1 12645645 0xc0f50d 0
/esfs/jtacquaviva/ioperf/file_write.00000008
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12515743 0xbef99f 0
1 12645660 0xc0f51c 0
5 12386024 0xbcfee8 0
7 12464371 0xbe30f3 0
3 12493956 0xbea484 0
6 12599270 0xc03fe6 0
0 12427675 0xbda19b 0
2 12574017 0xbfdd41 0
/esfs/jtacquaviva/ioperf/file_write.00000038
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464360 0xbe30e8 0
0 12427662 0xbda18e 0
5 12386015 0xbcfedf 0
6 12599261 0xc03fdd 0
3 12493947 0xbea47b 0
1 12645653 0xc0f515 0
2 12574010 0xbfdd3a 0
/esfs/jtacquaviva/ioperf/file_write.00000028
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12427682 0xbda1a2 0
1 12645667 0xc0f523 0
5 12386035 0xbcfef3 0
2 12574025 0xbfdd49 0
4 12515754 0xbef9aa 0
6 12599279 0xc03fef 0
7 12464380 0xbe30fc 0
3 12493965 0xbea48d 0
/esfs/jtacquaviva/ioperf/file_write.00000010
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12515736 0xbef998 0
3 12493946 0xbea47a 0
2 12574008 0xbfdd38 0
0 12427668 0xbda194 0
5 12386018 0xbcfee2 0
7 12464365 0xbe30ed 0
1 12645654 0xc0f516 0
/esfs/jtacquaviva/ioperf/file_write.00000043
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12515735 0xbef997 0
6 12599259 0xc03fdb 0
5 12386014 0xbcfede 0
1 12645650 0xc0f512 0
2 12574006 0xbfdd36 0
0 12427664 0xbda190 0
7 12464362 0xbe30ea 0
3 12493949 0xbea47d 0
/esfs/jtacquaviva/ioperf/file_write.00000036
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464368 0xbe30f0 0
6 12599267 0xc03fe3 0
0 12427672 0xbda198 0
3 12493954 0xbea482 0
1 12645659 0xc0f51b 0
5 12386023 0xbcfee7 0
2 12574014 0xbfdd3e 0
4 12515744 0xbef9a0 0
/esfs/jtacquaviva/ioperf/file_write.00000017
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12574029 0xbfdd4d 0
4 12515758 0xbef9ae 0
7 12464384 0xbe3100 0
1 12645673 0xc0f529 0
3 12493969 0xbea491 0
5 12386038 0xbcfef6 0
6 12599282 0xc03ff2 0
0 12427687 0xbda1a7 0
/esfs/jtacquaviva/ioperf/file_write.00000029
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12515728 0xbef990 0
5 12386007 0xbcfed7 0
0 12427656 0xbda188 0
1 12645642 0xc0f50a 0
6 12599252 0xc03fd4 0
3 12493938 0xbea472 0
7 12464354 0xbe30e2 0
2 12573999 0xbfdd2f 0
/esfs/jtacquaviva/ioperf/file_write.00000016
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12386030 0xbcfeee 0
6 12599274 0xc03fea 0
0 12427679 0xbda19f 0
2 12574021 0xbfdd45 0
4 12515750 0xbef9a6 0
7 12464376 0xbe30f8 0
1 12645665 0xc0f521 0
3 12493961 0xbea489 0
/esfs/jtacquaviva/ioperf/file_write.00000005
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464373 0xbe30f5 0
1 12645662 0xc0f51e 0
3 12493958 0xbea486 0
5 12386027 0xbcfeeb 0
6 12599271 0xc03fe7 0
0 12427676 0xbda19c 0
2 12574018 0xbfdd42 0
4 12515747 0xbef9a3 0
/esfs/jtacquaviva/ioperf/file_write.00000014
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12574002 0xbfdd32 0
3 12493942 0xbea476 0
0 12427660 0xbda18c 0
4 12515734 0xbef996 0
7 12464358 0xbe30e6 0
5 12386013 0xbcfedd 0
1 12645647 0xc0f50f 0
6 12599257 0xc03fd9 0
/esfs/jtacquaviva/ioperf/file_write.00000021
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12599280 0xc03ff0 0
0 12427685 0xbda1a5 0
2 12574027 0xbfdd4b 0
4 12515756 0xbef9ac 0
7 12464382 0xbe30fe 0
1 12645671 0xc0f527 0
3 12493967 0xbea48f 0
5 12386036 0xbcfef4 0
/esfs/jtacquaviva/ioperf/file_write.00000045
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12574037 0xbfdd55 0
4 12515766 0xbef9b6 0
7 12464392 0xbe3108 0
1 12645681 0xc0f531 0
3 12493977 0xbea499 0
5 12386046 0xbcfefe 0
6 12599290 0xc03ffa 0
0 12427695 0xbda1af 0
/esfs/jtacquaviva/ioperf/file_write.00000019
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12493951 0xbea47f 0
5 12386020 0xbcfee4 0
6 12599264 0xc03fe0 0
0 12427669 0xbda195 0
2 12574011 0xbfdd3b 0
4 12515740 0xbef99c 0
7 12464366 0xbe30ee 0
1 12645655 0xc0f517 0
/esfs/jtacquaviva/ioperf/file_write.00000047
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12464386 0xbe3102 0
1 12645675 0xc0f52b 0
3 12493971 0xbea493 0
5 12386040 0xbcfef8 0
6 12599284 0xc03ff4 0
0 12427689 0xbda1a9 0
2 12574031 0xbfdd4f 0
4 12515760 0xbef9b0 0
/esfs/jtacquaviva/ioperf/file_write.00000035
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12386012 0xbcfedc 0
6 12599256 0xc03fd8 0
4 12515733 0xbef995 0
7 12464357 0xbe30e5 0
1 12645646 0xc0f50e 0
2 12574003 0xbfdd33 0
0 12427661 0xbda18d 0
3 12493943 0xbea477 0
/esfs/jtacquaviva/ioperf/file_write.00000009
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12515759 0xbef9af 0
7 12464385 0xbe3101 0
1 12645674 0xc0f52a 0
3 12493970 0xbea492 0
5 12386039 0xbcfef7 0
6 12599283 0xc03ff3 0
0 12427688 0xbda1a8 0
2 12574030 0xbfdd4e 0
/esfs/jtacquaviva/ioperf/file_write.00000015
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12645638 0xc0f506 0
0 12427652 0xbda184 0
4 12515725 0xbef98d 0
3 12493934 0xbea46e 0
2 12573994 0xbfdd2a 0
7 12464349 0xbe30dd 0
5 12386005 0xbcfed5 0
6 12599249 0xc03fd1 0
/esfs/jtacquaviva/ioperf/file_write.00000027
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12599260 0xc03fdc 0
3 12493945 0xbea479 0
1 12645652 0xc0f514 0
0 12427665 0xbda191 0
2 12574009 0xbfdd39 0
7 12464363 0xbe30eb 0
4 12515738 0xbef99a 0
5 12386019 0xbcfee3 0
/esfs/jtacquaviva/ioperf/file_write.00000024
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12645676 0xc0f52c 0
6 12599285 0xc03ff5 0
0 12427690 0xbda1aa 0
2 12574032 0xbfdd50 0
7 12464387 0xbe3103 0
3 12493972 0xbea494 0
5 12386042 0xbcfefa 0
4 12515762 0xbef9b2 0
/esfs/jtacquaviva/ioperf/file_write.00000001
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12386034 0xbcfef2 0
6 12599277 0xc03fed 0
0 12427683 0xbda1a3 0
2 12574026 0xbfdd4a 0
4 12515755 0xbef9ab 0
7 12464381 0xbe30fd 0
1 12645670 0xc0f526 0
3 12493966 0xbea48e 0
/esfs/jtacquaviva/ioperf/file_write.00000037
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12645684 0xc0f534 0
3 12493980 0xbea49c 0
5 12386049 0xbcff01 0
6 12599293 0xc03ffd 0
0 12427698 0xbda1b2 0
2 12574040 0xbfdd58 0
4 12515769 0xbef9b9 0
7 12464395 0xbe310b 0
/esfs/jtacquaviva/ioperf/file_write.00000011
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12599250 0xc03fd2 0
0 12427654 0xbda186 0
2 12573996 0xbfdd2c 0
4 12515727 0xbef98f 0
7 12464351 0xbe30df 0
1 12645640 0xc0f508 0
3 12493936 0xbea470 0
5 12386006 0xbcfed6 0

View File

@ -0,0 +1,856 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 48 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 26843545600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:6#API:POSIX#T:10485760.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 03:39:20 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 26843545600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 03:39:20 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 48 (6 per node)
repetitions = 3
xfersize = 10 MiB
blocksize = 25 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1283750625280.
WARNING: Using actual aggregate bytes moved = 394715463680.
write 5547 26214400 10240 0.002594 67.86 0.000553 67.87 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1287116554240.
WARNING: Using actual aggregate bytes moved = 394400890880.
write 5602 26214400 10240 0.007060 67.13 0.000728 67.14 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1287756185600.
WARNING: Using actual aggregate bytes moved = 391370506240.
write 5491 26214400 10240 0.003122 67.97 0.000932 67.98 2
Max Write: 5601.99 MiB/sec (5874.11 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 5601.99 5490.81 5546.45 45.39 67.66164 0 48 6 3 1 0 1 0 0 1 26843545600 10485760 394715463680 POSIX 0
Finished: Wed Oct 24 03:42:48 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 48 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 48 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 26843545600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
+ tee -a ./output/COUNT:1#NN:8#PPN:6#API:POSIX#T:10485760.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 03:43:12 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 26843545600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 03:43:12 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/indread8/file
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 48 (6 per node)
repetitions = 3
xfersize = 10 MiB
blocksize = 25 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 7730941132800.
WARNING: Using actual aggregate bytes moved = 241182965760.
read 3825 26214400 10240 0.005837 60.13 0.005007 60.13 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 7730941132800.
WARNING: Using actual aggregate bytes moved = 280158535680.
read 4443 26214400 10240 0.005055 60.13 0.004936 60.13 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 7730941132800.
WARNING: Using actual aggregate bytes moved = 293894881280.
read 4660 26214400 10240 0.005025 60.14 0.004984 60.14 2
Max Read: 4660.37 MiB/sec (4886.75 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 4660.37 3825.06 4309.60 353.89 60.13453 0 48 6 3 1 0 1 0 0 1 26843545600 10485760 241182965760 POSIX 0
Finished: Wed Oct 24 03:46:12 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write.00000006
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12463843 0xbe2ee3 0
6 12598742 0xc03dd6 0
1 12645133 0xc0f30d 0
4 12515223 0xbef797 0
5 12385496 0xbcfcd8 0
3 12493433 0xbea279 0
2 12573490 0xbfdb32 0
0 12427152 0xbd9f90 0
/esfs/jtacquaviva/ioperf/file_write.00000023
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12385529 0xbcfcf9 0
6 12598775 0xc03df7 0
0 12427183 0xbd9faf 0
2 12573521 0xbfdb51 0
4 12515255 0xbef7b7 0
7 12463877 0xbe2f05 0
1 12645166 0xc0f32e 0
3 12493465 0xbea299 0
/esfs/jtacquaviva/ioperf/file_write.00000044
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573494 0xbfdb36 0
7 12463852 0xbe2eec 0
3 12493441 0xbea281 0
1 12645141 0xc0f315 0
0 12427158 0xbd9f96 0
4 12515232 0xbef7a0 0
6 12598752 0xc03de0 0
5 12385506 0xbcfce2 0
/esfs/jtacquaviva/ioperf/file_write.00000040
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12598768 0xc03df0 0
0 12427176 0xbd9fa8 0
2 12573514 0xbfdb4a 0
4 12515247 0xbef7af 0
7 12463869 0xbe2efd 0
1 12645158 0xc0f326 0
3 12493457 0xbea291 0
5 12385522 0xbcfcf2 0
/esfs/jtacquaviva/ioperf/file_write.00000002
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12493455 0xbea28f 0
5 12385519 0xbcfcef 0
6 12598765 0xc03ded 0
0 12427174 0xbd9fa6 0
2 12573512 0xbfdb48 0
4 12515246 0xbef7ae 0
7 12463868 0xbe2efc 0
1 12645157 0xc0f325 0
/esfs/jtacquaviva/ioperf/file_write.00000026
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573510 0xbfdb46 0
4 12515244 0xbef7ac 0
3 12493454 0xbea28e 0
5 12385518 0xbcfcee 0
6 12598764 0xc03dec 0
7 12463866 0xbe2efa 0
0 12427173 0xbd9fa5 0
1 12645156 0xc0f324 0
/esfs/jtacquaviva/ioperf/file_write.00000033
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12427171 0xbd9fa3 0
2 12573509 0xbfdb45 0
4 12515243 0xbef7ab 0
7 12463865 0xbe2ef9 0
1 12645154 0xc0f322 0
3 12493453 0xbea28d 0
5 12385517 0xbcfced 0
6 12598763 0xc03deb 0
/esfs/jtacquaviva/ioperf/file_write.00000000
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573484 0xbfdb2c 0
7 12463840 0xbe2ee0 0
0 12427147 0xbd9f8b 0
1 12645130 0xc0f30a 0
4 12515221 0xbef795 0
5 12385494 0xbcfcd6 0
3 12493430 0xbea276 0
6 12598741 0xc03dd5 0
/esfs/jtacquaviva/ioperf/file_write.00000020
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12493437 0xbea27d 0
2 12573496 0xbfdb38 0
5 12385504 0xbcfce0 0
6 12598751 0xc03ddf 0
1 12645143 0xc0f317 0
7 12463854 0xbe2eee 0
0 12427160 0xbd9f98 0
/esfs/jtacquaviva/ioperf/file_write.00000030
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573501 0xbfdb3d 0
4 12515235 0xbef7a3 0
7 12463857 0xbe2ef1 0
5 12385509 0xbcfce5 0
6 12598755 0xc03de3 0
1 12645146 0xc0f31a 0
3 12493446 0xbea286 0
0 12427164 0xbd9f9c 0
/esfs/jtacquaviva/ioperf/file_write.00000032
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12515228 0xbef79c 0
5 12385501 0xbcfcdd 0
1 12645138 0xc0f312 0
6 12598749 0xc03ddd 0
0 12427157 0xbd9f95 0
3 12493438 0xbea27e 0
7 12463853 0xbe2eed 0
2 12573497 0xbfdb39 0
/esfs/jtacquaviva/ioperf/file_write.00000004
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573503 0xbfdb3f 0
1 12645149 0xc0f31d 0
5 12385511 0xbcfce7 0
0 12427165 0xbd9f9d 0
4 12515237 0xbef7a5 0
3 12493449 0xbea289 0
6 12598759 0xc03de7 0
7 12463860 0xbe2ef4 0
/esfs/jtacquaviva/ioperf/file_write.00000007
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12427161 0xbd9f99 0
2 12573499 0xbfdb3b 0
4 12515233 0xbef7a1 0
7 12463855 0xbe2eef 0
1 12645144 0xc0f318 0
3 12493443 0xbea283 0
5 12385508 0xbcfce4 0
6 12598754 0xc03de2 0
/esfs/jtacquaviva/ioperf/file_write.00000018
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12645148 0xc0f31c 0
3 12493448 0xbea288 0
5 12385512 0xbcfce8 0
6 12598758 0xc03de6 0
0 12427166 0xbd9f9e 0
2 12573505 0xbfdb41 0
4 12515239 0xbef7a7 0
7 12463861 0xbe2ef5 0
/esfs/jtacquaviva/ioperf/file_write.00000041
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12463851 0xbe2eeb 0
4 12515231 0xbef79f 0
1 12645142 0xc0f316 0
6 12598750 0xc03dde 0
5 12385505 0xbcfce1 0
2 12573498 0xbfdb3a 0
0 12427159 0xbd9f97 0
3 12493442 0xbea282 0
/esfs/jtacquaviva/ioperf/file_write.00000031
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573487 0xbfdb2f 0
1 12645132 0xc0f30c 0
3 12493431 0xbea277 0
5 12385495 0xbcfcd7 0
4 12515222 0xbef796 0
6 12598743 0xc03dd7 0
0 12427149 0xbd9f8d 0
7 12463845 0xbe2ee5 0
/esfs/jtacquaviva/ioperf/file_write.00000012
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12427177 0xbd9fa9 0
2 12573515 0xbfdb4b 0
4 12515249 0xbef7b1 0
7 12463871 0xbe2eff 0
1 12645160 0xc0f328 0
3 12493459 0xbea293 0
5 12385523 0xbcfcf3 0
6 12598769 0xc03df1 0
/esfs/jtacquaviva/ioperf/file_write.00000013
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12385515 0xbcfceb 0
6 12598761 0xc03de9 0
0 12427169 0xbd9fa1 0
2 12573507 0xbfdb43 0
4 12515241 0xbef7a9 0
7 12463863 0xbe2ef7 0
1 12645152 0xc0f320 0
3 12493451 0xbea28b 0
/esfs/jtacquaviva/ioperf/file_write.00000042
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12463874 0xbe2f02 0
1 12645163 0xc0f32b 0
3 12493462 0xbea296 0
5 12385526 0xbcfcf6 0
6 12598772 0xc03df4 0
0 12427180 0xbd9fac 0
2 12573518 0xbfdb4e 0
4 12515252 0xbef7b4 0
/esfs/jtacquaviva/ioperf/file_write.00000025
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12427145 0xbd9f89 0
5 12385493 0xbcfcd5 0
4 12515219 0xbef793 0
1 12645129 0xc0f309 0
2 12573486 0xbfdb2e 0
3 12493429 0xbea275 0
7 12463842 0xbe2ee2 0
6 12598740 0xc03dd4 0
/esfs/jtacquaviva/ioperf/file_write.00000046
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12645155 0xc0f323 0
0 12427172 0xbd9fa4 0
4 12515245 0xbef7ad 0
3 12493456 0xbea290 0
5 12385520 0xbcfcf0 0
2 12573511 0xbfdb47 0
6 12598767 0xc03def 0
7 12463867 0xbe2efb 0
/esfs/jtacquaviva/ioperf/file_write.00000039
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573483 0xbfdb2b 0
4 12515218 0xbef792 0
7 12463839 0xbe2edf 0
5 12385491 0xbcfcd3 0
1 12645128 0xc0f308 0
3 12493428 0xbea274 0
6 12598738 0xc03dd2 0
0 12427146 0xbd9f8a 0
/esfs/jtacquaviva/ioperf/file_write.00000022
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12493427 0xbea273 0
5 12385492 0xbcfcd4 0
6 12598739 0xc03dd3 0
2 12573485 0xbfdb2d 0
4 12515220 0xbef794 0
7 12463841 0xbe2ee1 0
0 12427148 0xbd9f8c 0
1 12645131 0xc0f30b 0
/esfs/jtacquaviva/ioperf/file_write.00000034
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12515251 0xbef7b3 0
7 12463873 0xbe2f01 0
1 12645162 0xc0f32a 0
3 12493461 0xbea295 0
5 12385525 0xbcfcf5 0
6 12598771 0xc03df3 0
0 12427179 0xbd9fab 0
2 12573517 0xbfdb4d 0
/esfs/jtacquaviva/ioperf/file_write.00000003
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12493464 0xbea298 0
5 12385528 0xbcfcf8 0
6 12598774 0xc03df6 0
0 12427182 0xbd9fae 0
2 12573520 0xbfdb50 0
4 12515254 0xbef7b6 0
7 12463876 0xbe2f04 0
1 12645165 0xc0f32d 0
/esfs/jtacquaviva/ioperf/file_write.00000008
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573479 0xbfdb27 0
3 12493424 0xbea270 0
4 12515215 0xbef78f 0
5 12385489 0xbcfcd1 0
7 12463836 0xbe2edc 0
1 12645126 0xc0f306 0
0 12427143 0xbd9f87 0
/esfs/jtacquaviva/ioperf/file_write.00000038
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12598776 0xc03df8 0
0 12427184 0xbd9fb0 0
2 12573522 0xbfdb52 0
4 12515256 0xbef7b8 0
7 12463878 0xbe2f06 0
1 12645167 0xc0f32f 0
3 12493466 0xbea29a 0
5 12385530 0xbcfcfa 0
/esfs/jtacquaviva/ioperf/file_write.00000028
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12645164 0xc0f32c 0
3 12493463 0xbea297 0
5 12385527 0xbcfcf7 0
6 12598773 0xc03df5 0
0 12427181 0xbd9fad 0
2 12573519 0xbfdb4f 0
4 12515253 0xbef7b5 0
7 12463875 0xbe2f03 0
/esfs/jtacquaviva/ioperf/file_write.00000010
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573516 0xbfdb4c 0
4 12515250 0xbef7b2 0
7 12463872 0xbe2f00 0
1 12645161 0xc0f329 0
3 12493460 0xbea294 0
5 12385524 0xbcfcf4 0
6 12598770 0xc03df2 0
0 12427178 0xbd9faa 0
/esfs/jtacquaviva/ioperf/file_write.00000043
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12493423 0xbea26f 0
7 12463835 0xbe2edb 0
5 12385488 0xbcfcd0 0
1 12645124 0xc0f304 0
6 12598736 0xc03dd0 0
2 12573480 0xbfdb28 0
0 12427142 0xbd9f86 0
4 12515216 0xbef790 0
/esfs/jtacquaviva/ioperf/file_write.00000036
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12515259 0xbef7bb 0
7 12463881 0xbe2f09 0
1 12645170 0xc0f332 0
3 12493469 0xbea29d 0
0 12427187 0xbd9fb3 0
2 12573525 0xbfdb55 0
6 12598780 0xc03dfc 0
5 12385534 0xbcfcfe 0
/esfs/jtacquaviva/ioperf/file_write.00000017
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12598753 0xc03de1 0
0 12427162 0xbd9f9a 0
2 12573500 0xbfdb3c 0
4 12515234 0xbef7a2 0
7 12463856 0xbe2ef0 0
1 12645145 0xc0f319 0
3 12493444 0xbea284 0
5 12385507 0xbcfce3 0
/esfs/jtacquaviva/ioperf/file_write.00000029
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573493 0xbfdb35 0
6 12598748 0xc03ddc 0
4 12515230 0xbef79e 0
0 12427156 0xbd9f94 0
7 12463850 0xbe2eea 0
5 12385503 0xbcfcdf 0
1 12645139 0xc0f313 0
3 12493439 0xbea27f 0
/esfs/jtacquaviva/ioperf/file_write.00000016
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12598735 0xc03dcf 0
4 12515214 0xbef78e 0
1 12645125 0xc0f305 0
0 12427141 0xbd9f85 0
3 12493425 0xbea271 0
2 12573481 0xbfdb29 0
7 12463837 0xbe2edd 0
/esfs/jtacquaviva/ioperf/file_write.00000005
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12598747 0xc03ddb 0
4 12515229 0xbef79d 0
5 12385502 0xbcfcde 0
0 12427155 0xbd9f93 0
7 12463849 0xbe2ee9 0
2 12573495 0xbfdb37 0
1 12645140 0xc0f314 0
3 12493440 0xbea280 0
/esfs/jtacquaviva/ioperf/file_write.00000014
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573489 0xbfdb31 0
7 12463844 0xbe2ee4 0
0 12427150 0xbd9f8e 0
5 12385497 0xbcfcd9 0
4 12515225 0xbef799 0
6 12598744 0xc03dd8 0
3 12493434 0xbea27a 0
1 12645135 0xc0f30f 0
/esfs/jtacquaviva/ioperf/file_write.00000021
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12385521 0xbcfcf1 0
6 12598766 0xc03dee 0
0 12427175 0xbd9fa7 0
2 12573513 0xbfdb49 0
4 12515248 0xbef7b0 0
7 12463870 0xbe2efe 0
1 12645159 0xc0f327 0
3 12493458 0xbea292 0
/esfs/jtacquaviva/ioperf/file_write.00000045
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573488 0xbfdb30 0
3 12493432 0xbea278 0
4 12515224 0xbef798 0
5 12385498 0xbcfcda 0
0 12427151 0xbd9f8f 0
7 12463846 0xbe2ee6 0
1 12645134 0xc0f30e 0
/esfs/jtacquaviva/ioperf/file_write.00000019
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12598779 0xc03dfb 0
5 12385533 0xbcfcfd 0
0 12427188 0xbd9fb4 0
2 12573526 0xbfdb56 0
4 12515260 0xbef7bc 0
7 12463882 0xbe2f0a 0
1 12645171 0xc0f333 0
3 12493470 0xbea29e 0
/esfs/jtacquaviva/ioperf/file_write.00000047
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12463859 0xbe2ef3 0
3 12493447 0xbea287 0
6 12598756 0xc03de4 0
2 12573504 0xbfdb40 0
1 12645150 0xc0f31e 0
5 12385513 0xbcfce9 0
0 12427167 0xbd9f9f 0
4 12515238 0xbef7a6 0
/esfs/jtacquaviva/ioperf/file_write.00000035
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12427144 0xbd9f88 0
2 12573482 0xbfdb2a 0
4 12515217 0xbef791 0
7 12463838 0xbe2ede 0
1 12645127 0xc0f307 0
3 12493426 0xbea272 0
5 12385490 0xbcfcd2 0
6 12598737 0xc03dd1 0
/esfs/jtacquaviva/ioperf/file_write.00000009
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12598762 0xc03dea 0
0 12427170 0xbd9fa2 0
2 12573508 0xbfdb44 0
4 12515242 0xbef7aa 0
7 12463864 0xbe2ef8 0
1 12645153 0xc0f321 0
3 12493452 0xbea28c 0
5 12385516 0xbcfcec 0
/esfs/jtacquaviva/ioperf/file_write.00000015
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12385499 0xbcfcdb 0
2 12573491 0xbfdb33 0
7 12463847 0xbe2ee7 0
3 12493435 0xbea27b 0
0 12427153 0xbd9f91 0
1 12645136 0xc0f310 0
4 12515226 0xbef79a 0
6 12598745 0xc03dd9 0
/esfs/jtacquaviva/ioperf/file_write.00000027
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12493445 0xbea285 0
0 12427163 0xbd9f9b 0
2 12573502 0xbfdb3e 0
4 12515236 0xbef7a4 0
7 12463858 0xbe2ef2 0
1 12645147 0xc0f31b 0
5 12385510 0xbcfce6 0
6 12598757 0xc03de5 0
/esfs/jtacquaviva/ioperf/file_write.00000024
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12427185 0xbd9fb1 0
2 12573523 0xbfdb53 0
4 12515257 0xbef7b9 0
7 12463879 0xbe2f07 0
1 12645168 0xc0f330 0
3 12493467 0xbea29b 0
5 12385531 0xbcfcfb 0
6 12598777 0xc03df9 0
/esfs/jtacquaviva/ioperf/file_write.00000001
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573524 0xbfdb54 0
4 12515258 0xbef7ba 0
7 12463880 0xbe2f08 0
1 12645169 0xc0f331 0
3 12493468 0xbea29c 0
5 12385532 0xbcfcfc 0
6 12598778 0xc03dfa 0
0 12427186 0xbd9fb2 0
/esfs/jtacquaviva/ioperf/file_write.00000037
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12493450 0xbea28a 0
5 12385514 0xbcfcea 0
6 12598760 0xc03de8 0
0 12427168 0xbd9fa0 0
2 12573506 0xbfdb42 0
4 12515240 0xbef7a8 0
7 12463862 0xbe2ef6 0
1 12645151 0xc0f31f 0
/esfs/jtacquaviva/ioperf/file_write.00000011
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12427154 0xbd9f92 0
4 12515227 0xbef79b 0
1 12645137 0xc0f311 0
5 12385500 0xbcfcdc 0
6 12598746 0xc03dda 0
3 12493436 0xbea27c 0
7 12463848 0xbe2ee8 0
2 12573492 0xbfdb34 0

View File

@ -0,0 +1,855 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 48 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 26843545600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:6#API:POSIX#T:16384.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 07:10:23 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 26843545600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 07:10:23 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 48 (6 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 25 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288447180800.
WARNING: Using actual aggregate bytes moved = 31813091328.
write 274.83 26214400 16.00 0.002551 110.39 0.000442 110.39 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288427962368.
WARNING: Using actual aggregate bytes moved = 26790608896.
write 215.96 26214400 16.00 0.002872 118.30 0.000476 118.30 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288349925376.
WARNING: Using actual aggregate bytes moved = 27780775936.
write 214.21 26214400 16.00 0.003518 123.68 0.000549 123.68 2
Max Write: 274.83 MiB/sec (288.18 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 274.83 214.21 235.00 28.17 117.45843 0 48 6 3 1 0 1 0 0 1 26843545600 16384 31813091328 POSIX 0
Finished: Wed Oct 24 07:16:16 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 48 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ tee -a ./output/COUNT:1#NN:8#PPN:6#API:POSIX#T:16384.txt
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 48 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 26843545600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 07:16:22 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 26843545600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread8/file -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 07:16:22 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/indread8/file
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 48 (6 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 25 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 7730941132800.
WARNING: Using actual aggregate bytes moved = 6453608448.
read 102.39 26214400 16.00 0.001573 60.11 0.000848 60.11 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 7730941132800.
WARNING: Using actual aggregate bytes moved = 7719092224.
read 122.48 26214400 16.00 0.001499 60.10 0.000849 60.10 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 7730941132800.
WARNING: Using actual aggregate bytes moved = 7727251456.
read 122.60 26214400 16.00 0.000921 60.11 0.000827 60.11 2
Max Read: 122.60 MiB/sec (128.55 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 122.60 102.39 115.82 9.50 60.10760 0 48 6 3 1 0 1 0 0 1 26843545600 16384 6453608448 POSIX 0
Finished: Wed Oct 24 07:19:22 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write.00000006
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428710 0xbda5a6 0
6 12600305 0xc043f1 0
2 12575059 0xbfe153 0
4 12516784 0xbefdb0 0
7 12465419 0xbe350b 0
1 12646701 0xc0f92d 0
3 12495005 0xbea89d 0
5 12387064 0xbd02f8 0
/esfs/jtacquaviva/ioperf/file_write.00000023
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12600281 0xc043d9 0
1 12646675 0xc0f913 0
3 12494978 0xbea882 0
0 12428685 0xbda58d 0
5 12387038 0xbd02de 0
4 12516760 0xbefd98 0
7 12465393 0xbe34f1 0
2 12575034 0xbfe13a 0
/esfs/jtacquaviva/ioperf/file_write.00000044
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12387058 0xbd02f2 0
6 12600301 0xc043ed 0
0 12428706 0xbda5a2 0
2 12575055 0xbfe14f 0
4 12516780 0xbefdac 0
7 12465413 0xbe3505 0
1 12646696 0xc0f928 0
3 12494999 0xbea897 0
/esfs/jtacquaviva/ioperf/file_write.00000040
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428699 0xbda59b 0
7 12465406 0xbe34fe 0
3 12494992 0xbea890 0
2 12575049 0xbfe149 0
1 12646690 0xc0f922 0
4 12516774 0xbefda6 0
5 12387052 0xbd02ec 0
6 12600295 0xc043e7 0
/esfs/jtacquaviva/ioperf/file_write.00000002
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12575035 0xbfe13b 0
4 12516762 0xbefd9a 0
7 12465395 0xbe34f3 0
3 12494981 0xbea885 0
1 12646677 0xc0f915 0
0 12428688 0xbda590 0
5 12387041 0xbd02e1 0
/esfs/jtacquaviva/ioperf/file_write.00000026
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494979 0xbea883 0
7 12465394 0xbe34f2 0
6 12600283 0xc043db 0
0 12428687 0xbda58f 0
2 12575037 0xbfe13d 0
5 12387040 0xbd02e0 0
1 12646678 0xc0f916 0
/esfs/jtacquaviva/ioperf/file_write.00000033
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12575062 0xbfe156 0
1 12646703 0xc0f92f 0
5 12387065 0xbd02f9 0
0 12428713 0xbda5a9 0
4 12516786 0xbefdb2 0
3 12495007 0xbea89f 0
6 12600309 0xc043f5 0
7 12465421 0xbe350d 0
/esfs/jtacquaviva/ioperf/file_write.00000000
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12387031 0xbd02d7 0
7 12465385 0xbe34e9 0
1 12646668 0xc0f90c 0
3 12494971 0xbea87b 0
0 12428679 0xbda587 0
4 12516753 0xbefd91 0
6 12600274 0xc043d2 0
2 12575027 0xbfe133 0
/esfs/jtacquaviva/ioperf/file_write.00000020
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12465417 0xbe3509 0
5 12387062 0xbd02f6 0
1 12646700 0xc0f92c 0
3 12495003 0xbea89b 0
2 12575060 0xbfe154 0
0 12428711 0xbda5a7 0
6 12600306 0xc043f2 0
/esfs/jtacquaviva/ioperf/file_write.00000030
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12516794 0xbefdba 0
7 12465428 0xbe3514 0
1 12646711 0xc0f937 0
3 12495014 0xbea8a6 0
5 12387073 0xbd0301 0
6 12600316 0xc043fc 0
0 12428721 0xbda5b1 0
2 12575070 0xbfe15e 0
/esfs/jtacquaviva/ioperf/file_write.00000032
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12575023 0xbfe12f 0
1 12646664 0xc0f908 0
5 12387027 0xbd02d3 0
6 12600270 0xc043ce 0
4 12516749 0xbefd8d 0
3 12494968 0xbea878 0
0 12428676 0xbda584 0
7 12465382 0xbe34e6 0
/esfs/jtacquaviva/ioperf/file_write.00000004
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12600287 0xc043df 0
0 12428693 0xbda595 0
7 12465400 0xbe34f8 0
1 12646683 0xc0f91b 0
2 12575042 0xbfe142 0
4 12516768 0xbefda0 0
5 12387048 0xbd02e8 0
3 12494989 0xbea88d 0
/esfs/jtacquaviva/ioperf/file_write.00000007
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12516757 0xbefd95 0
7 12465389 0xbe34ed 0
1 12646672 0xc0f910 0
6 12600278 0xc043d6 0
0 12428682 0xbda58a 0
3 12494975 0xbea87f 0
5 12387036 0xbd02dc 0
2 12575031 0xbfe137 0
/esfs/jtacquaviva/ioperf/file_write.00000018
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12575050 0xbfe14a 0
4 12516775 0xbefda7 0
7 12465408 0xbe3500 0
1 12646691 0xc0f923 0
3 12494994 0xbea892 0
5 12387054 0xbd02ee 0
6 12600297 0xc043e9 0
0 12428701 0xbda59d 0
/esfs/jtacquaviva/ioperf/file_write.00000041
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12516754 0xbefd92 0
6 12600275 0xc043d3 0
2 12575028 0xbfe134 0
1 12646669 0xc0f90d 0
7 12465387 0xbe34eb 0
3 12494972 0xbea87c 0
5 12387033 0xbd02d9 0
/esfs/jtacquaviva/ioperf/file_write.00000031
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12646687 0xc0f91f 0
3 12494990 0xbea88e 0
5 12387049 0xbd02e9 0
6 12600292 0xc043e4 0
0 12428697 0xbda599 0
2 12575046 0xbfe146 0
4 12516771 0xbefda3 0
7 12465404 0xbe34fc 0
/esfs/jtacquaviva/ioperf/file_write.00000012
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494995 0xbea893 0
4 12516777 0xbefda9 0
6 12600298 0xc043ea 0
7 12465410 0xbe3502 0
0 12428703 0xbda59f 0
5 12387056 0xbd02f0 0
2 12575053 0xbfe14d 0
1 12646694 0xc0f926 0
/esfs/jtacquaviva/ioperf/file_write.00000013
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12575058 0xbfe152 0
1 12646699 0xc0f92b 0
3 12495002 0xbea89a 0
4 12516783 0xbefdaf 0
7 12465416 0xbe3508 0
5 12387061 0xbd02f5 0
0 12428709 0xbda5a5 0
6 12600304 0xc043f0 0
/esfs/jtacquaviva/ioperf/file_write.00000042
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12600284 0xc043dc 0
0 12428689 0xbda591 0
2 12575038 0xbfe13e 0
4 12516763 0xbefd9b 0
7 12465396 0xbe34f4 0
1 12646679 0xc0f917 0
3 12494982 0xbea886 0
5 12387042 0xbd02e2 0
/esfs/jtacquaviva/ioperf/file_write.00000025
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428708 0xbda5a4 0
2 12575057 0xbfe151 0
4 12516782 0xbefdae 0
7 12465415 0xbe3507 0
1 12646698 0xc0f92a 0
3 12495001 0xbea899 0
5 12387060 0xbd02f4 0
6 12600303 0xc043ef 0
/esfs/jtacquaviva/ioperf/file_write.00000046
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12600285 0xc043dd 0
0 12428690 0xbda592 0
2 12575039 0xbfe13f 0
4 12516764 0xbefd9c 0
7 12465397 0xbe34f5 0
1 12646680 0xc0f918 0
3 12494983 0xbea887 0
/esfs/jtacquaviva/ioperf/file_write.00000039
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12495008 0xbea8a0 0
0 12428715 0xbda5ab 0
7 12465422 0xbe350e 0
5 12387067 0xbd02fb 0
2 12575065 0xbfe159 0
1 12646706 0xc0f932 0
4 12516789 0xbefdb5 0
6 12600311 0xc043f7 0
/esfs/jtacquaviva/ioperf/file_write.00000022
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12575067 0xbfe15b 0
4 12516791 0xbefdb7 0
7 12465425 0xbe3511 0
1 12646708 0xc0f934 0
3 12495011 0xbea8a3 0
5 12387070 0xbd02fe 0
6 12600313 0xc043f9 0
0 12428719 0xbda5af 0
/esfs/jtacquaviva/ioperf/file_write.00000034
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494998 0xbea896 0
5 12387057 0xbd02f1 0
6 12600300 0xc043ec 0
0 12428705 0xbda5a1 0
2 12575054 0xbfe14e 0
4 12516779 0xbefdab 0
7 12465412 0xbe3504 0
1 12646695 0xc0f927 0
/esfs/jtacquaviva/ioperf/file_write.00000003
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12465418 0xbe350a 0
3 12495004 0xbea89c 0
2 12575061 0xbfe155 0
4 12516785 0xbefdb1 0
0 12428712 0xbda5a8 0
5 12387063 0xbd02f7 0
1 12646702 0xc0f92e 0
6 12600307 0xc043f3 0
/esfs/jtacquaviva/ioperf/file_write.00000008
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12646666 0xc0f90a 0
3 12494969 0xbea879 0
5 12387029 0xbd02d5 0
6 12600272 0xc043d0 0
0 12428677 0xbda585 0
2 12575025 0xbfe131 0
4 12516751 0xbefd8f 0
7 12465383 0xbe34e7 0
/esfs/jtacquaviva/ioperf/file_write.00000038
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12600302 0xc043ee 0
0 12428707 0xbda5a3 0
2 12575056 0xbfe150 0
4 12516781 0xbefdad 0
7 12465414 0xbe3506 0
1 12646697 0xc0f929 0
3 12495000 0xbea898 0
5 12387059 0xbd02f3 0
/esfs/jtacquaviva/ioperf/file_write.00000028
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494984 0xbea888 0
0 12428691 0xbda593 0
5 12387043 0xbd02e3 0
2 12575041 0xbfe141 0
7 12465398 0xbe34f6 0
6 12600288 0xc043e0 0
1 12646682 0xc0f91a 0
4 12516767 0xbefd9f 0
/esfs/jtacquaviva/ioperf/file_write.00000010
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428718 0xbda5ae 0
2 12575068 0xbfe15c 0
4 12516792 0xbefdb8 0
7 12465426 0xbe3512 0
1 12646709 0xc0f935 0
3 12495012 0xbea8a4 0
5 12387071 0xbd02ff 0
6 12600314 0xc043fa 0
/esfs/jtacquaviva/ioperf/file_write.00000043
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494970 0xbea87a 0
5 12387030 0xbd02d6 0
6 12600273 0xc043d1 0
0 12428678 0xbda586 0
2 12575026 0xbfe132 0
4 12516752 0xbefd90 0
7 12465384 0xbe34e8 0
1 12646667 0xc0f90b 0
/esfs/jtacquaviva/ioperf/file_write.00000036
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12600286 0xc043de 0
2 12575040 0xbfe140 0
0 12428692 0xbda594 0
5 12387044 0xbd02e4 0
4 12516765 0xbefd9d 0
1 12646681 0xc0f919 0
7 12465399 0xbe34f7 0
3 12494987 0xbea88b 0
/esfs/jtacquaviva/ioperf/file_write.00000017
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12387035 0xbd02db 0
6 12600279 0xc043d7 0
0 12428683 0xbda58b 0
1 12646673 0xc0f911 0
2 12575032 0xbfe138 0
3 12494976 0xbea880 0
4 12516759 0xbefd97 0
7 12465391 0xbe34ef 0
/esfs/jtacquaviva/ioperf/file_write.00000029
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12387051 0xbd02eb 0
2 12575048 0xbfe148 0
4 12516773 0xbefda5 0
1 12646689 0xc0f921 0
6 12600294 0xc043e6 0
7 12465407 0xbe34ff 0
0 12428700 0xbda59c 0
3 12494993 0xbea891 0
/esfs/jtacquaviva/ioperf/file_write.00000016
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12465381 0xbe34e5 0
3 12494967 0xbea877 0
0 12428675 0xbda583 0
1 12646665 0xc0f909 0
6 12600271 0xc043cf 0
2 12575024 0xbfe130 0
5 12387028 0xbd02d4 0
4 12516750 0xbefd8e 0
/esfs/jtacquaviva/ioperf/file_write.00000005
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12516776 0xbefda8 0
7 12465409 0xbe3501 0
5 12387053 0xbd02ed 0
6 12600296 0xc043e8 0
1 12646692 0xc0f924 0
0 12428702 0xbda59e 0
3 12494996 0xbea894 0
2 12575052 0xbfe14c 0
/esfs/jtacquaviva/ioperf/file_write.00000014
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428717 0xbda5ad 0
2 12575066 0xbfe15a 0
4 12516790 0xbefdb6 0
7 12465424 0xbe3510 0
1 12646707 0xc0f933 0
3 12495010 0xbea8a2 0
5 12387069 0xbd02fd 0
6 12600312 0xc043f8 0
/esfs/jtacquaviva/ioperf/file_write.00000021
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494985 0xbea889 0
5 12387046 0xbd02e6 0
7 12465401 0xbe34f9 0
4 12516766 0xbefd9e 0
6 12600290 0xc043e2 0
0 12428696 0xbda598 0
1 12646685 0xc0f91d 0
2 12575044 0xbfe144 0
/esfs/jtacquaviva/ioperf/file_write.00000045
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12465420 0xbe350c 0
3 12495006 0xbea89e 0
6 12600308 0xc043f4 0
2 12575063 0xbfe157 0
1 12646704 0xc0f930 0
5 12387066 0xbd02fa 0
0 12428714 0xbda5aa 0
4 12516787 0xbefdb3 0
/esfs/jtacquaviva/ioperf/file_write.00000019
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12428680 0xbda588 0
5 12387032 0xbd02d8 0
7 12465386 0xbe34ea 0
4 12516755 0xbefd93 0
3 12494973 0xbea87d 0
2 12575029 0xbfe135 0
1 12646670 0xc0f90e 0
6 12600276 0xc043d4 0
/esfs/jtacquaviva/ioperf/file_write.00000047
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494991 0xbea88f 0
5 12387050 0xbd02ea 0
6 12600293 0xc043e5 0
0 12428698 0xbda59a 0
2 12575047 0xbfe147 0
4 12516772 0xbefda4 0
7 12465405 0xbe34fd 0
1 12646688 0xc0f920 0
/esfs/jtacquaviva/ioperf/file_write.00000035
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12516758 0xbefd96 0
7 12465390 0xbe34ee 0
3 12494977 0xbea881 0
1 12646674 0xc0f912 0
5 12387037 0xbd02dd 0
0 12428684 0xbda58c 0
6 12600280 0xc043d8 0
2 12575033 0xbfe139 0
/esfs/jtacquaviva/ioperf/file_write.00000009
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12575051 0xbfe14b 0
4 12516778 0xbefdaa 0
5 12387055 0xbd02ef 0
1 12646693 0xc0f925 0
6 12600299 0xc043eb 0
3 12494997 0xbea895 0
0 12428704 0xbda5a0 0
7 12465411 0xbe3503 0
/esfs/jtacquaviva/ioperf/file_write.00000015
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12494986 0xbea88a 0
5 12387047 0xbd02e7 0
7 12465402 0xbe34fa 0
0 12428695 0xbda597 0
1 12646684 0xc0f91c 0
6 12600291 0xc043e3 0
2 12575045 0xbfe145 0
4 12516769 0xbefda1 0
/esfs/jtacquaviva/ioperf/file_write.00000027
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12600310 0xc043f6 0
2 12575064 0xbfe158 0
4 12516788 0xbefdb4 0
1 12646705 0xc0f931 0
0 12428716 0xbda5ac 0
7 12465423 0xbe350f 0
3 12495009 0xbea8a1 0
5 12387068 0xbd02fc 0
/esfs/jtacquaviva/ioperf/file_write.00000024
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12575030 0xbfe136 0
4 12516756 0xbefd94 0
7 12465388 0xbe34ec 0
1 12646671 0xc0f90f 0
3 12494974 0xbea87e 0
5 12387034 0xbd02da 0
6 12600277 0xc043d5 0
0 12428681 0xbda589 0
/esfs/jtacquaviva/ioperf/file_write.00000001
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12575069 0xbfe15d 0
4 12516793 0xbefdb9 0
7 12465427 0xbe3513 0
1 12646710 0xc0f936 0
3 12495013 0xbea8a5 0
5 12387072 0xbd0300 0
6 12600315 0xc043fb 0
0 12428720 0xbda5b0 0
/esfs/jtacquaviva/ioperf/file_write.00000037
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12465392 0xbe34f0 0
4 12516761 0xbefd99 0
5 12387039 0xbd02df 0
6 12600282 0xc043da 0
1 12646676 0xc0f914 0
0 12428686 0xbda58e 0
3 12494980 0xbea884 0
2 12575036 0xbfe13c 0
/esfs/jtacquaviva/ioperf/file_write.00000011
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12387045 0xbd02e5 0
6 12600289 0xc043e1 0
0 12428694 0xbda596 0
3 12494988 0xbea88c 0
2 12575043 0xbfe143 0
7 12465403 0xbe34fb 0
1 12646686 0xc0f91e 0
4 12516770 0xbefda2 0

View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 8 -np 64 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 20132659200 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:8#API:MPIIO#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 05:21:10 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 20132659200 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 05:21:10 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 64 (8 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 18.75 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288481689600.
WARNING: Using actual aggregate bytes moved = 27894272000.
write 436.81 19660800 100.00 0.014060 60.89 0.000730 60.90 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288481996800.
WARNING: Using actual aggregate bytes moved = 28205158400.
write 442.84 19660800 100.00 0.002115 60.74 0.000535 60.74 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288483020800.
WARNING: Using actual aggregate bytes moved = 28058112000.
write 440.55 19660800 100.00 0.007811 60.73 0.000577 60.74 2
Max Write: 442.84 MiB/sec (464.36 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 442.84 436.81 440.07 2.49 60.79296 0 64 8 3 0 0 1 0 0 1 20132659200 102400 27894272000 MPIIO 0
Finished: Wed Oct 24 05:24:13 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 8 -np 64 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 8 -np 64 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 20132659200 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
+ tee -a ./output/COUNT:1#NN:8#PPN:8#API:MPIIO#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 05:24:30 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 20132659200 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 05:24:30 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 64 (8 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 18.75 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 25346150400.
read 398.00 19660800 100.00 0.024084 60.71 0.003299 60.73 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 25875046400.
read 406.49 19660800 100.00 0.005116 60.70 0.005719 60.71 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 26402201600.
read 414.86 19660800 100.00 0.005228 60.68 0.011043 60.69 2
Max Read: 414.86 MiB/sec (435.01 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 414.86 398.00 406.45 6.88 60.71121 0 64 8 3 0 0 1 0 0 1 20132659200 102400 25346150400 MPIIO 0
Finished: Wed Oct 24 05:27:32 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12599539 0xc040f3 0
0 12427942 0xbda2a6 0
2 12574286 0xbfde4e 0
4 12516018 0xbefab2 0
7 12464645 0xbe3205 0
1 12645928 0xc0f628 0
3 12494227 0xbea593 0
5 12386296 0xbcfff8 0

View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 8 -np 64 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 20132659200 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:8#API:MPIIO#T:1048576.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 04:13:18 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 20132659200 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 04:13:18 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 64 (8 per node)
repetitions = 3
xfersize = 1 MiB
blocksize = 18.75 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288490188800.
WARNING: Using actual aggregate bytes moved = 307188727808.
write 4872 19660800 1024.00 0.013533 60.11 0.002723 60.13 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288488091648.
WARNING: Using actual aggregate bytes moved = 302947237888.
write 4802 19660800 1024.00 0.048004 60.11 0.000935 60.16 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288487043072.
WARNING: Using actual aggregate bytes moved = 302345355264.
write 4782 19660800 1024.00 0.182298 60.12 0.000586 60.30 2
Max Write: 4872.12 MiB/sec (5108.78 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 4872.12 4781.84 4818.79 38.63 60.19604 0 64 8 3 0 0 1 0 0 1 20132659200 1048576 307188727808 MPIIO 0
Finished: Wed Oct 24 04:16:25 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 8 -np 64 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 8 -np 64 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 20132659200 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
+ tee -a ./output/COUNT:1#NN:8#PPN:8#API:MPIIO#T:1048576.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 04:16:54 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 20132659200 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 04:16:54 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 64 (8 per node)
repetitions = 3
xfersize = 1 MiB
blocksize = 18.75 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 100134813696.
read 1588.11 19660800 1024.00 0.011837 60.12 0.000871 60.13 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 103155761152.
read 1636.14 19660800 1024.00 0.002649 60.12 0.000955 60.13 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 103716749312.
read 1645.49 19660800 1024.00 0.002954 60.11 0.000986 60.11 2
Max Read: 1645.49 MiB/sec (1725.42 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 1645.49 1588.11 1623.25 25.14 60.12338 0 64 8 3 0 0 1 0 0 1 20132659200 1048576 100134813696 MPIIO 0
Finished: Wed Oct 24 04:19:54 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12599027 0xc03ef3 0
0 12427434 0xbda0aa 0
2 12573773 0xbfdc4d 0
4 12515505 0xbef8b1 0
7 12464129 0xbe3001 0
1 12645418 0xc0f42a 0
3 12493715 0xbea393 0
5 12385781 0xbcfdf5 0

View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 8 -np 64 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 20132659200 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:8#API:MPIIO#T:10485760.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 03:06:20 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 20132659200 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 03:06:20 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 64 (8 per node)
repetitions = 3
xfersize = 10 MiB
blocksize = 18.75 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288395816960.
WARNING: Using actual aggregate bytes moved = 230256803840.
write 3605 19660800 10240 0.014762 60.90 0.004570 60.92 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288469217280.
WARNING: Using actual aggregate bytes moved = 221301964800.
write 3471.84 19660800 10240 0.021765 60.77 0.000796 60.79 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288448245760.
WARNING: Using actual aggregate bytes moved = 228243537920.
write 3593.63 19660800 10240 0.066377 60.50 0.000925 60.57 2
Max Write: 3604.83 MiB/sec (3779.94 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 3604.83 3471.84 3556.77 60.23 60.75853 0 64 8 3 0 0 1 0 0 1 20132659200 10485760 230256803840 MPIIO 0
Finished: Wed Oct 24 03:09:28 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 8 -np 64 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ tee -a ./output/COUNT:1#NN:8#PPN:8#API:MPIIO#T:10485760.txt
+ /opt/ddn/mvapich/bin/mpiexec -ppn 8 -np 64 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 20132659200 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 03:09:51 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 20132659200 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 03:09:51 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 64 (8 per node)
repetitions = 3
xfersize = 10 MiB
blocksize = 18.75 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 204126289920.
read 3234.82 19660800 10240 0.013093 60.17 0.005014 60.18 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 207230074880.
read 3281.32 19660800 10240 0.006754 60.22 0.005031 60.23 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 210973491200.
read 3342.84 19660800 10240 0.007098 60.18 0.004973 60.19 2
Max Read: 3342.84 MiB/sec (3505.22 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 3342.84 3234.82 3286.33 44.24 60.19892 0 64 8 3 0 0 1 0 0 1 20132659200 10485760 204126289920 MPIIO 0
Finished: Wed Oct 24 03:12:51 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573258 0xbfda4a 0
4 12514996 0xbef6b4 0
7 12463617 0xbe2e01 0
1 12644902 0xc0f226 0
3 12493203 0xbea193 0
5 12385267 0xbcfbf3 0
6 12598518 0xc03cf6 0
0 12426922 0xbd9eaa 0

View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 8 -np 64 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 20132659200 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:8#PPN:8#API:MPIIO#T:16384.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 06:32:43 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 20132659200 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 06:32:43 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 64 (8 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 18.75 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288485191680.
WARNING: Using actual aggregate bytes moved = 4896669696.
write 72.60 19660800 16.00 0.016341 64.30 0.000661 64.32 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288486846464.
WARNING: Using actual aggregate bytes moved = 4818714624.
write 71.58 19660800 16.00 0.002185 64.20 0.000805 64.20 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 1288489058304.
WARNING: Using actual aggregate bytes moved = 4859789312.
write 72.16 19660800 16.00 0.002538 64.23 0.000916 64.23 2
Max Write: 72.60 MiB/sec (76.13 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 72.60 71.58 72.11 0.42 64.24925 0 64 8 3 0 0 1 0 0 1 20132659200 16384 4896669696 MPIIO 0
Finished: Wed Oct 24 06:35:56 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 8 -np 64 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ tee -a ./output/COUNT:1#NN:8#PPN:8#API:MPIIO#T:16384.txt
+ /opt/ddn/mvapich/bin/mpiexec -ppn 8 -np 64 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07,isc17-c08,isc17-c09,isc17-c11,isc17-c12 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 20132659200 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 06:36:04 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 20132659200 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 06:36:04 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 64 (8 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 18.75 GiB
aggregate filesize = 1200 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 5439520768.
read 80.81 19660800 16.00 0.023834 64.17 0.004831 64.19 0
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 5462327296.
read 81.18 19660800 16.00 0.006555 64.16 0.007248 64.17 1
WARNING: Expected aggregate file size = 1288490188800.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 5475713024.
read 81.36 19660800 16.00 0.006923 64.18 0.004780 64.18 2
Max Read: 81.36 MiB/sec (85.32 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 81.36 80.81 81.12 0.23 64.18069 0 64 8 3 0 0 1 0 0 1 20132659200 16384 5439520768 MPIIO 0
Finished: Wed Oct 24 06:39:16 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 16 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12574801 0xbfe051 0
4 12516528 0xbefcb0 0
7 12465158 0xbe3406 0
1 12646443 0xc0f82b 0
3 12494746 0xbea79a 0
5 12386810 0xbd01fa 0
6 12600050 0xc042f2 0
0 12428454 0xbda4a6 0

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff