julian.kunkel
/
ddn-ime-evaluation
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

NN:2 experiment results

This commit is contained in:
Eugen Betke 2018-10-24 07:41:42 +02:00
parent a01de6b5e3
commit 2060b9b5f6
10 changed files with 1494 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

106
output/COUNT:1#NN:2#PPN:1#API:MPIIO#T:102400.txt Normal file
View File

@ -0,0 +1,106 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 2 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 161061273600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:2#PPN:1#API:MPIIO#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 02:27:32 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 161061273600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 02:27:32 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 2 (1 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 150 GiB
aggregate filesize = 300 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 322122547200.
WARNING: Stat() of aggregate file size = 322120192000.
WARNING: Using actual aggregate bytes moved = 9152307200.
write 144.74 157286400 100.00 0.006224 60.30 0.000231 60.30 0
WARNING: Expected aggregate file size = 322122547200.
WARNING: Stat() of aggregate file size = 322102067200.
WARNING: Using actual aggregate bytes moved = 8739942400.
write 138.30 157286400 100.00 0.007606 60.26 0.000292 60.27 1
WARNING: Expected aggregate file size = 322122547200.
WARNING: Stat() of aggregate file size = 322116096000.
WARNING: Using actual aggregate bytes moved = 8823091200.
write 139.60 157286400 100.00 0.008529 60.27 0.000236 60.28 2
Max Write: 144.74 MiB/sec (151.77 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 144.74 138.30 140.88 2.78 60.28332 0 2 1 3 0 0 1 0 0 1 161061273600 102400 9152307200 MPIIO 0
Finished: Wed Oct 24 02:30:34 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 2 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 2 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 161061273600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
+ tee -a ./output/COUNT:1#NN:2#PPN:1#API:MPIIO#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 02:30:37 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 161061273600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 02:30:37 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 2 (1 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 150 GiB
aggregate filesize = 300 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 322122547200.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 1769369600.
read 28.01 157286400 100.00 0.008440 60.24 0.000252 60.25 0
WARNING: Expected aggregate file size = 322122547200.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 1797017600.
read 28.45 157286400 100.00 0.000544 60.24 0.000175 60.24 1
WARNING: Expected aggregate file size = 322122547200.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 1825484800.
read 28.90 157286400 100.00 0.000529 60.23 0.000203 60.23 2
Max Read: 28.90 MiB/sec (30.31 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 28.90 28.01 28.45 0.37 60.23937 0 2 1 3 0 0 1 0 0 1 161061273600 102400 1769369600 MPIIO 0
Finished: Wed Oct 24 02:33:38 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 4 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 4
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12426702 0xbd9dce 0
2 12573040 0xbfd970 0
4 12514776 0xbef5d8 0
7 12463398 0xbe2d26 0

116
output/COUNT:1#NN:2#PPN:1#API:POSIX#T:102400.txt Normal file
View File

@ -0,0 +1,116 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 2 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 161061273600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:2#PPN:1#API:POSIX#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 02:21:03 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 161061273600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 02:21:03 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 2 (1 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 150 GiB
aggregate filesize = 300 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 322122547200.
WARNING: Stat() of aggregate file size = 322122547200.
WARNING: Using actual aggregate bytes moved = 55821209600.
write 847.99 157286400 100.00 0.000261 62.78 0.000140 62.78 0
WARNING: Expected aggregate file size = 322122547200.
WARNING: Stat() of aggregate file size = 322121932800.
WARNING: Using actual aggregate bytes moved = 55736627200.
write 838.00 157286400 100.00 0.000501 63.43 0.000133 63.43 1
WARNING: Expected aggregate file size = 322122547200.
WARNING: Stat() of aggregate file size = 322120499200.
WARNING: Using actual aggregate bytes moved = 55724441600.
write 858.35 157286400 100.00 0.000525 61.91 0.000169 61.91 2
Max Write: 858.35 MiB/sec (900.05 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 858.35 838.00 848.11 8.31 62.70713 0 2 1 3 1 0 1 0 0 1 161061273600 102400 55821209600 POSIX 0
Finished: Wed Oct 24 02:24:18 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 2 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 2 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 161061273600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread2/file -r
+ tee -a ./output/COUNT:1#NN:2#PPN:1#API:POSIX#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 02:24:31 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 161061273600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread2/file -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 02:24:31 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/indread2/file
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 2 (1 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 150 GiB
aggregate filesize = 300 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 322122547200.
WARNING: Stat() of aggregate file size = 322122547200.
WARNING: Using actual aggregate bytes moved = 948531200.
read 15.06 157286400 100.00 0.000357 60.06 0.000114 60.07 0
WARNING: Expected aggregate file size = 322122547200.
WARNING: Stat() of aggregate file size = 322122547200.
WARNING: Using actual aggregate bytes moved = 968192000.
read 15.37 157286400 100.00 0.000138 60.07 0.000134 60.07 1
WARNING: Expected aggregate file size = 322122547200.
WARNING: Stat() of aggregate file size = 322122547200.
WARNING: Using actual aggregate bytes moved = 981913600.
read 15.59 157286400 100.00 0.000160 60.05 0.000147 60.05 2
Max Read: 15.59 MiB/sec (16.35 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 15.59 15.06 15.34 0.22 60.06217 0 2 1 3 1 0 1 0 0 1 161061273600 102400 948531200 POSIX 0
Finished: Wed Oct 24 02:27:32 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 4 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write.00000000
lmm_stripe_count: 4
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12514774 0xbef5d6 0
7 12463396 0xbe2d24 0
1 12644682 0xc0f14a 0
3 12492977 0xbea0b1 0
/esfs/jtacquaviva/ioperf/file_write.00000001
lmm_stripe_count: 4
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12385042 0xbcfb12 0
6 12598295 0xc03c17 0
0 12426700 0xbd9dcc 0
2 12573038 0xbfd96e 0

106
output/COUNT:1#NN:2#PPN:2#API:MPIIO#T:102400.txt Normal file
View File

@ -0,0 +1,106 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 4 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 80530636800 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:2#PPN:2#API:MPIIO#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 02:53:13 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 80530636800 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 02:53:13 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 4 (2 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 75 GiB
aggregate filesize = 300 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 322122547200.
WARNING: Stat() of aggregate file size = 322105958400.
WARNING: Using actual aggregate bytes moved = 10540441600.
write 166.85 78643200 100.00 0.007447 60.24 0.000342 60.25 0
WARNING: Expected aggregate file size = 322122547200.
WARNING: Stat() of aggregate file size = 322109440000.
WARNING: Using actual aggregate bytes moved = 10210406400.
write 161.52 78643200 100.00 0.001175 60.29 0.000247 60.29 1
WARNING: Expected aggregate file size = 322122547200.
WARNING: Stat() of aggregate file size = 322110464000.
WARNING: Using actual aggregate bytes moved = 10192896000.
write 161.39 78643200 100.00 0.001328 60.23 0.000345 60.23 2
Max Write: 166.85 MiB/sec (174.95 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 166.85 161.39 163.25 2.54 60.25571 0 4 2 3 0 0 1 0 0 1 80530636800 102400 10540441600 MPIIO 0
Finished: Wed Oct 24 02:56:15 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 4 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 4 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 80530636800 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
+ tee -a ./output/COUNT:1#NN:2#PPN:2#API:MPIIO#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 02:56:22 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 80530636800 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 02:56:22 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 4 (2 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 75 GiB
aggregate filesize = 300 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 322122547200.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 3588812800.
read 56.84 78643200 100.00 0.009064 60.20 0.000314 60.21 0
WARNING: Expected aggregate file size = 322122547200.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 3636019200.
read 57.56 78643200 100.00 0.000675 60.24 0.000270 60.25 1
WARNING: Expected aggregate file size = 322122547200.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 3678412800.
read 58.27 78643200 100.00 0.000650 60.20 0.000324 60.20 2
Max Read: 58.27 MiB/sec (61.11 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 58.27 56.84 57.56 0.58 60.21749 0 4 2 3 0 0 1 0 0 1 80530636800 102400 3588812800 MPIIO 0
Finished: Wed Oct 24 02:59:23 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 4 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 4
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12598325 0xc03c35 0
0 12426729 0xbd9de9 0
2 12573066 0xbfd98a 0
4 12514803 0xbef5f3 0

140
output/COUNT:1#NN:2#PPN:2#API:POSIX#T:102400.txt Normal file
View File

@ -0,0 +1,140 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 4 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 80530636800 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:2#PPN:2#API:POSIX#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 02:46:35 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 80530636800 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 02:46:35 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 4 (2 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 75 GiB
aggregate filesize = 300 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 322122547200.
WARNING: Stat() of aggregate file size = 322112102400.
WARNING: Using actual aggregate bytes moved = 54546124800.
write 762.68 78643200 100.00 0.000504 68.20 0.000188 68.21 0
WARNING: Expected aggregate file size = 322122547200.
WARNING: Stat() of aggregate file size = 322119065600.
WARNING: Using actual aggregate bytes moved = 54886092800.
write 772.96 78643200 100.00 0.003134 67.71 0.000180 67.72 1
WARNING: Expected aggregate file size = 322122547200.
WARNING: Stat() of aggregate file size = 322122547200.
WARNING: Using actual aggregate bytes moved = 53985075200.
write 765.28 78643200 100.00 0.012096 67.26 0.000198 67.28 2
Max Write: 772.96 MiB/sec (810.51 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 772.96 762.68 766.97 4.36 67.73304 0 4 2 3 1 0 1 0 0 1 80530636800 102400 54546124800 POSIX 0
Finished: Wed Oct 24 02:50:03 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 4 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 4 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 80530636800 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread2/file -r
+ tee -a ./output/COUNT:1#NN:2#PPN:2#API:POSIX#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 02:50:11 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 80530636800 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread2/file -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 02:50:11 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/indread2/file
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 4 (2 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 75 GiB
aggregate filesize = 300 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 322122547200.
WARNING: Stat() of aggregate file size = 644245094400.
WARNING: Using actual aggregate bytes moved = 2212761600.
read 35.15 78643200 100.00 0.002042 60.03 0.001796 60.03 0
WARNING: Expected aggregate file size = 322122547200.
WARNING: Stat() of aggregate file size = 644245094400.
WARNING: Using actual aggregate bytes moved = 2382848000.
read 37.85 78643200 100.00 0.001738 60.04 0.001798 60.04 1
WARNING: Expected aggregate file size = 322122547200.
WARNING: Stat() of aggregate file size = 644245094400.
WARNING: Using actual aggregate bytes moved = 2385408000.
read 37.89 78643200 100.00 0.001828 60.04 0.001813 60.04 2
Max Read: 37.89 MiB/sec (39.73 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 37.89 35.15 36.96 1.28 60.03886 0 4 2 3 1 0 1 0 0 1 80530636800 102400 2212761600 POSIX 0
Finished: Wed Oct 24 02:53:12 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 4 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write.00000002
lmm_stripe_count: 4
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12644708 0xc0f164 0
3 12493005 0xbea0cd 0
5 12385070 0xbcfb2e 0
6 12598323 0xc03c33 0
/esfs/jtacquaviva/ioperf/file_write.00000000
lmm_stripe_count: 4
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12493004 0xbea0cc 0
0 12426727 0xbd9de7 0
2 12573063 0xbfd987 0
4 12514801 0xbef5f1 0
/esfs/jtacquaviva/ioperf/file_write.00000003
lmm_stripe_count: 4
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12463423 0xbe2d3f 0
1 12644707 0xc0f163 0
5 12385069 0xbcfb2d 0
6 12598322 0xc03c32 0
/esfs/jtacquaviva/ioperf/file_write.00000001
lmm_stripe_count: 4
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573064 0xbfd988 0
4 12514802 0xbef5f2 0
7 12463424 0xbe2d40 0
1 12644709 0xc0f165 0

106
output/COUNT:1#NN:2#PPN:4#API:MPIIO#T:102400.txt Normal file
View File

@ -0,0 +1,106 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 40265318400 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:2#PPN:4#API:MPIIO#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 02:14:51 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 40265318400 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 02:14:51 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 8 (4 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 37.50 GiB
aggregate filesize = 300 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 322122547200.
WARNING: Stat() of aggregate file size = 322122035200.
WARNING: Using actual aggregate bytes moved = 12138905600.
write 192.22 39321600 100.00 0.009217 60.22 0.000538 60.23 0
WARNING: Expected aggregate file size = 322122547200.
WARNING: Stat() of aggregate file size = 322115788800.
WARNING: Using actual aggregate bytes moved = 11619225600.
write 184.01 39321600 100.00 0.016119 60.20 0.000315 60.22 1
WARNING: Expected aggregate file size = 322122547200.
WARNING: Stat() of aggregate file size = 322117017600.
WARNING: Using actual aggregate bytes moved = 11599974400.
write 183.70 39321600 100.00 0.011791 60.21 0.000311 60.22 2
Max Write: 192.22 MiB/sec (201.55 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 192.22 183.70 186.64 3.94 60.22143 0 8 4 3 0 0 1 0 0 1 40265318400 102400 12138905600 MPIIO 0
Finished: Wed Oct 24 02:17:53 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 40265318400 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
+ tee -a ./output/COUNT:1#NN:2#PPN:4#API:MPIIO#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 02:18:01 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 40265318400 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 02:18:01 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 8 (4 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 37.50 GiB
aggregate filesize = 300 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 322122547200.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 6817075200.
read 108.00 39321600 100.00 0.010313 60.19 0.000503 60.20 0
WARNING: Expected aggregate file size = 322122547200.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 6965555200.
read 110.35 39321600 100.00 0.001125 60.20 0.000497 60.20 1
WARNING: Expected aggregate file size = 322122547200.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 7063040000.
read 111.91 39321600 100.00 0.001088 60.19 0.000494 60.19 2
Max Read: 111.91 MiB/sec (117.35 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 111.91 108.00 110.09 1.61 60.19408 0 8 4 3 0 0 1 0 0 1 40265318400 102400 6817075200 MPIIO 0
Finished: Wed Oct 24 02:21:02 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 4 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 4
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12598292 0xc03c14 0
0 12426697 0xbd9dc9 0
2 12573035 0xbfd96b 0
4 12514771 0xbef5d3 0

188
output/COUNT:1#NN:2#PPN:4#API:POSIX#T:102400.txt Normal file
View File

@ -0,0 +1,188 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 40265318400 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:2#PPN:4#API:POSIX#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 02:08:15 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 40265318400 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 02:08:15 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 8 (4 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 37.50 GiB
aggregate filesize = 300 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 322122547200.
WARNING: Stat() of aggregate file size = 322121728000.
WARNING: Using actual aggregate bytes moved = 68218572800.
write 1013.21 39321600 100.00 0.000907 64.21 0.000215 64.21 0
WARNING: Expected aggregate file size = 322122547200.
WARNING: Stat() of aggregate file size = 322111692800.
WARNING: Using actual aggregate bytes moved = 57584742400.
write 799.93 39321600 100.00 0.010344 68.64 0.000266 68.65 1
WARNING: Expected aggregate file size = 322122547200.
WARNING: Stat() of aggregate file size = 322122547200.
WARNING: Using actual aggregate bytes moved = 63105126400.
write 889.64 39321600 100.00 0.001342 67.65 0.000287 67.65 2
Max Write: 1013.21 MiB/sec (1062.42 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 1013.21 799.93 900.92 87.44 66.83676 0 8 4 3 1 0 1 0 0 1 40265318400 102400 68218572800 POSIX 0
Finished: Wed Oct 24 02:11:38 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 40265318400 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread2/file -r
+ tee -a ./output/COUNT:1#NN:2#PPN:4#API:POSIX#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 02:11:49 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 40265318400 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread2/file -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 02:11:49 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/indread2/file
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 8 (4 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 37.50 GiB
aggregate filesize = 300 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 322122547200.
WARNING: Stat() of aggregate file size = 1288490188800.
WARNING: Using actual aggregate bytes moved = 4944384000.
read 78.44 39321600 100.00 0.000905 60.11 0.000325 60.11 0
WARNING: Expected aggregate file size = 322122547200.
WARNING: Stat() of aggregate file size = 1288490188800.
WARNING: Using actual aggregate bytes moved = 5427609600.
read 86.23 39321600 100.00 0.000325 60.03 0.000304 60.03 1
WARNING: Expected aggregate file size = 322122547200.
WARNING: Stat() of aggregate file size = 1288490188800.
WARNING: Using actual aggregate bytes moved = 5643366400.
read 89.66 39321600 100.00 0.000326 60.03 0.000377 60.03 2
Max Read: 89.66 MiB/sec (94.01 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 89.66 78.44 84.77 4.69 60.05699 0 8 4 3 1 0 1 0 0 1 40265318400 102400 4944384000 POSIX 0
Finished: Wed Oct 24 02:14:49 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 4 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write.00000006
lmm_stripe_count: 4
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12426694 0xbd9dc6 0
2 12573032 0xbfd968 0
4 12514769 0xbef5d1 0
7 12463390 0xbe2d1e 0
/esfs/jtacquaviva/ioperf/file_write.00000002
lmm_stripe_count: 4
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12644677 0xc0f145 0
3 12492971 0xbea0ab 0
5 12385036 0xbcfb0c 0
6 12598289 0xc03c11 0
/esfs/jtacquaviva/ioperf/file_write.00000000
lmm_stripe_count: 4
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12385034 0xbcfb0a 0
6 12598287 0xc03c0f 0
0 12426692 0xbd9dc4 0
2 12573030 0xbfd966 0
/esfs/jtacquaviva/ioperf/file_write.00000004
lmm_stripe_count: 4
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12463389 0xbe2d1d 0
1 12644676 0xc0f144 0
3 12492970 0xbea0aa 0
5 12385035 0xbcfb0b 0
/esfs/jtacquaviva/ioperf/file_write.00000007
lmm_stripe_count: 4
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12598288 0xc03c10 0
0 12426693 0xbd9dc5 0
2 12573031 0xbfd967 0
4 12514768 0xbef5d0 0
/esfs/jtacquaviva/ioperf/file_write.00000003
lmm_stripe_count: 4
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12492972 0xbea0ac 0
5 12385037 0xbcfb0d 0
6 12598290 0xc03c12 0
0 12426695 0xbd9dc7 0
/esfs/jtacquaviva/ioperf/file_write.00000005
lmm_stripe_count: 4
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12514767 0xbef5cf 0
7 12463388 0xbe2d1c 0
1 12644675 0xc0f143 0
3 12492969 0xbea0a9 0
/esfs/jtacquaviva/ioperf/file_write.00000001
lmm_stripe_count: 4
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573033 0xbfd969 0
4 12514770 0xbef5d2 0
7 12463391 0xbe2d1f 0
1 12644678 0xc0f146 0

106
output/COUNT:1#NN:2#PPN:6#API:MPIIO#T:102400.txt Normal file
View File

@ -0,0 +1,106 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 12 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 26843545600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:2#PPN:6#API:MPIIO#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 02:40:22 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 26843545600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 02:40:22 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 12 (6 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 25 GiB
aggregate filesize = 300 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 322122547200.
WARNING: Stat() of aggregate file size = 322102681600.
WARNING: Using actual aggregate bytes moved = 13213696000.
write 208.94 26214400 100.00 0.010154 60.30 0.001372 60.31 0
WARNING: Expected aggregate file size = 322122547200.
WARNING: Stat() of aggregate file size = 322116710400.
WARNING: Using actual aggregate bytes moved = 12777472000.
write 201.84 26214400 100.00 0.002291 60.37 0.000645 60.37 1
WARNING: Expected aggregate file size = 322122547200.
WARNING: Stat() of aggregate file size = 322121216000.
WARNING: Using actual aggregate bytes moved = 12773990400.
write 202.37 26214400 100.00 0.018547 60.18 0.000691 60.20 2
Max Write: 208.94 MiB/sec (219.09 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 208.94 201.84 204.38 3.23 60.29436 0 12 6 3 0 0 1 0 0 1 26843545600 102400 13213696000 MPIIO 0
Finished: Wed Oct 24 02:43:25 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 12 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 12 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 26843545600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
+ tee -a ./output/COUNT:1#NN:2#PPN:6#API:MPIIO#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 02:43:33 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 26843545600 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 02:43:33 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 12 (6 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 25 GiB
aggregate filesize = 300 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 322122547200.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 9874636800.
read 156.41 26214400 100.00 0.014414 60.19 0.001085 60.21 0
WARNING: Expected aggregate file size = 322122547200.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 10107084800.
read 160.15 26214400 100.00 0.002008 60.19 0.001028 60.19 1
WARNING: Expected aggregate file size = 322122547200.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 10151116800.
read 160.86 26214400 100.00 0.001991 60.18 0.001041 60.18 2
Max Read: 160.86 MiB/sec (168.67 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 160.86 156.41 159.14 1.95 60.19314 0 12 6 3 0 0 1 0 0 1 26843545600 102400 9874636800 MPIIO 0
Finished: Wed Oct 24 02:46:34 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 4 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 4
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12514795 0xbef5eb 0
7 12463418 0xbe2d3a 0
1 12644703 0xc0f15f 0
3 12492999 0xbea0c7 0

236
output/COUNT:1#NN:2#PPN:6#API:POSIX#T:102400.txt Normal file
View File

@ -0,0 +1,236 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 12 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 26843545600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:2#PPN:6#API:POSIX#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 02:33:40 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 26843545600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 02:33:40 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 12 (6 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 25 GiB
aggregate filesize = 300 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 322122547200.
WARNING: Stat() of aggregate file size = 322121011200.
WARNING: Using actual aggregate bytes moved = 69853900800.
write 981.14 26214400 100.00 0.001717 67.90 0.000605 67.90 0
WARNING: Expected aggregate file size = 322122547200.
WARNING: Stat() of aggregate file size = 322104934400.
WARNING: Using actual aggregate bytes moved = 62405529600.
write 873.57 26214400 100.00 0.002215 68.13 0.000602 68.13 1
WARNING: Expected aggregate file size = 322122547200.
WARNING: Stat() of aggregate file size = 322121523200.
WARNING: Using actual aggregate bytes moved = 62417715200.
write 873.08 26214400 100.00 0.002147 68.18 0.000584 68.18 2
Max Write: 981.14 MiB/sec (1028.80 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 981.14 873.08 909.26 50.82 68.06862 0 12 6 3 1 0 1 0 0 1 26843545600 102400 69853900800 POSIX 0
Finished: Wed Oct 24 02:37:09 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 12 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 12 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 26843545600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread2/file -r
+ tee -a ./output/COUNT:1#NN:2#PPN:6#API:POSIX#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 02:37:19 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 26843545600 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread2/file -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 02:37:19 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/indread2/file
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 12 (6 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 25 GiB
aggregate filesize = 300 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 322122547200.
WARNING: Stat() of aggregate file size = 1932735283200.
WARNING: Using actual aggregate bytes moved = 8842240000.
read 140.48 26214400 100.00 0.006565 60.03 0.005931 60.03 0
WARNING: Expected aggregate file size = 322122547200.
WARNING: Stat() of aggregate file size = 1932735283200.
WARNING: Using actual aggregate bytes moved = 9492377600.
read 150.81 26214400 100.00 0.006016 60.03 0.006016 60.03 1
WARNING: Expected aggregate file size = 322122547200.
WARNING: Stat() of aggregate file size = 1932735283200.
WARNING: Using actual aggregate bytes moved = 9903206400.
read 157.31 26214400 100.00 0.006038 60.04 0.006017 60.04 2
Max Read: 157.31 MiB/sec (164.95 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 157.31 140.48 149.53 6.93 60.03092 0 12 6 3 1 0 1 0 0 1 26843545600 102400 8842240000 POSIX 0
Finished: Wed Oct 24 02:40:19 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 4 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write.00000006
lmm_stripe_count: 4
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12492995 0xbea0c3 0
5 12385061 0xbcfb25 0
6 12598314 0xc03c2a 0
0 12426719 0xbd9ddf 0
/esfs/jtacquaviva/ioperf/file_write.00000002
lmm_stripe_count: 4
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12463412 0xbe2d34 0
5 12385059 0xbcfb23 0
6 12598312 0xc03c28 0
0 12426718 0xbd9dde 0
/esfs/jtacquaviva/ioperf/file_write.00000000
lmm_stripe_count: 4
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12644698 0xc0f15a 0
3 12492994 0xbea0c2 0
5 12385060 0xbcfb24 0
6 12598313 0xc03c29 0
/esfs/jtacquaviva/ioperf/file_write.00000004
lmm_stripe_count: 4
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12385058 0xbcfb22 0
6 12598310 0xc03c26 0
0 12426716 0xbd9ddc 0
2 12573053 0xbfd97d 0
/esfs/jtacquaviva/ioperf/file_write.00000007
lmm_stripe_count: 4
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573052 0xbfd97c 0
4 12514788 0xbef5e4 0
7 12463410 0xbe2d32 0
1 12644696 0xc0f158 0
/esfs/jtacquaviva/ioperf/file_write.00000003
lmm_stripe_count: 4
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12514793 0xbef5e9 0
7 12463415 0xbe2d37 0
1 12644700 0xc0f15c 0
3 12492996 0xbea0c4 0
/esfs/jtacquaviva/ioperf/file_write.00000008
lmm_stripe_count: 4
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12492993 0xbea0c1 0
2 12573055 0xbfd97f 0
4 12514791 0xbef5e7 0
7 12463413 0xbe2d35 0
/esfs/jtacquaviva/ioperf/file_write.00000010
lmm_stripe_count: 4
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573056 0xbfd980 0
4 12514792 0xbef5e8 0
7 12463414 0xbe2d36 0
1 12644699 0xc0f15b 0
/esfs/jtacquaviva/ioperf/file_write.00000005
lmm_stripe_count: 4
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12463416 0xbe2d38 0
1 12644701 0xc0f15d 0
3 12492997 0xbea0c5 0
5 12385063 0xbcfb27 0
/esfs/jtacquaviva/ioperf/file_write.00000009
lmm_stripe_count: 4
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12598311 0xc03c27 0
0 12426717 0xbd9ddd 0
2 12573054 0xbfd97e 0
4 12514790 0xbef5e6 0
/esfs/jtacquaviva/ioperf/file_write.00000001
lmm_stripe_count: 4
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12514789 0xbef5e5 0
7 12463411 0xbe2d33 0
1 12644697 0xc0f159 0
3 12492992 0xbea0c0 0
/esfs/jtacquaviva/ioperf/file_write.00000011
lmm_stripe_count: 4
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12385062 0xbcfb26 0
6 12598315 0xc03c2b 0
0 12426720 0xbd9de0 0
2 12573057 0xbfd981 0

106
output/COUNT:1#NN:2#PPN:8#API:MPIIO#T:102400.txt Normal file
View File

@ -0,0 +1,106 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 8 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 20132659200 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:2#PPN:8#API:MPIIO#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 02:01:57 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 20132659200 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 02:01:57 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 16 (8 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 18.75 GiB
aggregate filesize = 300 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 322122547200.
WARNING: Stat() of aggregate file size = 322118246400.
WARNING: Using actual aggregate bytes moved = 13937049600.
write 220.34 19660800 100.00 0.010409 60.31 0.000809 60.32 0
WARNING: Expected aggregate file size = 322122547200.
WARNING: Stat() of aggregate file size = 322108006400.
WARNING: Using actual aggregate bytes moved = 13646131200.
write 214.79 19660800 100.00 0.001975 60.59 0.000297 60.59 1
WARNING: Expected aggregate file size = 322122547200.
WARNING: Stat() of aggregate file size = 322122035200.
WARNING: Using actual aggregate bytes moved = 13902131200.
write 220.23 19660800 100.00 0.010326 60.19 0.000369 60.20 2
Max Write: 220.34 MiB/sec (231.04 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 220.34 214.79 218.45 2.59 60.37087 0 16 8 3 0 0 1 0 0 1 20132659200 102400 13937049600 MPIIO 0
Finished: Wed Oct 24 02:05:01 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 8 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 8 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 20132659200 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
+ tee -a ./output/COUNT:1#NN:2#PPN:8#API:MPIIO#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Wed Oct 24 02:05:12 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 20132659200 -D 60 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 02:05:12 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 16 (8 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 18.75 GiB
aggregate filesize = 300 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 322122547200.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 12402892800.
read 196.49 19660800 100.00 0.007103 60.19 0.000384 60.20 0
WARNING: Expected aggregate file size = 322122547200.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 12679884800.
read 200.92 19660800 100.00 0.001326 60.18 0.004191 60.19 1
WARNING: Expected aggregate file size = 322122547200.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 12746854400.
read 201.98 19660800 100.00 0.001368 60.18 0.000464 60.18 2
Max Read: 201.98 MiB/sec (211.80 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 201.98 196.49 199.80 2.38 60.18917 0 16 8 3 0 0 1 0 0 1 20132659200 102400 12402892800 MPIIO 0
Finished: Wed Oct 24 02:08:13 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 4 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 4
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12492960 0xbea0a0 0
5 12385024 0xbcfb00 0
6 12598278 0xc03c06 0
0 12426683 0xbd9dbb 0

284
output/COUNT:1#NN:2#PPN:8#API:POSIX#T:102400.txt Normal file
View File

@ -0,0 +1,284 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 8 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 20132659200 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:2#PPN:8#API:POSIX#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 01:55:15 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 20132659200 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 01:55:15 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 16 (8 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 18.75 GiB
aggregate filesize = 300 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 322122547200.
WARNING: Stat() of aggregate file size = 322108006400.
WARNING: Using actual aggregate bytes moved = 71829913600.
write 1021.71 19660800 100.00 0.002206 67.04 0.000258 67.05 0
WARNING: Expected aggregate file size = 322122547200.
WARNING: Stat() of aggregate file size = 322092441600.
WARNING: Using actual aggregate bytes moved = 63509299200.
write 914.74 19660800 100.00 0.002308 66.21 0.000350 66.21 1
WARNING: Expected aggregate file size = 322122547200.
WARNING: Stat() of aggregate file size = 322093875200.
WARNING: Using actual aggregate bytes moved = 62901657600.
write 890.62 19660800 100.00 0.003864 67.35 0.000550 67.36 2
Max Write: 1021.71 MiB/sec (1071.34 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 1021.71 890.62 942.36 56.97 66.87144 0 16 8 3 1 0 1 0 0 1 20132659200 102400 71829913600 POSIX 0
Finished: Wed Oct 24 01:58:40 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 8 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 8 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 20132659200 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread2/file -r
+ tee -a ./output/COUNT:1#NN:2#PPN:8#API:POSIX#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Wed Oct 24 01:58:54 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 20132659200 -D 60 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread2/file -r
Machine: Linux isc17-c04
Test 0 started: Wed Oct 24 01:58:54 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/indread2/file
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 16 (8 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 18.75 GiB
aggregate filesize = 300 GiB
Using stonewalling = 60 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 322122547200.
WARNING: Stat() of aggregate file size = 2576980377600.
WARNING: Using actual aggregate bytes moved = 11899084800.
read 189.03 19660800 100.00 0.001379 60.03 0.000453 60.03 0
WARNING: Expected aggregate file size = 322122547200.
WARNING: Stat() of aggregate file size = 2576980377600.
WARNING: Using actual aggregate bytes moved = 12504166400.
read 198.65 19660800 100.00 0.000466 60.03 0.000467 60.03 1
WARNING: Expected aggregate file size = 322122547200.
WARNING: Stat() of aggregate file size = 2576980377600.
WARNING: Using actual aggregate bytes moved = 12973363200.
read 206.14 19660800 100.00 0.000516 60.02 0.004267 60.02 2
Max Read: 206.14 MiB/sec (216.15 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 206.14 189.03 197.94 7.00 60.02689 0 16 8 3 1 0 1 0 0 1 20132659200 102400 11899084800 POSIX 0
Finished: Wed Oct 24 02:01:54 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 4 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write.00000006
lmm_stripe_count: 4
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573017 0xbfd959 0
4 12514755 0xbef5c3 0
7 12463377 0xbe2d11 0
1 12644663 0xc0f137 0
/esfs/jtacquaviva/ioperf/file_write.00000002
lmm_stripe_count: 4
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12573013 0xbfd955 0
4 12514751 0xbef5bf 0
7 12463373 0xbe2d0d 0
1 12644659 0xc0f133 0
/esfs/jtacquaviva/ioperf/file_write.00000000
lmm_stripe_count: 4
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12385022 0xbcfafe 0
6 12598275 0xc03c03 0
0 12426681 0xbd9db9 0
2 12573018 0xbfd95a 0
/esfs/jtacquaviva/ioperf/file_write.00000004
lmm_stripe_count: 4
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12514756 0xbef5c4 0
7 12463378 0xbe2d12 0
1 12644664 0xc0f138 0
3 12492958 0xbea09e 0
/esfs/jtacquaviva/ioperf/file_write.00000007
lmm_stripe_count: 4
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12598276 0xc03c04 0
0 12426682 0xbd9dba 0
2 12573019 0xbfd95b 0
4 12514757 0xbef5c5 0
/esfs/jtacquaviva/ioperf/file_write.00000012
lmm_stripe_count: 4
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12514752 0xbef5c0 0
7 12463374 0xbe2d0e 0
1 12644660 0xc0f134 0
3 12492954 0xbea09a 0
/esfs/jtacquaviva/ioperf/file_write.00000013
lmm_stripe_count: 4
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12492957 0xbea09d 0
5 12385021 0xbcfafd 0
6 12598274 0xc03c02 0
0 12426680 0xbd9db8 0
/esfs/jtacquaviva/ioperf/file_write.00000003
lmm_stripe_count: 4
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12385018 0xbcfafa 0
6 12598271 0xc03bff 0
0 12426677 0xbd9db5 0
2 12573014 0xbfd956 0
/esfs/jtacquaviva/ioperf/file_write.00000008
lmm_stripe_count: 4
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12644662 0xc0f136 0
3 12492956 0xbea09c 0
5 12385020 0xbcfafc 0
6 12598273 0xc03c01 0
/esfs/jtacquaviva/ioperf/file_write.00000010
lmm_stripe_count: 4
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12492953 0xbea099 0
5 12385017 0xbcfaf9 0
6 12598270 0xc03bfe 0
0 12426676 0xbd9db4 0
/esfs/jtacquaviva/ioperf/file_write.00000005
lmm_stripe_count: 4
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12426675 0xbd9db3 0
2 12573012 0xbfd954 0
4 12514750 0xbef5be 0
7 12463372 0xbe2d0c 0
/esfs/jtacquaviva/ioperf/file_write.00000014
lmm_stripe_count: 4
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12644658 0xc0f132 0
3 12492952 0xbea098 0
5 12385016 0xbcfaf8 0
6 12598269 0xc03bfd 0
/esfs/jtacquaviva/ioperf/file_write.00000009
lmm_stripe_count: 4
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12463375 0xbe2d0f 0
1 12644661 0xc0f135 0
3 12492955 0xbea09b 0
5 12385019 0xbcfafb 0
/esfs/jtacquaviva/ioperf/file_write.00000015
lmm_stripe_count: 4
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12426679 0xbd9db7 0
2 12573016 0xbfd958 0
4 12514754 0xbef5c2 0
7 12463376 0xbe2d10 0
/esfs/jtacquaviva/ioperf/file_write.00000001
lmm_stripe_count: 4
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12598272 0xc03c00 0
0 12426678 0xbd9db6 0
2 12573015 0xbfd957 0
4 12514753 0xbef5c1 0
/esfs/jtacquaviva/ioperf/file_write.00000011
lmm_stripe_count: 4
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12463371 0xbe2d0b 0
1 12644657 0xc0f131 0
3 12492951 0xbea097 0
5 12385015 0xbcfaf7 0