julian.kunkel
/
ddn-ime-evaluation
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

output of NN:4 experiment

This commit is contained in:
Eugen Betke 2018-10-23 19:57:05 +02:00
parent 7895dde428
commit 93c6510ffa
40 changed files with 9369 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

110
output/COUNT:1#NN:4#PPN:1#API:MPIIO#T:102400.txt Normal file
View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 4 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 161061273600 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:4#PPN:1#API:MPIIO#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Tue Oct 23 17:33:07 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 161061273600 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 17:33:07 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 4 (1 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 150 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644241920000.
WARNING: Using actual aggregate bytes moved = 21748326400.
write 172.18 157286400 100.00 0.009202 120.45 0.000310 120.46 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644232396800.
WARNING: Using actual aggregate bytes moved = 22290432000.
write 176.47 157286400 100.00 0.001228 120.46 0.000333 120.46 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644241817600.
WARNING: Using actual aggregate bytes moved = 22330777600.
write 176.79 157286400 100.00 0.010521 120.45 0.000398 120.46 2
Max Write: 176.79 MiB/sec (185.38 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 176.79 172.18 175.15 2.10 120.45918 0 4 1 3 0 0 1 0 0 1 161061273600 102400 21748326400 MPIIO 0
Finished: Tue Oct 23 17:39:10 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 4 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 4 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 161061273600 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
+ tee -a ./output/COUNT:1#NN:4#PPN:1#API:MPIIO#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Tue Oct 23 17:39:14 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 161061273600 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 17:39:14 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 4 (1 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 150 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 6192947200.
read 49.05 157286400 100.00 0.008955 120.39 0.000342 120.40 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 6269952000.
read 49.67 157286400 100.00 0.000880 120.39 0.000382 120.39 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 6352691200.
read 50.32 157286400 100.00 0.000738 120.39 0.000425 120.39 2
Max Read: 50.32 MiB/sec (52.77 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 50.32 49.05 49.68 0.52 120.39366 0 4 1 3 0 0 1 0 0 1 161061273600 102400 6192947200 MPIIO 0
Finished: Tue Oct 23 17:45:15 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 8 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12595409 0xc030d1 0
0 12423800 0xbd9278 0
2 12570144 0xbfce20 0
4 12511882 0xbeea8a 0
7 12460511 0xbe21df 0
1 12641788 0xc0e5fc 0
3 12490091 0xbe956b 0
5 12382154 0xbcefca 0

110
output/COUNT:1#NN:4#PPN:1#API:MPIIO#T:1048576.txt Normal file
View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 4 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 161061273600 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:4#PPN:1#API:MPIIO#T:1048576.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Tue Oct 23 15:25:48 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 161061273600 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 15:25:48 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 4 (1 per node)
repetitions = 3
xfersize = 1 MiB
blocksize = 150 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644242997248.
WARNING: Using actual aggregate bytes moved = 212338737152.
write 1686.86 157286400 1024.00 0.006641 120.04 0.000478 120.05 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644245094400.
WARNING: Using actual aggregate bytes moved = 205848051712.
write 1634.24 157286400 1024.00 0.080504 120.04 0.000451 120.12 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644245094400.
WARNING: Using actual aggregate bytes moved = 206961639424.
write 1643.63 157286400 1024.00 0.019706 120.06 0.000495 120.08 2
Max Write: 1686.86 MiB/sec (1768.80 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 1686.86 1634.24 1654.91 22.91 120.08517 0 4 1 3 0 0 1 0 0 1 161061273600 1048576 212338737152 MPIIO 0
Finished: Tue Oct 23 15:31:55 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 4 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 4 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 161061273600 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
+ tee -a ./output/COUNT:1#NN:4#PPN:1#API:MPIIO#T:1048576.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Tue Oct 23 15:32:05 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 161061273600 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 15:32:05 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 4 (1 per node)
repetitions = 3
xfersize = 1 MiB
blocksize = 150 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 39368785920.
read 312.59 157286400 1024.00 0.058501 120.05 0.000358 120.11 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 41206939648.
read 327.35 157286400 1024.00 0.000652 120.05 0.000412 120.05 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 41827696640.
read 332.28 157286400 1024.00 0.000724 120.05 0.000473 120.05 2
Max Read: 332.28 MiB/sec (348.43 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 332.28 312.59 324.08 8.37 120.06845 0 4 1 3 0 0 1 0 0 1 161061273600 1048576 39368785920 MPIIO 0
Finished: Tue Oct 23 15:38:05 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 8 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12489823 0xbe945f 0
5 12381889 0xbceec1 0
6 12595143 0xc02fc7 0
0 12423533 0xbd916d 0
2 12569878 0xbfcd16 0
4 12511615 0xbee97f 0
7 12460245 0xbe20d5 0
1 12641521 0xc0e4f1 0

110
output/COUNT:1#NN:4#PPN:1#API:MPIIO#T:10485760.txt Normal file
View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 4 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 161061273600 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:4#PPN:1#API:MPIIO#T:10485760.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Tue Oct 23 13:20:29 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 161061273600 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 13:20:29 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 4 (1 per node)
repetitions = 3
xfersize = 10 MiB
blocksize = 150 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644161208320.
WARNING: Using actual aggregate bytes moved = 132414177280.
write 1050.21 157286400 10240 0.007375 120.23 0.000407 120.24 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644245094400.
WARNING: Using actual aggregate bytes moved = 129866137600.
write 1027.36 157286400 10240 0.007845 120.54 0.000372 120.55 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644161208320.
WARNING: Using actual aggregate bytes moved = 130233139200.
write 1033.68 157286400 10240 0.122595 120.03 0.000444 120.15 2
Max Write: 1050.21 MiB/sec (1101.22 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 1050.21 1027.36 1037.08 9.63 120.31594 0 4 1 3 0 0 1 0 0 1 161061273600 10485760 132414177280 MPIIO 0
Finished: Tue Oct 23 13:26:36 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 4 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 4 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 161061273600 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
+ tee -a ./output/COUNT:1#NN:4#PPN:1#API:MPIIO#T:10485760.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Tue Oct 23 13:26:45 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 161061273600 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 13:26:45 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 4 (1 per node)
repetitions = 3
xfersize = 10 MiB
blocksize = 150 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 126573608960.
read 1004.58 157286400 10240 0.096766 120.06 0.001348 120.16 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 130369454080.
read 1035.43 157286400 10240 0.001863 120.07 0.001404 120.08 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 130757427200.
read 1038.48 157286400 10240 0.001742 120.08 0.001492 120.08 2
Max Read: 1038.48 MiB/sec (1088.93 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 1038.48 1004.58 1026.16 15.32 120.10480 0 4 1 3 0 0 1 0 0 1 161061273600 10485760 126573608960 MPIIO 0
Finished: Tue Oct 23 13:32:45 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 8 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12511347 0xbee873 0
7 12459977 0xbe1fc9 0
1 12641255 0xc0e3e7 0
3 12489558 0xbe9356 0
5 12381621 0xbcedb5 0
6 12594876 0xc02ebc 0
0 12423268 0xbd9064 0
2 12569610 0xbfcc0a 0

110
output/COUNT:1#NN:4#PPN:1#API:MPIIO#T:16384.txt Normal file
View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 4 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 161061273600 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:4#PPN:1#API:MPIIO#T:16384.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Tue Oct 23 19:43:12 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 161061273600 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 19:43:12 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 4 (1 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 150 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644228169728.
WARNING: Using actual aggregate bytes moved = 4151771136.
write 32.23 157286400 16.00 0.008332 122.82 0.000333 122.83 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644238917632.
WARNING: Using actual aggregate bytes moved = 4132012032.
write 32.11 157286400 16.00 0.001266 122.72 0.000340 122.72 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644242669568.
WARNING: Using actual aggregate bytes moved = 4086202368.
write 31.73 157286400 16.00 0.001292 122.81 0.000405 122.81 2
Max Write: 32.23 MiB/sec (33.80 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 32.23 31.73 32.03 0.21 122.78557 0 4 1 3 0 0 1 0 0 1 161061273600 16384 4151771136 MPIIO 0
Finished: Tue Oct 23 19:49:21 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 4 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 4 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 161061273600 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
+ tee -a ./output/COUNT:1#NN:4#PPN:1#API:MPIIO#T:16384.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Tue Oct 23 19:49:24 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 161061273600 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 19:49:24 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 4 (1 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 150 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 1192853504.
read 9.27 157286400 16.00 0.008091 122.71 0.000308 122.72 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 1200439296.
read 9.33 157286400 16.00 0.000840 122.71 0.000319 122.71 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 1203126272.
read 9.35 157286400 16.00 0.000668 122.70 0.000378 122.71 2
Max Read: 9.35 MiB/sec (9.80 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 9.35 9.27 9.32 0.03 122.71224 0 4 1 3 0 0 1 0 0 1 161061273600 16384 1192853504 MPIIO 0
Finished: Tue Oct 23 19:55:32 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 8 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12595676 0xc031dc 0
0 12424067 0xbd9383 0
2 12570409 0xbfcf29 0
4 12512147 0xbeeb93 0
7 12460779 0xbe22eb 0
1 12642056 0xc0e708 0
3 12490355 0xbe9673 0
5 12382422 0xbcf0d6 0

156
output/COUNT:1#NN:4#PPN:1#API:POSIX#T:102400.txt Normal file
View File

@ -0,0 +1,156 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 4 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 161061273600 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:4#PPN:1#API:POSIX#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Tue Oct 23 17:19:56 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 161061273600 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 17:19:56 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 4 (1 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 150 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644245094400.
WARNING: Using actual aggregate bytes moved = 125717913600.
write 885.58 157286400 100.00 0.000318 135.38 0.000244 135.38 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644242636800.
WARNING: Using actual aggregate bytes moved = 120786944000.
write 854.88 157286400 100.00 0.001151 134.74 0.000649 134.75 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644244275200.
WARNING: Using actual aggregate bytes moved = 120288153600.
write 843.59 157286400 100.00 0.000671 135.98 0.000366 135.99 2
Max Write: 885.58 MiB/sec (928.60 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 885.58 843.59 861.35 17.74 135.37182 0 4 1 3 1 0 1 0 0 1 161061273600 102400 125717913600 POSIX 0
Finished: Tue Oct 23 17:26:51 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 4 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 4 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 161061273600 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread4/file -r
+ tee -a ./output/COUNT:1#NN:4#PPN:1#API:POSIX#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Tue Oct 23 17:27:06 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 161061273600 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread4/file -r
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 17:27:06 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/indread4/file
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 4 (1 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 150 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644245094400.
WARNING: Using actual aggregate bytes moved = 4841779200.
read 38.46 157286400 100.00 0.000376 120.07 0.000206 120.07 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644245094400.
WARNING: Using actual aggregate bytes moved = 4859494400.
read 38.60 157286400 100.00 0.000189 120.06 0.000252 120.06 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644245094400.
WARNING: Using actual aggregate bytes moved = 4985446400.
read 39.60 157286400 100.00 0.000351 120.06 0.000281 120.06 2
Max Read: 39.60 MiB/sec (41.52 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 39.60 38.46 38.89 0.51 120.06592 0 4 1 3 1 0 1 0 0 1 161061273600 102400 4841779200 POSIX 0
Finished: Tue Oct 23 17:33:07 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 8 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write.00000002
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12460504 0xbe21d8 0
6 12595404 0xc030cc 0
1 12641783 0xc0e5f7 0
5 12382150 0xbcefc6 0
3 12490087 0xbe9567 0
4 12511877 0xbeea85 0
0 12423796 0xbd9274 0
2 12570140 0xbfce1c 0
/esfs/jtacquaviva/ioperf/file_write.00000000
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12490085 0xbe9565 0
7 12460505 0xbe21d9 0
6 12595403 0xc030cb 0
5 12382149 0xbcefc5 0
1 12641782 0xc0e5f6 0
0 12423795 0xbd9273 0
2 12570139 0xbfce1b 0
4 12511878 0xbeea86 0
/esfs/jtacquaviva/ioperf/file_write.00000003
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12570138 0xbfce1a 0
5 12382148 0xbcefc4 0
0 12423794 0xbd9272 0
4 12511876 0xbeea84 0
3 12490086 0xbe9566 0
7 12460507 0xbe21db 0
6 12595405 0xc030cd 0
1 12641785 0xc0e5f9 0
/esfs/jtacquaviva/ioperf/file_write.00000001
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12460506 0xbe21da 0
1 12641784 0xc0e5f8 0
3 12490088 0xbe9568 0
5 12382151 0xbcefc7 0
6 12595406 0xc030ce 0
0 12423797 0xbd9275 0
2 12570141 0xbfce1d 0
4 12511879 0xbeea87 0

156
output/COUNT:1#NN:4#PPN:1#API:POSIX#T:1048576.txt Normal file
View File

@ -0,0 +1,156 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 4 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 161061273600 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:4#PPN:1#API:POSIX#T:1048576.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Tue Oct 23 15:13:18 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 161061273600 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 15:13:18 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 4 (1 per node)
repetitions = 3
xfersize = 1 MiB
blocksize = 150 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644245094400.
WARNING: Using actual aggregate bytes moved = 612789911552.
write 4815 157286400 1024.00 0.000327 121.36 0.000196 121.36 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644245094400.
WARNING: Using actual aggregate bytes moved = 633187860480.
write 4946 157286400 1024.00 0.006639 122.07 0.000347 122.08 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644245094400.
WARNING: Using actual aggregate bytes moved = 629982363648.
write 4948 157286400 1024.00 0.000828 121.42 0.000381 121.42 2
Max Write: 4948.20 MiB/sec (5188.57 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 4948.20 4815.35 4903.29 62.18 121.62049 0 4 1 3 1 0 1 0 0 1 161061273600 1048576 612789911552 POSIX 0
Finished: Tue Oct 23 15:19:34 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 4 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 4 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 161061273600 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread4/file -r
+ tee -a ./output/COUNT:1#NN:4#PPN:1#API:POSIX#T:1048576.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Tue Oct 23 15:19:47 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 161061273600 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread4/file -r
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 15:19:47 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/indread4/file
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 4 (1 per node)
repetitions = 3
xfersize = 1 MiB
blocksize = 150 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644245094400.
WARNING: Using actual aggregate bytes moved = 37196136448.
read 295.57 157286400 1024.00 0.000418 120.02 0.001498 120.02 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644245094400.
WARNING: Using actual aggregate bytes moved = 38987104256.
read 309.79 157286400 1024.00 0.001498 120.02 0.001716 120.02 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644245094400.
WARNING: Using actual aggregate bytes moved = 42807066624.
read 340.13 157286400 1024.00 0.001742 120.02 0.001961 120.03 2
Max Read: 340.13 MiB/sec (356.65 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 340.13 295.57 315.16 18.58 120.02006 0 4 1 3 1 0 1 0 0 1 161061273600 1048576 37196136448 POSIX 0
Finished: Tue Oct 23 15:25:47 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 8 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write.00000002
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12511610 0xbee97a 0
7 12460240 0xbe20d0 0
1 12641516 0xc0e4ec 0
3 12489818 0xbe945a 0
5 12381884 0xbceebc 0
6 12595138 0xc02fc2 0
0 12423528 0xbd9168 0
2 12569872 0xbfcd10 0
/esfs/jtacquaviva/ioperf/file_write.00000000
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12569871 0xbfcd0f 0
4 12511609 0xbee979 0
7 12460239 0xbe20cf 0
1 12641515 0xc0e4eb 0
3 12489817 0xbe9459 0
5 12381883 0xbceebb 0
6 12595137 0xc02fc1 0
0 12423527 0xbd9167 0
/esfs/jtacquaviva/ioperf/file_write.00000003
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12489819 0xbe945b 0
2 12569873 0xbfcd11 0
7 12460242 0xbe20d2 0
5 12381886 0xbceebe 0
6 12595140 0xc02fc4 0
4 12511612 0xbee97c 0
1 12641518 0xc0e4ee 0
0 12423530 0xbd916a 0
/esfs/jtacquaviva/ioperf/file_write.00000001
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12460241 0xbe20d1 0
5 12381885 0xbceebd 0
6 12595139 0xc02fc3 0
0 12423529 0xbd9169 0
4 12511611 0xbee97b 0
1 12641517 0xc0e4ed 0
3 12489820 0xbe945c 0
2 12569874 0xbfcd12 0

156
output/COUNT:1#NN:4#PPN:1#API:POSIX#T:10485760.txt Normal file
View File

@ -0,0 +1,156 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 4 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 161061273600 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:4#PPN:1#API:POSIX#T:10485760.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Tue Oct 23 13:08:00 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 161061273600 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 13:08:00 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 4 (1 per node)
repetitions = 3
xfersize = 10 MiB
blocksize = 150 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644245094400.
WARNING: Using actual aggregate bytes moved = 617181347840.
write 4851 157286400 10240 0.000324 121.32 0.000263 121.32 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644245094400.
WARNING: Using actual aggregate bytes moved = 612693442560.
write 4831 157286400 10240 0.000618 120.94 0.000348 120.94 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644245094400.
WARNING: Using actual aggregate bytes moved = 609358970880.
write 4797 157286400 10240 0.000704 121.15 0.000420 121.16 2
Max Write: 4851.50 MiB/sec (5087.16 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 4851.50 4796.56 4826.47 22.69 121.13935 0 4 1 3 1 0 1 0 0 1 161061273600 10485760 617181347840 POSIX 0
Finished: Tue Oct 23 13:14:14 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 4 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 4 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 161061273600 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread4/file -r
+ tee -a ./output/COUNT:1#NN:4#PPN:1#API:POSIX#T:10485760.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Tue Oct 23 13:14:27 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 161061273600 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread4/file -r
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 13:14:27 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/indread4/file
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 4 (1 per node)
repetitions = 3
xfersize = 10 MiB
blocksize = 150 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644245094400.
WARNING: Using actual aggregate bytes moved = 43065016320.
read 341.83 157286400 10240 0.000383 120.15 0.000185 120.15 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644245094400.
WARNING: Using actual aggregate bytes moved = 47448064000.
read 376.70 157286400 10240 0.000199 120.12 0.000268 120.12 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644245094400.
WARNING: Using actual aggregate bytes moved = 51097108480.
read 405.74 157286400 10240 0.000406 120.10 0.000342 120.10 2
Max Read: 405.74 MiB/sec (425.45 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 405.74 341.83 374.76 26.13 120.12427 0 4 1 3 1 0 1 0 0 1 161061273600 10485760 43065016320 POSIX 0
Finished: Tue Oct 23 13:20:28 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 8 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write.00000002
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12594871 0xc02eb7 0
4 12511344 0xbee870 0
5 12381616 0xbcedb0 0
7 12459973 0xbe1fc5 0
0 12423264 0xbd9060 0
2 12569607 0xbfcc07 0
1 12641252 0xc0e3e4 0
3 12489555 0xbe9353 0
/esfs/jtacquaviva/ioperf/file_write.00000000
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12511341 0xbee86d 0
3 12489552 0xbe9350 0
6 12594870 0xc02eb6 0
7 12459971 0xbe1fc3 0
5 12381615 0xbcedaf 0
0 12423262 0xbd905e 0
1 12641250 0xc0e3e2 0
2 12569605 0xbfcc05 0
/esfs/jtacquaviva/ioperf/file_write.00000003
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12569604 0xbfcc04 0
4 12511342 0xbee86e 0
6 12594872 0xc02eb8 0
7 12459972 0xbe1fc4 0
1 12641251 0xc0e3e3 0
0 12423265 0xbd9061 0
3 12489554 0xbe9352 0
5 12381617 0xbcedb1 0
/esfs/jtacquaviva/ioperf/file_write.00000001
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12641249 0xc0e3e1 0
5 12381614 0xbcedae 0
4 12511343 0xbee86f 0
3 12489553 0xbe9351 0
6 12594873 0xc02eb9 0
7 12459974 0xbe1fc6 0
0 12423263 0xbd905f 0
2 12569606 0xbfcc06 0

156
output/COUNT:1#NN:4#PPN:1#API:POSIX#T:16384.txt Normal file
View File

@ -0,0 +1,156 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 4 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 161061273600 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:4#PPN:1#API:POSIX#T:16384.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Tue Oct 23 19:29:35 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 161061273600 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 19:29:35 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 4 (1 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 150 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644244635648.
WARNING: Using actual aggregate bytes moved = 40813363200.
write 267.64 157286400 16.00 0.000310 145.43 0.000237 145.43 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644244176896.
WARNING: Using actual aggregate bytes moved = 35383197696.
write 225.17 157286400 16.00 0.000671 149.86 0.000399 149.86 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644244439040.
WARNING: Using actual aggregate bytes moved = 35950215168.
write 235.39 157286400 16.00 0.000748 145.65 0.000365 145.65 2
Max Write: 267.64 MiB/sec (280.64 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 267.64 225.17 242.73 18.10 146.98075 0 4 1 3 1 0 1 0 0 1 161061273600 16384 40813363200 POSIX 0
Finished: Tue Oct 23 19:37:01 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 4 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 1 -np 4 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 161061273600 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread4/file -r
+ tee -a ./output/COUNT:1#NN:4#PPN:1#API:POSIX#T:16384.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Tue Oct 23 19:37:10 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 161061273600 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread4/file -r
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 19:37:10 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/indread4/file
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 4 (1 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 150 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644245094400.
WARNING: Using actual aggregate bytes moved = 863141888.
read 6.82 157286400 16.00 0.000385 120.61 0.000198 120.62 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644245094400.
WARNING: Using actual aggregate bytes moved = 928907264.
read 7.34 157286400 16.00 0.000210 120.63 0.000257 120.63 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644245094400.
WARNING: Using actual aggregate bytes moved = 960020480.
read 7.59 157286400 16.00 0.000359 120.61 0.000311 120.61 2
Max Read: 7.59 MiB/sec (7.96 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 7.59 6.82 7.25 0.32 120.61795 0 4 1 3 1 0 1 0 0 1 161061273600 16384 863141888 POSIX 0
Finished: Tue Oct 23 19:43:12 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 8 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write.00000002
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12424061 0xbd937d 0
2 12570404 0xbfcf24 0
5 12382417 0xbcf0d1 0
4 12512142 0xbeeb8e 0
7 12460774 0xbe22e6 0
3 12490352 0xbe9670 0
1 12642053 0xc0e705 0
6 12595673 0xc031d9 0
/esfs/jtacquaviva/ioperf/file_write.00000000
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12642050 0xc0e702 0
3 12490350 0xbe966e 0
7 12460773 0xbe22e5 0
5 12382418 0xbcf0d2 0
6 12595672 0xc031d8 0
0 12424064 0xbd9380 0
2 12570406 0xbfcf26 0
4 12512144 0xbeeb90 0
/esfs/jtacquaviva/ioperf/file_write.00000003
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12382416 0xbcf0d0 0
3 12490349 0xbe966d 0
6 12595670 0xc031d6 0
1 12642051 0xc0e703 0
0 12424062 0xbd937e 0
2 12570405 0xbfcf25 0
4 12512143 0xbeeb8f 0
7 12460775 0xbe22e7 0
/esfs/jtacquaviva/ioperf/file_write.00000001
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12512141 0xbeeb8d 0
7 12460772 0xbe22e4 0
2 12570403 0xbfcf23 0
1 12642052 0xc0e704 0
3 12490351 0xbe966f 0
6 12595671 0xc031d7 0
0 12424063 0xbd937f 0
5 12382419 0xbcf0d3 0

110
output/COUNT:1#NN:4#PPN:2#API:MPIIO#T:102400.txt Normal file
View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 80530636800 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:4#PPN:2#API:MPIIO#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Tue Oct 23 17:07:42 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 80530636800 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 17:07:42 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 8 (2 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 75 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644238848000.
WARNING: Using actual aggregate bytes moved = 26757836800.
write 211.65 78643200 100.00 0.014009 120.55 0.000766 120.57 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644245094400.
WARNING: Using actual aggregate bytes moved = 26511564800.
write 210.05 78643200 100.00 0.001410 120.37 0.000429 120.37 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644243968000.
WARNING: Using actual aggregate bytes moved = 26426470400.
write 209.31 78643200 100.00 0.001454 120.41 0.000524 120.41 2
Max Write: 211.65 MiB/sec (221.93 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 211.65 209.31 210.34 0.98 120.44825 0 8 2 3 0 0 1 0 0 1 80530636800 102400 26757836800 MPIIO 0
Finished: Tue Oct 23 17:13:45 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 80530636800 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
+ tee -a ./output/COUNT:1#NN:4#PPN:2#API:MPIIO#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Tue Oct 23 17:13:55 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 80530636800 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 17:13:55 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 8 (2 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 75 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 11646873600.
read 92.27 78643200 100.00 0.012915 120.37 0.000757 120.38 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 11850444800.
read 93.90 78643200 100.00 0.001423 120.36 0.000799 120.36 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 12047564800.
read 95.45 78643200 100.00 0.001225 120.37 0.000891 120.37 2
Max Read: 95.45 MiB/sec (100.09 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 95.45 92.27 93.87 1.30 120.36974 0 8 2 3 0 0 1 0 0 1 80530636800 102400 11646873600 MPIIO 0
Finished: Tue Oct 23 17:19:56 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 8 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12423784 0xbd9268 0
2 12570129 0xbfce11 0
4 12511867 0xbeea7b 0
7 12460495 0xbe21cf 0
1 12641773 0xc0e5ed 0
3 12490076 0xbe955c 0
5 12382139 0xbcefbb 0
6 12595394 0xc030c2 0

110
output/COUNT:1#NN:4#PPN:2#API:MPIIO#T:1048576.txt Normal file
View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 80530636800 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:4#PPN:2#API:MPIIO#T:1048576.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Tue Oct 23 15:00:47 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 80530636800 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 15:00:47 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 8 (2 per node)
repetitions = 3
xfersize = 1 MiB
blocksize = 75 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644242997248.
WARNING: Using actual aggregate bytes moved = 283739422720.
write 2253.94 78643200 1024.00 0.015969 120.04 0.000874 120.05 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644245094400.
WARNING: Using actual aggregate bytes moved = 299593891840.
write 2272.56 78643200 1024.00 5.67 120.06 0.000470 125.72 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644245094400.
WARNING: Using actual aggregate bytes moved = 289455210496.
write 2279.21 78643200 1024.00 0.210202 120.90 0.000569 121.11 2
Max Write: 2279.21 MiB/sec (2389.92 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 2279.21 2253.94 2268.57 10.69 122.29775 0 8 2 3 0 0 1 0 0 1 80530636800 1048576 283739422720 MPIIO 0
Finished: Tue Oct 23 15:07:05 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 80530636800 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
+ tee -a ./output/COUNT:1#NN:4#PPN:2#API:MPIIO#T:1048576.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Tue Oct 23 15:07:17 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 80530636800 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 15:07:17 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 8 (2 per node)
repetitions = 3
xfersize = 1 MiB
blocksize = 75 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 67901587456.
read 538.53 78643200 1024.00 0.185210 120.06 0.002948 120.24 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 71020052480.
read 564.13 78643200 1024.00 0.003314 120.06 0.002985 120.06 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 71370276864.
read 566.94 78643200 1024.00 0.003374 120.05 0.003087 120.05 2
Max Read: 566.94 MiB/sec (594.48 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 566.94 538.53 556.54 12.78 120.12012 0 8 2 3 0 0 1 0 0 1 80530636800 1048576 67901587456 MPIIO 0
Finished: Tue Oct 23 15:13:17 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 8 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12569861 0xbfcd05 0
4 12511600 0xbee970 0
7 12460230 0xbe20c6 0
1 12641506 0xc0e4e2 0
3 12489808 0xbe9450 0
5 12381874 0xbceeb2 0
6 12595128 0xc02fb8 0
0 12423518 0xbd915e 0

110
output/COUNT:1#NN:4#PPN:2#API:MPIIO#T:10485760.txt Normal file
View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 80530636800 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:4#PPN:2#API:MPIIO#T:10485760.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Tue Oct 23 12:55:32 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 80530636800 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 12:55:32 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 8 (2 per node)
repetitions = 3
xfersize = 10 MiB
blocksize = 75 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644234608640.
WARNING: Using actual aggregate bytes moved = 179159695360.
write 1420.25 78643200 10240 0.007930 120.29 0.006416 120.30 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644182179840.
WARNING: Using actual aggregate bytes moved = 176527769600.
write 1348.98 78643200 10240 4.57 120.23 0.000428 124.80 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644213637120.
WARNING: Using actual aggregate bytes moved = 174766161920.
write 1383.91 78643200 10240 0.158274 120.28 0.000552 120.43 2
Max Write: 1420.25 MiB/sec (1489.24 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 1420.25 1348.98 1384.38 29.10 121.84475 0 8 2 3 0 0 1 0 0 1 80530636800 10485760 179159695360 MPIIO 0
Finished: Tue Oct 23 13:01:47 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 80530636800 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
+ tee -a ./output/COUNT:1#NN:4#PPN:2#API:MPIIO#T:10485760.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Tue Oct 23 13:01:58 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 80530636800 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 13:01:58 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 8 (2 per node)
repetitions = 3
xfersize = 10 MiB
blocksize = 75 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 197813862400.
read 1569.28 78643200 10240 0.136744 120.07 0.011313 120.21 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 201053962240.
read 1597.15 78643200 10240 0.004045 120.05 0.003663 120.05 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 202113024000.
read 1605.54 78643200 10240 0.004049 120.05 0.003656 120.05 2
Max Read: 1605.54 MiB/sec (1683.53 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 1605.54 1569.28 1590.66 15.50 120.10623 0 8 2 3 0 0 1 0 0 1 80530636800 10485760 197813862400 MPIIO 0
Finished: Tue Oct 23 13:07:58 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 8 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12511331 0xbee863 0
7 12459962 0xbe1fba 0
1 12641240 0xc0e3d8 0
3 12489543 0xbe9347 0
5 12381605 0xbceda5 0
6 12594861 0xc02ead 0
0 12423253 0xbd9055 0
2 12569595 0xbfcbfb 0

110
output/COUNT:1#NN:4#PPN:2#API:MPIIO#T:16384.txt Normal file
View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 80530636800 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:4#PPN:2#API:MPIIO#T:16384.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Tue Oct 23 19:17:13 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 80530636800 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 19:17:13 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 8 (2 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 75 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644244897792.
WARNING: Using actual aggregate bytes moved = 4958371840.
write 38.65 78643200 16.00 0.010314 122.33 0.000427 122.34 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644244963328.
WARNING: Using actual aggregate bytes moved = 4869046272.
write 37.92 78643200 16.00 0.001364 122.46 0.000469 122.46 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644244373504.
WARNING: Using actual aggregate bytes moved = 4879908864.
write 38.01 78643200 16.00 0.001414 122.43 0.000551 122.43 2
Max Write: 38.65 MiB/sec (40.53 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 38.65 37.92 38.19 0.33 122.41003 0 8 2 3 0 0 1 0 0 1 80530636800 16384 4958371840 MPIIO 0
Finished: Tue Oct 23 19:23:21 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 80530636800 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
+ tee -a ./output/COUNT:1#NN:4#PPN:2#API:MPIIO#T:16384.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Tue Oct 23 19:23:27 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 80530636800 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 19:23:27 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 8 (2 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 75 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 2316926976.
read 18.06 78643200 16.00 0.012380 122.33 0.000689 122.34 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 2329788416.
read 18.16 78643200 16.00 0.001141 122.32 0.000642 122.32 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 2330476544.
read 18.17 78643200 16.00 0.001050 122.32 0.000578 122.32 2
Max Read: 18.17 MiB/sec (19.05 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 18.17 18.06 18.13 0.05 122.32860 0 8 2 3 0 0 1 0 0 1 80530636800 16384 2316926976 MPIIO 0
Finished: Tue Oct 23 19:29:34 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 8 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12642040 0xc0e6f8 0
3 12490341 0xbe9665 0
5 12382407 0xbcf0c7 0
6 12595661 0xc031cd 0
0 12424052 0xbd9374 0
2 12570394 0xbfcf1a 0
4 12512132 0xbeeb84 0
7 12460763 0xbe22db 0

215
output/COUNT:1#NN:4#PPN:2#API:POSIX#T:102400.txt Normal file
View File

@ -0,0 +1,215 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 80530636800 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:4#PPN:2#API:POSIX#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Tue Oct 23 16:54:31 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 80530636800 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 16:54:31 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 8 (2 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 75 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644236902400.
WARNING: Using actual aggregate bytes moved = 117236531200.
write 828.06 78643200 100.00 0.000699 135.02 0.000316 135.02 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644240998400.
WARNING: Using actual aggregate bytes moved = 111485132800.
write 757.67 78643200 100.00 0.000920 140.33 0.000422 140.33 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644242636800.
WARNING: Using actual aggregate bytes moved = 106132992000.
write 740.69 78643200 100.00 0.001038 136.65 0.000458 136.65 2
Max Write: 828.06 MiB/sec (868.29 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 828.06 740.69 775.48 37.83 137.33232 0 8 2 3 1 0 1 0 0 1 80530636800 102400 117236531200 POSIX 0
Finished: Tue Oct 23 17:01:29 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 80530636800 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread4/file -r
+ tee -a ./output/COUNT:1#NN:4#PPN:2#API:POSIX#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Tue Oct 23 17:01:41 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 80530636800 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread4/file -r
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 17:01:41 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/indread4/file
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 8 (2 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 75 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 1288490188800.
WARNING: Using actual aggregate bytes moved = 11224780800.
read 89.17 78643200 100.00 0.001026 120.05 0.000745 120.05 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 1288490188800.
WARNING: Using actual aggregate bytes moved = 11294924800.
read 89.74 78643200 100.00 0.000736 120.03 0.000791 120.03 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 1288490188800.
WARNING: Using actual aggregate bytes moved = 11951308800.
read 94.95 78643200 100.00 0.000792 120.04 0.000783 120.04 2
Max Read: 94.95 MiB/sec (99.56 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 94.95 89.17 91.29 2.60 120.04057 0 8 2 3 1 0 1 0 0 1 80530636800 102400 11224780800 POSIX 0
Finished: Tue Oct 23 17:07:41 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 8 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write.00000006
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12511860 0xbeea74 0
3 12490070 0xbe9556 0
5 12382133 0xbcefb5 0
7 12460488 0xbe21c8 0
6 12595387 0xc030bb 0
2 12570123 0xbfce0b 0
0 12423777 0xbd9261 0
1 12641766 0xc0e5e6 0
/esfs/jtacquaviva/ioperf/file_write.00000002
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12570120 0xbfce08 0
1 12641762 0xc0e5e2 0
5 12382130 0xbcefb2 0
0 12423774 0xbd925e 0
4 12511858 0xbeea72 0
3 12490068 0xbe9554 0
6 12595385 0xc030b9 0
/esfs/jtacquaviva/ioperf/file_write.00000000
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12641769 0xc0e5e9 0
3 12490073 0xbe9559 0
5 12382136 0xbcefb8 0
6 12595391 0xc030bf 0
0 12423781 0xbd9265 0
2 12570126 0xbfce0e 0
4 12511864 0xbeea78 0
7 12460492 0xbe21cc 0
/esfs/jtacquaviva/ioperf/file_write.00000004
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12490071 0xbe9557 0
5 12382134 0xbcefb6 0
7 12460490 0xbe21ca 0
6 12595388 0xc030bc 0
0 12423780 0xbd9264 0
1 12641768 0xc0e5e8 0
4 12511863 0xbeea77 0
/esfs/jtacquaviva/ioperf/file_write.00000007
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12570122 0xbfce0a 0
4 12511859 0xbeea73 0
7 12460487 0xbe21c7 0
1 12641764 0xc0e5e4 0
3 12490069 0xbe9555 0
5 12382132 0xbcefb4 0
6 12595386 0xc030ba 0
0 12423776 0xbd9260 0
/esfs/jtacquaviva/ioperf/file_write.00000003
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12423778 0xbd9262 0
4 12511861 0xbeea75 0
2 12570125 0xbfce0d 0
3 12490072 0xbe9558 0
1 12641767 0xc0e5e7 0
6 12595390 0xc030be 0
7 12460491 0xbe21cb 0
/esfs/jtacquaviva/ioperf/file_write.00000005
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12460486 0xbe21c6 0
3 12490067 0xbe9553 0
6 12595384 0xc030b8 0
2 12570121 0xbfce09 0
1 12641763 0xc0e5e3 0
5 12382131 0xbcefb3 0
0 12423775 0xbd925f 0
/esfs/jtacquaviva/ioperf/file_write.00000001
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12641765 0xc0e5e5 0
2 12570124 0xbfce0c 0
0 12423779 0xbd9263 0
7 12460489 0xbe21c9 0
4 12511862 0xbeea76 0
5 12382135 0xbcefb7 0
6 12595389 0xc030bd 0

219
output/COUNT:1#NN:4#PPN:2#API:POSIX#T:1048576.txt Normal file
View File

@ -0,0 +1,219 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 80530636800 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:4#PPN:2#API:POSIX#T:1048576.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Tue Oct 23 14:48:24 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 80530636800 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 14:48:24 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 8 (2 per node)
repetitions = 3
xfersize = 1 MiB
blocksize = 75 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644245094400.
WARNING: Using actual aggregate bytes moved = 608387989504.
write 4786 78643200 1024.00 0.000725 121.22 0.000423 121.22 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644245094400.
WARNING: Using actual aggregate bytes moved = 609847607296.
write 4797 78643200 1024.00 0.001124 121.24 0.000509 121.25 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644240900096.
WARNING: Using actual aggregate bytes moved = 611880796160.
write 4814 78643200 1024.00 0.001115 121.21 0.000595 121.21 2
Max Write: 4814.17 MiB/sec (5048.02 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 4814.17 4786.34 4799.12 11.47 121.22616 0 8 2 3 1 0 1 0 0 1 80530636800 1048576 608387989504 POSIX 0
Finished: Tue Oct 23 14:54:35 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 80530636800 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread4/file -r
+ tee -a ./output/COUNT:1#NN:4#PPN:2#API:POSIX#T:1048576.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Tue Oct 23 14:54:46 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 80530636800 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread4/file -r
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 14:54:46 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/indread4/file
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 8 (2 per node)
repetitions = 3
xfersize = 1 MiB
blocksize = 75 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 1288490188800.
WARNING: Using actual aggregate bytes moved = 84376813568.
read 670.48 78643200 1024.00 0.000865 120.01 0.000685 120.01 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 1288490188800.
WARNING: Using actual aggregate bytes moved = 96340017152.
read 765.49 78643200 1024.00 0.000597 120.02 0.000973 120.02 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 1288490188800.
WARNING: Using actual aggregate bytes moved = 108033736704.
read 858.42 78643200 1024.00 0.000708 120.02 0.000751 120.02 2
Max Read: 858.42 MiB/sec (900.12 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 858.42 670.48 764.80 76.73 120.02034 0 8 2 3 1 0 1 0 0 1 80530636800 1048576 84376813568 POSIX 0
Finished: Tue Oct 23 15:00:46 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 8 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write.00000006
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12489798 0xbe9446 0
7 12460221 0xbe20bd 0
5 12381865 0xbceea9 0
1 12641497 0xc0e4d9 0
0 12423510 0xbd9156 0
6 12595121 0xc02fb1 0
2 12569854 0xbfccfe 0
/esfs/jtacquaviva/ioperf/file_write.00000002
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12569851 0xbfccfb 0
4 12511593 0xbee969 0
6 12595120 0xc02fb0 0
7 12460223 0xbe20bf 0
1 12641499 0xc0e4db 0
5 12381867 0xbceeab 0
3 12489800 0xbe9448 0
0 12423511 0xbd9157 0
/esfs/jtacquaviva/ioperf/file_write.00000000
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12460220 0xbe20bc 0
4 12511591 0xbee967 0
1 12641496 0xc0e4d8 0
3 12489797 0xbe9445 0
6 12595118 0xc02fae 0
5 12381864 0xbceea8 0
2 12569852 0xbfccfc 0
0 12423509 0xbd9155 0
/esfs/jtacquaviva/ioperf/file_write.00000004
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12511594 0xbee96a 0
7 12460224 0xbe20c0 0
1 12641500 0xc0e4dc 0
3 12489801 0xbe9449 0
5 12381868 0xbceeac 0
6 12595122 0xc02fb2 0
0 12423512 0xbd9158 0
2 12569855 0xbfccff 0
/esfs/jtacquaviva/ioperf/file_write.00000007
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12489804 0xbe944c 0
5 12381871 0xbceeaf 0
6 12595125 0xc02fb5 0
0 12423515 0xbd915b 0
2 12569858 0xbfcd02 0
4 12511597 0xbee96d 0
7 12460227 0xbe20c3 0
1 12641503 0xc0e4df 0
/esfs/jtacquaviva/ioperf/file_write.00000003
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12460225 0xbe20c1 0
1 12641501 0xc0e4dd 0
3 12489802 0xbe944a 0
5 12381869 0xbceead 0
6 12595123 0xc02fb3 0
0 12423513 0xbd9159 0
2 12569856 0xbfcd00 0
4 12511595 0xbee96b 0
/esfs/jtacquaviva/ioperf/file_write.00000005
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12595119 0xc02faf 0
0 12423508 0xbd9154 0
4 12511592 0xbee968 0
3 12489799 0xbe9447 0
2 12569853 0xbfccfd 0
7 12460222 0xbe20be 0
1 12641498 0xc0e4da 0
5 12381866 0xbceeaa 0
/esfs/jtacquaviva/ioperf/file_write.00000001
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12641502 0xc0e4de 0
3 12489803 0xbe944b 0
5 12381870 0xbceeae 0
6 12595124 0xc02fb4 0
0 12423514 0xbd915a 0
2 12569857 0xbfcd01 0
4 12511596 0xbee96c 0
7 12460226 0xbe20c2 0

217
output/COUNT:1#NN:4#PPN:2#API:POSIX#T:10485760.txt Normal file
View File

@ -0,0 +1,217 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 80530636800 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:4#PPN:2#API:POSIX#T:10485760.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Tue Oct 23 12:43:14 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 80530636800 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 12:43:14 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 8 (2 per node)
repetitions = 3
xfersize = 10 MiB
blocksize = 75 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644245094400.
WARNING: Using actual aggregate bytes moved = 641550254080.
write 5089 78643200 10240 0.000573 120.22 0.000230 120.23 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644245094400.
WARNING: Using actual aggregate bytes moved = 635091025920.
write 5024 78643200 10240 0.004379 120.55 0.000278 120.56 1
write 5143 78643200 10240 0.001038 119.46 0.000359 119.46 2
Max Write: 5143.06 MiB/sec (5392.89 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 5143.06 5023.89 5085.32 48.72 120.08182 0 8 2 3 1 0 1 0 0 1 80530636800 10485760 641550254080 POSIX 0
Finished: Tue Oct 23 12:49:21 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 80530636800 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread4/file -r
+ tee -a ./output/COUNT:1#NN:4#PPN:2#API:POSIX#T:10485760.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Tue Oct 23 12:49:31 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 80530636800 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread4/file -r
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 12:49:31 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/indread4/file
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 8 (2 per node)
repetitions = 3
xfersize = 10 MiB
blocksize = 75 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 1288490188800.
WARNING: Using actual aggregate bytes moved = 98545172480.
read 782.41 78643200 10240 0.000781 120.11 0.000504 120.12 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 1288490188800.
WARNING: Using actual aggregate bytes moved = 114514984960.
read 909.11 78643200 10240 0.000445 120.13 0.000691 120.13 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 1288490188800.
WARNING: Using actual aggregate bytes moved = 125745233920.
read 998.59 78643200 10240 0.000428 120.09 0.000529 120.09 2
Max Read: 998.59 MiB/sec (1047.10 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 998.59 782.41 896.70 88.69 120.11108 0 8 2 3 1 0 1 0 0 1 80530636800 10485760 98545172480 POSIX 0
Finished: Tue Oct 23 12:55:31 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 8 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write.00000006
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12511324 0xbee85c 0
7 12459955 0xbe1fb3 0
1 12641233 0xc0e3d1 0
3 12489536 0xbe9340 0
5 12381597 0xbced9d 0
6 12594854 0xc02ea6 0
0 12423246 0xbd904e 0
2 12569588 0xbfcbf4 0
/esfs/jtacquaviva/ioperf/file_write.00000002
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12459956 0xbe1fb4 0
1 12641234 0xc0e3d2 0
3 12489537 0xbe9341 0
5 12381598 0xbced9e 0
6 12594855 0xc02ea7 0
0 12423247 0xbd904f 0
2 12569589 0xbfcbf5 0
4 12511325 0xbee85d 0
/esfs/jtacquaviva/ioperf/file_write.00000000
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12381594 0xbced9a 0
7 12459952 0xbe1fb0 0
3 12489533 0xbe933d 0
6 12594851 0xc02ea3 0
4 12511323 0xbee85b 0
0 12423244 0xbd904c 0
1 12641232 0xc0e3d0 0
2 12569587 0xbfcbf3 0
/esfs/jtacquaviva/ioperf/file_write.00000004
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12641235 0xc0e3d3 0
3 12489538 0xbe9342 0
5 12381599 0xbced9f 0
6 12594856 0xc02ea8 0
0 12423248 0xbd9050 0
2 12569590 0xbfcbf6 0
4 12511326 0xbee85e 0
7 12459957 0xbe1fb5 0
/esfs/jtacquaviva/ioperf/file_write.00000007
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12381601 0xbceda1 0
6 12594858 0xc02eaa 0
0 12423250 0xbd9052 0
2 12569592 0xbfcbf8 0
4 12511328 0xbee860 0
7 12459959 0xbe1fb7 0
1 12641237 0xc0e3d5 0
3 12489540 0xbe9344 0
/esfs/jtacquaviva/ioperf/file_write.00000003
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12423243 0xbd904b 0
1 12641230 0xc0e3ce 0
2 12569585 0xbfcbf1 0
3 12489534 0xbe933e 0
4 12511322 0xbee85a 0
5 12381596 0xbced9c 0
6 12594852 0xc02ea4 0
7 12459954 0xbe1fb2 0
/esfs/jtacquaviva/ioperf/file_write.00000005
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12511321 0xbee859 0
7 12459953 0xbe1fb1 0
2 12569586 0xbfcbf2 0
3 12489535 0xbe933f 0
5 12381595 0xbced9b 0
1 12641231 0xc0e3cf 0
6 12594853 0xc02ea5 0
0 12423245 0xbd904d 0
/esfs/jtacquaviva/ioperf/file_write.00000001
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12489539 0xbe9343 0
5 12381600 0xbceda0 0
6 12594857 0xc02ea9 0
0 12423249 0xbd9051 0
2 12569591 0xbfcbf7 0
4 12511327 0xbee85f 0
7 12459958 0xbe1fb6 0
1 12641236 0xc0e3d4 0

219
output/COUNT:1#NN:4#PPN:2#API:POSIX#T:16384.txt Normal file
View File

@ -0,0 +1,219 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 80530636800 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:4#PPN:2#API:POSIX#T:16384.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Tue Oct 23 19:03:29 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 80530636800 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 19:03:29 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 8 (2 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 75 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644243685376.
WARNING: Using actual aggregate bytes moved = 40747286528.
write 263.33 78643200 16.00 0.000714 147.57 0.000358 147.57 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644244963328.
WARNING: Using actual aggregate bytes moved = 35523215360.
write 219.55 78643200 16.00 0.001083 154.30 0.000380 154.31 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644244832256.
WARNING: Using actual aggregate bytes moved = 34455846912.
write 217.47 78643200 16.00 0.033717 151.07 0.000377 151.10 2
Max Write: 263.33 MiB/sec (276.12 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 263.33 217.47 233.45 21.14 150.99379 0 8 2 3 1 0 1 0 0 1 80530636800 16384 40747286528 POSIX 0
Finished: Tue Oct 23 19:11:04 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 2 -np 8 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 80530636800 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread4/file -r
+ tee -a ./output/COUNT:1#NN:4#PPN:2#API:POSIX#T:16384.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Tue Oct 23 19:11:11 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 80530636800 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread4/file -r
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 19:11:11 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/indread4/file
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 8 (2 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 75 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 1288490188800.
WARNING: Using actual aggregate bytes moved = 2004221952.
read 15.89 78643200 16.00 0.006006 120.28 0.005699 120.28 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 1288490188800.
WARNING: Using actual aggregate bytes moved = 2332983296.
read 18.50 78643200 16.00 0.005696 120.28 0.005706 120.28 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 1288490188800.
WARNING: Using actual aggregate bytes moved = 2369224704.
read 18.79 78643200 16.00 0.005829 120.28 0.005733 120.28 2
Max Read: 18.79 MiB/sec (19.70 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 18.79 15.89 17.72 1.30 120.28114 0 8 2 3 1 0 1 0 0 1 80530636800 16384 2004221952 POSIX 0
Finished: Tue Oct 23 19:17:12 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 8 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write.00000006
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12424045 0xbd936d 0
7 12460757 0xbe22d5 0
3 12490335 0xbe965f 0
6 12595655 0xc031c7 0
2 12570388 0xbfcf14 0
1 12642035 0xc0e6f3 0
5 12382402 0xbcf0c2 0
4 12512127 0xbeeb7f 0
/esfs/jtacquaviva/ioperf/file_write.00000002
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12512126 0xbeeb7e 0
1 12642034 0xc0e6f2 0
5 12382401 0xbcf0c1 0
0 12424046 0xbd936e 0
7 12460759 0xbe22d7 0
3 12490336 0xbe9660 0
6 12595656 0xc031c8 0
2 12570390 0xbfcf16 0
/esfs/jtacquaviva/ioperf/file_write.00000000
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12595651 0xc031c3 0
2 12570385 0xbfcf11 0
0 12424042 0xbd936a 0
1 12642031 0xc0e6ef 0
4 12512123 0xbeeb7b 0
5 12382399 0xbcf0bf 0
7 12460755 0xbe22d3 0
/esfs/jtacquaviva/ioperf/file_write.00000004
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12424047 0xbd936f 0
2 12570389 0xbfcf15 0
5 12382404 0xbcf0c4 0
4 12512128 0xbeeb80 0
7 12460760 0xbe22d8 0
1 12642037 0xc0e6f5 0
3 12490338 0xbe9662 0
6 12595658 0xc031ca 0
/esfs/jtacquaviva/ioperf/file_write.00000007
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12512122 0xbeeb7a 0
7 12460753 0xbe22d1 0
1 12642030 0xc0e6ee 0
3 12490332 0xbe965c 0
0 12424041 0xbd9369 0
2 12570384 0xbfcf10 0
5 12382397 0xbcf0bd 0
6 12595652 0xc031c4 0
/esfs/jtacquaviva/ioperf/file_write.00000003
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12460754 0xbe22d2 0
3 12490333 0xbe965d 0
1 12642032 0xc0e6f0 0
6 12595654 0xc031c6 0
0 12424043 0xbd936b 0
4 12512125 0xbeeb7d 0
2 12570387 0xbfcf13 0
5 12382400 0xbcf0c0 0
/esfs/jtacquaviva/ioperf/file_write.00000005
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12382398 0xbcf0be 0
6 12595653 0xc031c5 0
4 12512124 0xbeeb7c 0
3 12490334 0xbe965e 0
7 12460756 0xbe22d4 0
2 12570386 0xbfcf12 0
0 12424044 0xbd936c 0
1 12642033 0xc0e6f1 0
/esfs/jtacquaviva/ioperf/file_write.00000001
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12460758 0xbe22d6 0
1 12642036 0xc0e6f4 0
3 12490337 0xbe9661 0
5 12382403 0xbcf0c3 0
2 12570391 0xbfcf17 0
4 12512129 0xbeeb81 0
6 12595657 0xc031c9 0
0 12424048 0xbd9370 0

110
output/COUNT:1#NN:4#PPN:4#API:MPIIO#T:102400.txt Normal file
View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 40265318400 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:4#PPN:4#API:MPIIO#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Tue Oct 23 16:42:14 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 40265318400 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 16:42:14 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 16 (4 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 37.50 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644244787200.
WARNING: Using actual aggregate bytes moved = 33528627200.
write 265.42 39321600 100.00 0.020465 120.45 0.000569 120.47 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644238336000.
WARNING: Using actual aggregate bytes moved = 34003046400.
write 268.91 39321600 100.00 0.002097 120.59 0.000756 120.59 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644244582400.
WARNING: Using actual aggregate bytes moved = 33401344000.
write 264.50 39321600 100.00 0.002320 120.43 0.000827 120.43 2
Max Write: 268.91 MiB/sec (281.97 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 268.91 264.50 266.28 1.90 120.49698 0 16 4 3 0 0 1 0 0 1 40265318400 102400 33528627200 MPIIO 0
Finished: Tue Oct 23 16:48:18 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 40265318400 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
+ tee -a ./output/COUNT:1#NN:4#PPN:4#API:MPIIO#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Tue Oct 23 16:48:29 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 40265318400 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 16:48:29 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 16 (4 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 37.50 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 20974080000.
read 166.16 39321600 100.00 0.021844 120.36 0.000895 120.38 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 21722624000.
read 172.13 39321600 100.00 0.001451 120.35 0.000886 120.36 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 22202163200.
read 175.91 39321600 100.00 0.001512 120.37 0.000953 120.37 2
Max Read: 175.91 MiB/sec (184.45 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 175.91 166.16 171.40 4.01 120.36819 0 16 4 3 0 0 1 0 0 1 40265318400 102400 20974080000 MPIIO 0
Finished: Tue Oct 23 16:54:30 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 8 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12490049 0xbe9541 0
5 12382113 0xbcefa1 0
6 12595367 0xc030a7 0
0 12423757 0xbd924d 0
2 12570103 0xbfcdf7 0
4 12511841 0xbeea61 0
7 12460469 0xbe21b5 0
1 12641746 0xc0e5d2 0

110
output/COUNT:1#NN:4#PPN:4#API:MPIIO#T:1048576.txt Normal file
View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 40265318400 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:4#PPN:4#API:MPIIO#T:1048576.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Tue Oct 23 14:35:46 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 40265318400 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 14:35:46 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 16 (4 per node)
repetitions = 3
xfersize = 1 MiB
blocksize = 37.50 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644244045824.
WARNING: Using actual aggregate bytes moved = 411177058304.
write 3265.72 39321600 1024.00 0.011774 120.06 0.000838 120.07 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644244045824.
WARNING: Using actual aggregate bytes moved = 407516479488.
write 3101.02 39321600 1024.00 5.26 120.06 0.000540 125.33 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644241948672.
WARNING: Using actual aggregate bytes moved = 397838123008.
write 3014.75 39321600 1024.00 5.81 120.04 0.000569 125.85 2
Max Write: 3265.72 MiB/sec (3424.36 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 3265.72 3014.75 3127.17 104.11 123.75013 0 16 4 3 0 0 1 0 0 1 40265318400 1048576 411177058304 MPIIO 0
Finished: Tue Oct 23 14:42:10 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 40265318400 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
+ tee -a ./output/COUNT:1#NN:4#PPN:4#API:MPIIO#T:1048576.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Tue Oct 23 14:42:22 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 40265318400 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 14:42:22 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 16 (4 per node)
repetitions = 3
xfersize = 1 MiB
blocksize = 37.50 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 108108185600.
read 857.67 39321600 1024.00 0.144430 120.06 0.000746 120.21 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 111812804608.
read 888.08 39321600 1024.00 0.001363 120.07 0.000696 120.07 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 112235380736.
read 891.52 39321600 1024.00 0.001251 120.06 0.000683 120.06 2
Max Read: 891.52 MiB/sec (934.83 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 891.52 857.67 879.09 15.21 120.11342 0 16 4 3 0 0 1 0 0 1 40265318400 1048576 108108185600 MPIIO 0
Finished: Tue Oct 23 14:48:22 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 8 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12381846 0xbcee96 0
6 12595101 0xc02f9d 0
0 12423492 0xbd9144 0
2 12569834 0xbfccea 0
4 12511574 0xbee956 0
7 12460203 0xbe20ab 0
1 12641479 0xc0e4c7 0
3 12489780 0xbe9434 0

110
output/COUNT:1#NN:4#PPN:4#API:MPIIO#T:10485760.txt Normal file
View File

@ -0,0 +1,110 @@
+ tee -a ./output/COUNT:1#NN:4#PPN:4#API:MPIIO#T:10485760.txt
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 40265318400 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Tue Oct 23 12:30:38 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 40265318400 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 12:30:38 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 16 (4 per node)
repetitions = 3
xfersize = 10 MiB
blocksize = 37.50 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644213637120.
WARNING: Using actual aggregate bytes moved = 247222763520.
write 1957.09 39321600 10240 0.025481 120.44 0.000464 120.47 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644224122880.
WARNING: Using actual aggregate bytes moved = 248617369600.
write 1901.67 39321600 10240 4.36 120.32 0.000425 124.68 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644224122880.
WARNING: Using actual aggregate bytes moved = 251878440960.
write 1916.60 39321600 10240 4.96 120.37 0.000825 125.33 2
Max Write: 1957.09 MiB/sec (2052.16 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 1957.09 1901.67 1925.12 23.41 123.49353 0 16 4 3 0 0 1 0 0 1 40265318400 10485760 247222763520 MPIIO 0
Finished: Tue Oct 23 12:37:00 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ tee -a ./output/COUNT:1#NN:4#PPN:4#API:MPIIO#T:10485760.txt
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 40265318400 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Tue Oct 23 12:37:12 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 40265318400 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 12:37:12 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 16 (4 per node)
repetitions = 3
xfersize = 10 MiB
blocksize = 37.50 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 278302556160.
read 2206.14 39321600 10240 0.195211 120.11 0.000531 120.31 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 282570260480.
read 2244.18 39321600 10240 0.001118 120.08 0.000605 120.08 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 284719841280.
read 2261.20 39321600 10240 0.001117 120.08 0.000633 120.08 2
Max Read: 2261.20 MiB/sec (2371.04 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 2261.20 2206.14 2237.17 23.02 120.15556 0 16 4 3 0 0 1 0 0 1 40265318400 10485760 278302556160 MPIIO 0
Finished: Tue Oct 23 12:43:12 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 8 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12489515 0xbe932b 0
5 12381578 0xbced8a 0
6 12594835 0xc02e93 0
0 12423227 0xbd903b 0
2 12569569 0xbfcbe1 0
4 12511304 0xbee848 0
7 12459936 0xbe1fa0 0
1 12641214 0xc0e3be 0

110
output/COUNT:1#NN:4#PPN:4#API:MPIIO#T:16384.txt Normal file
View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 40265318400 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:4#PPN:4#API:MPIIO#T:16384.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Tue Oct 23 18:51:06 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 40265318400 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 18:51:06 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 16 (4 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 37.50 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644241801216.
WARNING: Using actual aggregate bytes moved = 5837783040.
write 45.53 39321600 16.00 0.010779 122.26 0.000374 122.27 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644239130624.
WARNING: Using actual aggregate bytes moved = 5792432128.
write 45.20 39321600 16.00 0.001457 122.22 0.000474 122.23 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644236886016.
WARNING: Using actual aggregate bytes moved = 5819957248.
write 45.37 39321600 16.00 0.001491 122.32 0.000553 122.32 2
Max Write: 45.53 MiB/sec (47.74 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 45.53 45.20 45.37 0.14 122.27394 0 16 4 3 0 0 1 0 0 1 40265318400 16384 5837783040 MPIIO 0
Finished: Tue Oct 23 18:57:14 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 40265318400 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
+ tee -a ./output/COUNT:1#NN:4#PPN:4#API:MPIIO#T:16384.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Tue Oct 23 18:57:21 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 40265318400 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 18:57:21 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 16 (4 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 37.50 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 4233822208.
read 33.04 39321600 16.00 0.017074 122.20 0.010243 122.21 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 4250664960.
read 33.18 39321600 16.00 0.005700 122.19 0.005170 122.19 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 4272160768.
read 33.34 39321600 16.00 0.005748 122.19 0.005150 122.19 2
Max Read: 33.34 MiB/sec (34.96 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 33.34 33.04 33.19 0.12 122.19844 0 16 4 3 0 0 1 0 0 1 40265318400 16384 4233822208 MPIIO 0
Finished: Tue Oct 23 19:03:28 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 8 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12570366 0xbfcefe 0
4 12512105 0xbeeb69 0
7 12460736 0xbe22c0 0
1 12642013 0xc0e6dd 0
3 12490315 0xbe964b 0
5 12382380 0xbcf0ac 0
6 12595634 0xc031b2 0
0 12424024 0xbd9358 0

348
output/COUNT:1#NN:4#PPN:4#API:POSIX#T:102400.txt Normal file
View File

@ -0,0 +1,348 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 40265318400 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:4#PPN:4#API:POSIX#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Tue Oct 23 16:29:10 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 40265318400 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 16:29:10 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 16 (4 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 37.50 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644238540800.
WARNING: Using actual aggregate bytes moved = 125125939200.
write 874.64 39321600 100.00 0.001231 136.43 0.000377 136.43 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644238540800.
WARNING: Using actual aggregate bytes moved = 113909555200.
write 789.23 39321600 100.00 0.001982 137.64 0.000406 137.64 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644238540800.
WARNING: Using actual aggregate bytes moved = 120449331200.
write 850.95 39321600 100.00 0.005781 134.98 0.000431 134.99 2
Max Write: 874.64 MiB/sec (917.13 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 874.64 789.23 838.27 36.00 136.35567 0 16 4 3 1 0 1 0 0 1 40265318400 102400 125125939200 POSIX 0
Finished: Tue Oct 23 16:36:02 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 40265318400 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread4/file -r
+ tee -a ./output/COUNT:1#NN:4#PPN:4#API:POSIX#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Tue Oct 23 16:36:12 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 40265318400 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread4/file -r
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 16:36:12 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/indread4/file
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 16 (4 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 37.50 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 2576980377600.
WARNING: Using actual aggregate bytes moved = 27953971200.
read 222.10 39321600 100.00 0.001319 120.03 0.000710 120.03 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 2576980377600.
WARNING: Using actual aggregate bytes moved = 28784230400.
read 228.71 39321600 100.00 0.000707 120.02 0.000645 120.02 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 2576980377600.
WARNING: Using actual aggregate bytes moved = 29994086400.
read 238.31 39321600 100.00 0.000874 120.03 0.000615 120.03 2
Max Read: 238.31 MiB/sec (249.89 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 238.31 222.10 229.71 6.65 120.02784 0 16 4 3 1 0 1 0 0 1 40265318400 102400 27953971200 POSIX 0
Finished: Tue Oct 23 16:42:12 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 8 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write.00000006
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12641738 0xc0e5ca 0
3 12490041 0xbe9539 0
5 12382105 0xbcef99 0
6 12595359 0xc0309f 0
0 12423749 0xbd9245 0
2 12570094 0xbfcdee 0
4 12511833 0xbeea59 0
7 12460461 0xbe21ad 0
/esfs/jtacquaviva/ioperf/file_write.00000002
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12423753 0xbd9249 0
2 12570098 0xbfcdf2 0
4 12511837 0xbeea5d 0
7 12460465 0xbe21b1 0
1 12641742 0xc0e5ce 0
3 12490045 0xbe953d 0
5 12382109 0xbcef9d 0
6 12595363 0xc030a3 0
/esfs/jtacquaviva/ioperf/file_write.00000000
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12382107 0xbcef9b 0
6 12595361 0xc030a1 0
0 12423751 0xbd9247 0
2 12570096 0xbfcdf0 0
4 12511835 0xbeea5b 0
7 12460463 0xbe21af 0
1 12641740 0xc0e5cc 0
3 12490043 0xbe953b 0
/esfs/jtacquaviva/ioperf/file_write.00000004
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12570088 0xbfcde8 0
1 12641731 0xc0e5c3 0
5 12382100 0xbcef94 0
3 12490036 0xbe9534 0
6 12595354 0xc0309a 0
0 12423744 0xbd9240 0
4 12511828 0xbeea54 0
7 12460456 0xbe21a8 0
/esfs/jtacquaviva/ioperf/file_write.00000007
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12490031 0xbe952f 0
6 12595349 0xc03095 0
4 12511823 0xbeea4f 0
5 12382096 0xbcef90 0
0 12423740 0xbd923c 0
2 12570085 0xbfcde5 0
7 12460452 0xbe21a4 0
1 12641729 0xc0e5c1 0
/esfs/jtacquaviva/ioperf/file_write.00000012
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12382097 0xbcef91 0
2 12570086 0xbfcde6 0
7 12460453 0xbe21a5 0
1 12641730 0xc0e5c2 0
6 12595351 0xc03097 0
4 12511826 0xbeea52 0
0 12423743 0xbd923f 0
3 12490035 0xbe9533 0
/esfs/jtacquaviva/ioperf/file_write.00000013
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12570099 0xbfcdf3 0
4 12511838 0xbeea5e 0
7 12460466 0xbe21b2 0
1 12641743 0xc0e5cf 0
3 12490046 0xbe953e 0
5 12382110 0xbcef9e 0
6 12595364 0xc030a4 0
0 12423754 0xbd924a 0
/esfs/jtacquaviva/ioperf/file_write.00000003
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12460458 0xbe21aa 0
1 12641736 0xc0e5c8 0
3 12490039 0xbe9537 0
5 12382103 0xbcef97 0
6 12595358 0xc0309e 0
0 12423747 0xbd9243 0
2 12570092 0xbfcdec 0
4 12511832 0xbeea58 0
/esfs/jtacquaviva/ioperf/file_write.00000008
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12490042 0xbe953a 0
5 12382106 0xbcef9a 0
6 12595360 0xc030a0 0
0 12423750 0xbd9246 0
2 12570095 0xbfcdef 0
4 12511834 0xbeea5a 0
7 12460462 0xbe21ae 0
1 12641739 0xc0e5cb 0
/esfs/jtacquaviva/ioperf/file_write.00000010
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12460451 0xbe21a3 0
5 12382095 0xbcef8f 0
0 12423739 0xbd923b 0
2 12570084 0xbfcde4 0
1 12641728 0xc0e5c0 0
6 12595350 0xc03096 0
3 12490032 0xbe9530 0
4 12511824 0xbeea50 0
/esfs/jtacquaviva/ioperf/file_write.00000005
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12490037 0xbe9535 0
5 12382101 0xbcef95 0
2 12570090 0xbfcdea 0
4 12511829 0xbeea55 0
7 12460457 0xbe21a9 0
6 12595355 0xc0309b 0
0 12423746 0xbd9242 0
1 12641735 0xc0e5c7 0
/esfs/jtacquaviva/ioperf/file_write.00000014
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12511830 0xbeea56 0
5 12382102 0xbcef96 0
7 12460460 0xbe21ac 0
6 12595357 0xc0309d 0
1 12641737 0xc0e5c9 0
3 12490040 0xbe9538 0
0 12423748 0xbd9244 0
2 12570093 0xbfcded 0
/esfs/jtacquaviva/ioperf/file_write.00000009
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12423745 0xbd9241 0
1 12641734 0xc0e5c6 0
6 12595356 0xc0309c 0
7 12460459 0xbe21ab 0
3 12490038 0xbe9536 0
2 12570091 0xbfcdeb 0
5 12382104 0xbcef98 0
4 12511831 0xbeea57 0
/esfs/jtacquaviva/ioperf/file_write.00000015
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12595362 0xc030a2 0
0 12423752 0xbd9248 0
2 12570097 0xbfcdf1 0
4 12511836 0xbeea5c 0
7 12460464 0xbe21b0 0
1 12641741 0xc0e5cd 0
3 12490044 0xbe953c 0
5 12382108 0xbcef9c 0
/esfs/jtacquaviva/ioperf/file_write.00000001
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12423741 0xbd923d 0
4 12511825 0xbeea51 0
3 12490033 0xbe9531 0
2 12570087 0xbfcde7 0
7 12460454 0xbe21a6 0
5 12382099 0xbcef93 0
6 12595353 0xc03099 0
1 12641732 0xc0e5c4 0
/esfs/jtacquaviva/ioperf/file_write.00000011
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12382098 0xbcef92 0
0 12423742 0xbd923e 0
3 12490034 0xbe9532 0
6 12595352 0xc03098 0
4 12511827 0xbeea53 0
7 12460455 0xbe21a7 0
2 12570089 0xbfcde9 0
1 12641733 0xc0e5c5 0

345
output/COUNT:1#NN:4#PPN:4#API:POSIX#T:1048576.txt Normal file
View File

@ -0,0 +1,345 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 40265318400 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:4#PPN:4#API:POSIX#T:1048576.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Tue Oct 23 14:23:17 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 40265318400 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 14:23:17 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 16 (4 per node)
repetitions = 3
xfersize = 1 MiB
blocksize = 37.50 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644245094400.
WARNING: Using actual aggregate bytes moved = 606304468992.
write 4626 39321600 1024.00 0.001079 124.99 0.000297 124.99 0
write 4999 39321600 1024.00 0.001644 122.89 0.000423 122.89 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644245094400.
WARNING: Using actual aggregate bytes moved = 643769040896.
write 4990 39321600 1024.00 0.001563 123.02 0.000498 123.02 2
Max Write: 4999.44 MiB/sec (5242.29 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 4999.44 4626.05 4871.99 173.95 123.63617 0 16 4 3 1 0 1 0 0 1 40265318400 1048576 606304468992 POSIX 0
Finished: Tue Oct 23 14:29:32 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 40265318400 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread4/file -r
+ tee -a ./output/COUNT:1#NN:4#PPN:4#API:POSIX#T:1048576.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Tue Oct 23 14:29:44 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 40265318400 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread4/file -r
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 14:29:44 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/indread4/file
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 16 (4 per node)
repetitions = 3
xfersize = 1 MiB
blocksize = 37.50 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 2576980377600.
WARNING: Using actual aggregate bytes moved = 181194981376.
read 1439.78 39321600 1024.00 0.001215 120.02 0.000552 120.02 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 2576980377600.
WARNING: Using actual aggregate bytes moved = 220225077248.
read 1749.91 39321600 1024.00 0.000579 120.02 0.000494 120.02 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 2576980377600.
WARNING: Using actual aggregate bytes moved = 227791601664.
read 1810.02 39321600 1024.00 0.000698 120.02 0.000670 120.02 2
Max Read: 1810.02 MiB/sec (1897.95 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 1810.02 1439.78 1666.57 162.23 120.01958 0 16 4 3 1 0 1 0 0 1 40265318400 1048576 181194981376 POSIX 0
Finished: Tue Oct 23 14:35:44 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 8 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write.00000006
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12460195 0xbe20a3 0
1 12641471 0xc0e4bf 0
3 12489772 0xbe942c 0
5 12381838 0xbcee8e 0
6 12595093 0xc02f95 0
0 12423484 0xbd913c 0
2 12569826 0xbfcce2 0
4 12511565 0xbee94d 0
/esfs/jtacquaviva/ioperf/file_write.00000002
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12595083 0xc02f8b 0
2 12569816 0xbfccd8 0
0 12423474 0xbd9132 0
4 12511555 0xbee943 0
1 12641461 0xc0e4b5 0
5 12381828 0xbcee84 0
7 12460186 0xbe209a 0
3 12489765 0xbe9425 0
/esfs/jtacquaviva/ioperf/file_write.00000000
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12489766 0xbe9426 0
5 12381832 0xbcee88 0
6 12595087 0xc02f8f 0
0 12423478 0xbd9136 0
2 12569820 0xbfccdc 0
4 12511559 0xbee947 0
7 12460189 0xbe209d 0
1 12641465 0xc0e4b9 0
/esfs/jtacquaviva/ioperf/file_write.00000004
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12569829 0xbfcce5 0
4 12511568 0xbee950 0
7 12460198 0xbe20a6 0
1 12641475 0xc0e4c3 0
3 12489776 0xbe9430 0
5 12381842 0xbcee92 0
6 12595097 0xc02f99 0
0 12423488 0xbd9140 0
/esfs/jtacquaviva/ioperf/file_write.00000007
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12595089 0xc02f91 0
0 12423480 0xbd9138 0
2 12569822 0xbfccde 0
4 12511561 0xbee949 0
7 12460191 0xbe209f 0
1 12641467 0xc0e4bb 0
3 12489768 0xbe9428 0
5 12381834 0xbcee8a 0
/esfs/jtacquaviva/ioperf/file_write.00000012
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12511570 0xbee952 0
7 12460200 0xbe20a8 0
1 12641476 0xc0e4c4 0
3 12489777 0xbe9431 0
5 12381843 0xbcee93 0
6 12595098 0xc02f9a 0
0 12423489 0xbd9141 0
2 12569831 0xbfcce7 0
/esfs/jtacquaviva/ioperf/file_write.00000013
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12489762 0xbe9422 0
0 12423475 0xbd9133 0
5 12381829 0xbcee85 0
2 12569819 0xbfccdb 0
7 12460187 0xbe209b 0
4 12511557 0xbee945 0
1 12641464 0xc0e4b8 0
6 12595086 0xc02f8e 0
/esfs/jtacquaviva/ioperf/file_write.00000003
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12569824 0xbfcce0 0
4 12511563 0xbee94b 0
7 12460193 0xbe20a1 0
1 12641469 0xc0e4bd 0
3 12489770 0xbe942a 0
5 12381836 0xbcee8c 0
6 12595091 0xc02f93 0
0 12423482 0xbd913a 0
/esfs/jtacquaviva/ioperf/file_write.00000008
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12489774 0xbe942e 0
5 12381840 0xbcee90 0
2 12569828 0xbfcce4 0
4 12511567 0xbee94f 0
1 12641473 0xc0e4c1 0
6 12595095 0xc02f97 0
7 12460199 0xbe20a7 0
0 12423487 0xbd913f 0
/esfs/jtacquaviva/ioperf/file_write.00000010
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12511564 0xbee94c 0
7 12460194 0xbe20a2 0
1 12641470 0xc0e4be 0
3 12489771 0xbe942b 0
5 12381837 0xbcee8d 0
6 12595092 0xc02f94 0
0 12423483 0xbd913b 0
2 12569825 0xbfcce1 0
/esfs/jtacquaviva/ioperf/file_write.00000005
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12460185 0xbe2099 0
3 12489764 0xbe9424 0
2 12569818 0xbfccda 0
1 12641462 0xc0e4b6 0
6 12595085 0xc02f8d 0
0 12423476 0xbd9134 0
5 12381831 0xbcee87 0
4 12511558 0xbee946 0
/esfs/jtacquaviva/ioperf/file_write.00000014
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12641472 0xc0e4c0 0
3 12489773 0xbe942d 0
5 12381839 0xbcee8f 0
6 12595094 0xc02f96 0
0 12423485 0xbd913d 0
2 12569827 0xbfcce3 0
4 12511566 0xbee94e 0
7 12460196 0xbe20a4 0
/esfs/jtacquaviva/ioperf/file_write.00000009
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12381833 0xbcee89 0
6 12595088 0xc02f90 0
0 12423479 0xbd9137 0
2 12569821 0xbfccdd 0
4 12511560 0xbee948 0
7 12460190 0xbe209e 0
1 12641466 0xc0e4ba 0
3 12489767 0xbe9427 0
/esfs/jtacquaviva/ioperf/file_write.00000015
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12423481 0xbd9139 0
2 12569823 0xbfccdf 0
4 12511562 0xbee94a 0
7 12460192 0xbe20a0 0
1 12641468 0xc0e4bc 0
3 12489769 0xbe9429 0
5 12381835 0xbcee8b 0
6 12595090 0xc02f92 0
/esfs/jtacquaviva/ioperf/file_write.00000001
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12423486 0xbd913e 0
7 12460197 0xbe20a5 0
3 12489775 0xbe942f 0
5 12381841 0xbcee91 0
2 12569830 0xbfcce6 0
1 12641474 0xc0e4c2 0
4 12511569 0xbee951 0
6 12595096 0xc02f98 0
/esfs/jtacquaviva/ioperf/file_write.00000011
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12489763 0xbe9423 0
6 12595084 0xc02f8c 0
2 12569817 0xbfccd9 0
4 12511556 0xbee944 0
5 12381830 0xbcee86 0
1 12641463 0xc0e4b7 0
0 12423477 0xbd9135 0
7 12460188 0xbe209c 0

338
output/COUNT:1#NN:4#PPN:4#API:POSIX#T:10485760.txt Normal file
View File

@ -0,0 +1,338 @@
+ tee -a ./output/COUNT:1#NN:4#PPN:4#API:POSIX#T:10485760.txt
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 40265318400 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Tue Oct 23 12:19:04 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 40265318400 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 12:19:04 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 16 (4 per node)
repetitions = 3
xfersize = 10 MiB
blocksize = 37.50 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
write 5780 39321600 10240 0.001069 106.30 0.000287 106.30 0
write 5871 39321600 10240 0.001675 104.65 0.000314 104.65 1
write 5804 39321600 10240 0.004186 105.86 0.000410 105.87 2
Max Write: 5870.76 MiB/sec (6155.94 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 5870.76 5779.72 5818.02 38.54 105.60748 0 16 4 3 1 0 1 0 0 1 40265318400 10485760 644245094400 POSIX 0
Finished: Tue Oct 23 12:24:25 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 40265318400 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread4/file -r
+ tee -a ./output/COUNT:1#NN:4#PPN:4#API:POSIX#T:10485760.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Tue Oct 23 12:24:36 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 40265318400 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread4/file -r
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 12:24:36 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/indread4/file
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 16 (4 per node)
repetitions = 3
xfersize = 10 MiB
blocksize = 37.50 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 2576980377600.
WARNING: Using actual aggregate bytes moved = 196796743680.
read 1562.08 39321600 10240 0.001144 120.15 0.000530 120.15 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 2576980377600.
WARNING: Using actual aggregate bytes moved = 243112345600.
read 1930.22 39321600 10240 0.000539 120.12 0.000569 120.12 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 2576980377600.
WARNING: Using actual aggregate bytes moved = 244790067200.
read 1942.28 39321600 10240 0.000613 120.19 0.000658 120.19 2
Max Read: 1942.28 MiB/sec (2036.63 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 1942.28 1562.08 1811.53 176.46 120.15249 0 16 4 3 1 0 1 0 0 1 40265318400 10485760 196796743680 POSIX 0
Finished: Tue Oct 23 12:30:36 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 8 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write.00000006
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12641196 0xc0e3ac 0
3 12489497 0xbe9319 0
5 12381560 0xbced78 0
6 12594817 0xc02e81 0
0 12423209 0xbd9029 0
2 12569550 0xbfcbce 0
4 12511287 0xbee837 0
7 12459918 0xbe1f8e 0
/esfs/jtacquaviva/ioperf/file_write.00000002
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12423223 0xbd9037 0
2 12569564 0xbfcbdc 0
4 12511300 0xbee844 0
7 12459932 0xbe1f9c 0
1 12641210 0xc0e3ba 0
3 12489511 0xbe9327 0
5 12381574 0xbced86 0
6 12594831 0xc02e8f 0
/esfs/jtacquaviva/ioperf/file_write.00000000
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12569565 0xbfcbdd 0
4 12511301 0xbee845 0
7 12459933 0xbe1f9d 0
1 12641211 0xc0e3bb 0
3 12489512 0xbe9328 0
5 12381575 0xbced87 0
6 12594832 0xc02e90 0
0 12423224 0xbd9038 0
/esfs/jtacquaviva/ioperf/file_write.00000004
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12641205 0xc0e3b5 0
3 12489506 0xbe9322 0
5 12381569 0xbced81 0
6 12594826 0xc02e8a 0
0 12423218 0xbd9032 0
2 12569559 0xbfcbd7 0
4 12511295 0xbee83f 0
7 12459927 0xbe1f97 0
/esfs/jtacquaviva/ioperf/file_write.00000007
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12511292 0xbee83c 0
6 12594822 0xc02e86 0
7 12459924 0xbe1f94 0
3 12489503 0xbe931f 0
2 12569555 0xbfcbd3 0
1 12641202 0xc0e3b2 0
5 12381566 0xbced7e 0
0 12423215 0xbd902f 0
/esfs/jtacquaviva/ioperf/file_write.00000012
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12489502 0xbe931e 0
7 12459923 0xbe1f93 0
5 12381565 0xbced7d 0
6 12594823 0xc02e87 0
1 12641201 0xc0e3b1 0
0 12423214 0xbd902e 0
2 12569556 0xbfcbd4 0
/esfs/jtacquaviva/ioperf/file_write.00000013
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12459926 0xbe1f96 0
1 12641204 0xc0e3b4 0
3 12489505 0xbe9321 0
5 12381568 0xbced80 0
6 12594825 0xc02e89 0
0 12423217 0xbd9031 0
2 12569558 0xbfcbd6 0
4 12511294 0xbee83e 0
/esfs/jtacquaviva/ioperf/file_write.00000003
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12594830 0xc02e8e 0
0 12423222 0xbd9036 0
2 12569563 0xbfcbdb 0
4 12511299 0xbee843 0
7 12459931 0xbe1f9b 0
1 12641209 0xc0e3b9 0
3 12489510 0xbe9326 0
5 12381573 0xbced85 0
/esfs/jtacquaviva/ioperf/file_write.00000008
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12489507 0xbe9323 0
5 12381570 0xbced82 0
6 12594827 0xc02e8b 0
0 12423219 0xbd9033 0
2 12569560 0xbfcbd8 0
4 12511296 0xbee840 0
7 12459928 0xbe1f98 0
1 12641206 0xc0e3b6 0
/esfs/jtacquaviva/ioperf/file_write.00000010
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12511293 0xbee83d 0
7 12459925 0xbe1f95 0
1 12641203 0xc0e3b3 0
3 12489504 0xbe9320 0
5 12381567 0xbced7f 0
6 12594824 0xc02e88 0
0 12423216 0xbd9030 0
2 12569557 0xbfcbd5 0
/esfs/jtacquaviva/ioperf/file_write.00000005
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12381572 0xbced84 0
6 12594829 0xc02e8d 0
0 12423221 0xbd9035 0
2 12569562 0xbfcbda 0
4 12511298 0xbee842 0
7 12459930 0xbe1f9a 0
1 12641208 0xc0e3b8 0
3 12489509 0xbe9325 0
/esfs/jtacquaviva/ioperf/file_write.00000014
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12594818 0xc02e82 0
7 12459919 0xbe1f8f 0
1 12641197 0xc0e3ad 0
0 12423210 0xbd902a 0
3 12489499 0xbe931b 0
2 12569552 0xbfcbd0 0
5 12381563 0xbced7b 0
4 12511290 0xbee83a 0
/esfs/jtacquaviva/ioperf/file_write.00000009
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12511291 0xbee83b 0
7 12459922 0xbe1f92 0
5 12381564 0xbced7c 0
6 12594821 0xc02e85 0
0 12423213 0xbd902d 0
1 12641200 0xc0e3b0 0
3 12489501 0xbe931d 0
2 12569554 0xbfcbd2 0
/esfs/jtacquaviva/ioperf/file_write.00000015
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12381571 0xbced83 0
6 12594828 0xc02e8c 0
0 12423220 0xbd9034 0
2 12569561 0xbfcbd9 0
4 12511297 0xbee841 0
7 12459929 0xbe1f99 0
1 12641207 0xc0e3b7 0
3 12489508 0xbe9324 0
/esfs/jtacquaviva/ioperf/file_write.00000001
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12489498 0xbe931a 0
4 12511288 0xbee838 0
5 12381561 0xbced79 0
7 12459920 0xbe1f90 0
6 12594819 0xc02e83 0
1 12641198 0xc0e3ae 0
0 12423211 0xbd902b 0
2 12569553 0xbfcbd1 0
/esfs/jtacquaviva/ioperf/file_write.00000011
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12569551 0xbfcbcf 0
4 12511289 0xbee839 0
5 12381562 0xbced7a 0
7 12459921 0xbe1f91 0
3 12489500 0xbe931c 0
1 12641199 0xc0e3af 0
6 12594820 0xc02e84 0
0 12423212 0xbd902c 0

347
output/COUNT:1#NN:4#PPN:4#API:POSIX#T:16384.txt Normal file
View File

@ -0,0 +1,347 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 40265318400 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:4#PPN:4#API:POSIX#T:16384.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Tue Oct 23 18:37:25 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 40265318400 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 18:37:25 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 16 (4 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 37.50 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644235657216.
WARNING: Using actual aggregate bytes moved = 39476084736.
write 259.23 39321600 16.00 0.001206 145.23 0.000377 145.23 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644238819328.
WARNING: Using actual aggregate bytes moved = 35021586432.
write 225.93 39321600 16.00 0.023860 147.81 0.000416 147.83 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644238917632.
WARNING: Using actual aggregate bytes moved = 31526256640.
write 192.02 39321600 16.00 0.018627 156.55 0.000476 156.57 2
Max Write: 259.23 MiB/sec (271.82 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 259.23 192.02 225.73 27.44 149.87821 0 16 4 3 1 0 1 0 0 1 40265318400 16384 39476084736 POSIX 0
Finished: Tue Oct 23 18:44:56 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 4 -np 16 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 40265318400 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread4/file -r
+ tee -a ./output/COUNT:1#NN:4#PPN:4#API:POSIX#T:16384.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Tue Oct 23 18:45:04 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 40265318400 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread4/file -r
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 18:45:04 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/indread4/file
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 16 (4 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 37.50 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 2576980377600.
WARNING: Using actual aggregate bytes moved = 5504909312.
read 43.69 39321600 16.00 0.003983 120.15 0.003403 120.15 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 2576980377600.
WARNING: Using actual aggregate bytes moved = 5849333760.
read 46.43 39321600 16.00 0.003422 120.15 0.003406 120.15 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 2576980377600.
WARNING: Using actual aggregate bytes moved = 5870469120.
read 46.60 39321600 16.00 0.003577 120.15 0.003422 120.15 2
Max Read: 46.60 MiB/sec (48.86 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 46.60 43.69 45.57 1.33 120.15097 0 16 4 3 1 0 1 0 0 1 40265318400 16384 5504909312 POSIX 0
Finished: Tue Oct 23 18:51:05 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 8 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write.00000006
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12512094 0xbeeb5e 0
7 12460725 0xbe22b5 0
1 12642002 0xc0e6d2 0
3 12490303 0xbe963f 0
5 12382369 0xbcf0a1 0
6 12595623 0xc031a7 0
0 12424013 0xbd934d 0
2 12570355 0xbfcef3 0
/esfs/jtacquaviva/ioperf/file_write.00000002
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12642009 0xc0e6d9 0
3 12490310 0xbe9646 0
5 12382376 0xbcf0a8 0
6 12595630 0xc031ae 0
0 12424020 0xbd9354 0
2 12570362 0xbfcefa 0
4 12512101 0xbeeb65 0
7 12460732 0xbe22bc 0
/esfs/jtacquaviva/ioperf/file_write.00000000
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12490311 0xbe9647 0
5 12382377 0xbcf0a9 0
6 12595631 0xc031af 0
0 12424021 0xbd9355 0
2 12570363 0xbfcefb 0
4 12512102 0xbeeb66 0
7 12460733 0xbe22bd 0
1 12642010 0xc0e6da 0
/esfs/jtacquaviva/ioperf/file_write.00000004
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12570359 0xbfcef7 0
4 12512098 0xbeeb62 0
7 12460729 0xbe22b9 0
1 12642006 0xc0e6d6 0
3 12490307 0xbe9643 0
5 12382373 0xbcf0a5 0
6 12595627 0xc031ab 0
0 12424017 0xbd9351 0
/esfs/jtacquaviva/ioperf/file_write.00000007
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12382362 0xbcf09a 0
2 12570348 0xbfceec 0
7 12460718 0xbe22ae 0
1 12641995 0xc0e6cb 0
6 12595616 0xc031a0 0
4 12512088 0xbeeb58 0
0 12424007 0xbd9347 0
3 12490298 0xbe963a 0
/esfs/jtacquaviva/ioperf/file_write.00000012
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12460726 0xbe22b6 0
1 12642003 0xc0e6d3 0
3 12490304 0xbe9640 0
0 12424014 0xbd934e 0
2 12570356 0xbfcef4 0
4 12512095 0xbeeb5f 0
5 12382370 0xbcf0a2 0
6 12595625 0xc031a9 0
/esfs/jtacquaviva/ioperf/file_write.00000013
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12460731 0xbe22bb 0
1 12642008 0xc0e6d8 0
3 12490309 0xbe9645 0
5 12382375 0xbcf0a7 0
6 12595629 0xc031ad 0
0 12424019 0xbd9353 0
2 12570361 0xbfcef9 0
4 12512100 0xbeeb64 0
/esfs/jtacquaviva/ioperf/file_write.00000003
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12424016 0xbd9350 0
2 12570358 0xbfcef6 0
4 12512097 0xbeeb61 0
7 12460728 0xbe22b8 0
1 12642005 0xc0e6d5 0
3 12490306 0xbe9642 0
5 12382372 0xbcf0a4 0
6 12595626 0xc031aa 0
/esfs/jtacquaviva/ioperf/file_write.00000008
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12424006 0xbd9346 0
4 12512087 0xbeeb57 0
3 12490297 0xbe9639 0
2 12570349 0xbfceed 0
7 12460719 0xbe22af 0
1 12641996 0xc0e6cc 0
5 12382363 0xbcf09b 0
6 12595617 0xc031a1 0
/esfs/jtacquaviva/ioperf/file_write.00000010
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12595624 0xc031a8 0
7 12460727 0xbe22b7 0
0 12424015 0xbd934f 0
1 12642004 0xc0e6d4 0
2 12570357 0xbfcef5 0
4 12512096 0xbeeb60 0
3 12490305 0xbe9641 0
5 12382371 0xbcf0a3 0
/esfs/jtacquaviva/ioperf/file_write.00000005
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12382365 0xbcf09d 0
4 12512091 0xbeeb5b 0
0 12424010 0xbd934a 0
6 12595620 0xc031a4 0
2 12570353 0xbfcef1 0
1 12642000 0xc0e6d0 0
7 12460723 0xbe22b3 0
/esfs/jtacquaviva/ioperf/file_write.00000014
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12570350 0xbfceee 0
4 12512089 0xbeeb59 0
7 12460720 0xbe22b0 0
1 12641997 0xc0e6cd 0
3 12490299 0xbe963b 0
5 12382364 0xbcf09c 0
6 12595618 0xc031a2 0
0 12424008 0xbd9348 0
/esfs/jtacquaviva/ioperf/file_write.00000009
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12641999 0xc0e6cf 0
4 12512092 0xbeeb5c 0
5 12382368 0xbcf0a0 0
3 12490301 0xbe963d 0
0 12424011 0xbd934b 0
2 12570354 0xbfcef2 0
7 12460724 0xbe22b4 0
6 12595622 0xc031a6 0
/esfs/jtacquaviva/ioperf/file_write.00000015
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12512090 0xbeeb5a 0
7 12460721 0xbe22b1 0
1 12641998 0xc0e6ce 0
6 12595619 0xc031a3 0
0 12424009 0xbd9349 0
2 12570351 0xbfceef 0
3 12490300 0xbe963c 0
5 12382367 0xbcf09f 0
/esfs/jtacquaviva/ioperf/file_write.00000001
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12382366 0xbcf09e 0
2 12570352 0xbfcef0 0
7 12460722 0xbe22b2 0
3 12490302 0xbe963e 0
1 12642001 0xc0e6d1 0
6 12595621 0xc031a5 0
4 12512093 0xbeeb5d 0
0 12424012 0xbd934c 0
/esfs/jtacquaviva/ioperf/file_write.00000011
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12512099 0xbeeb63 0
7 12460730 0xbe22ba 0
1 12642007 0xc0e6d7 0
3 12490308 0xbe9644 0
5 12382374 0xbcf0a6 0
6 12595628 0xc031ac 0
0 12424018 0xbd9352 0
2 12570360 0xbfcef8 0

110
output/COUNT:1#NN:4#PPN:6#API:MPIIO#T:102400.txt Normal file
View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 24 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 26843545600 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:4#PPN:6#API:MPIIO#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Tue Oct 23 16:16:50 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 26843545600 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 16:16:50 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 24 (6 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 25 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644236390400.
WARNING: Using actual aggregate bytes moved = 36909977600.
write 292.31 26214400 100.00 0.015111 120.41 0.001314 120.42 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644238848000.
WARNING: Using actual aggregate bytes moved = 37690060800.
write 298.57 26214400 100.00 0.005295 120.38 0.001247 120.39 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644242432000.
WARNING: Using actual aggregate bytes moved = 36800819200.
write 291.29 26214400 100.00 0.005400 120.48 0.001350 120.48 2
Max Write: 298.57 MiB/sec (313.08 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 298.57 291.29 294.06 3.22 120.42991 0 24 6 3 0 0 1 0 0 1 26843545600 102400 36909977600 MPIIO 0
Finished: Tue Oct 23 16:22:53 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 24 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 24 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 26843545600 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
+ tee -a ./output/COUNT:1#NN:4#PPN:6#API:MPIIO#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Tue Oct 23 16:23:07 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 26843545600 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 16:23:07 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 24 (6 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 25 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 28863385600.
read 228.64 26214400 100.00 0.027721 120.36 0.008785 120.39 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 30064128000.
read 238.21 26214400 100.00 0.010009 120.36 0.008875 120.36 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 30530867200.
read 241.92 26214400 100.00 0.009805 120.36 0.008783 120.36 2
Max Read: 241.92 MiB/sec (253.67 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 241.92 228.64 236.26 5.59 120.36884 0 24 6 3 0 0 1 0 0 1 26843545600 102400 28863385600 MPIIO 0
Finished: Tue Oct 23 16:29:08 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 8 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12511789 0xbeea2d 0
7 12460419 0xbe2183 0
1 12641695 0xc0e59f 0
3 12489998 0xbe950e 0
5 12382062 0xbcef6e 0
6 12595316 0xc03074 0
0 12423707 0xbd921b 0
2 12570052 0xbfcdc4 0

110
output/COUNT:1#NN:4#PPN:6#API:MPIIO#T:1048576.txt Normal file
View File

@ -0,0 +1,110 @@
+ tee -a ./output/COUNT:1#NN:4#PPN:6#API:MPIIO#T:1048576.txt
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 24 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 26843545600 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Tue Oct 23 14:10:37 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 26843545600 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 14:10:37 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 24 (6 per node)
repetitions = 3
xfersize = 1 MiB
blocksize = 25 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644242997248.
WARNING: Using actual aggregate bytes moved = 451727589376.
write 3587.43 26214400 1024.00 0.018934 120.07 0.002534 120.09 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644245094400.
WARNING: Using actual aggregate bytes moved = 469256634368.
write 3559.74 26214400 1024.00 5.66 120.06 0.001558 125.72 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644245094400.
WARNING: Using actual aggregate bytes moved = 469776728064.
write 3544.85 26214400 1024.00 6.33 120.05 0.001587 126.38 2
Max Write: 3587.43 MiB/sec (3761.69 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 3587.43 3544.85 3564.01 17.64 124.06245 0 24 6 3 0 0 1 0 0 1 26843545600 1048576 451727589376 MPIIO 0
Finished: Tue Oct 23 14:17:01 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 24 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 24 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 26843545600 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
+ tee -a ./output/COUNT:1#NN:4#PPN:6#API:MPIIO#T:1048576.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Tue Oct 23 14:17:15 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 26843545600 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 14:17:15 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 24 (6 per node)
repetitions = 3
xfersize = 1 MiB
blocksize = 25 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 135660568576.
read 1076.28 26214400 1024.00 0.144026 120.07 0.003836 120.21 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 140058296320.
read 1112.38 26214400 1024.00 0.004709 120.07 0.003639 120.08 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 140638158848.
read 1117.09 26214400 1024.00 0.004612 120.06 0.003690 120.06 2
Max Read: 1117.09 MiB/sec (1171.35 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 1117.09 1076.28 1101.92 18.23 120.11552 0 24 6 3 0 0 1 0 0 1 26843545600 1048576 135660568576 MPIIO 0
Finished: Tue Oct 23 14:23:15 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 8 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12460151 0xbe2077 0
1 12641428 0xc0e494 0
3 12489729 0xbe9401 0
5 12381795 0xbcee63 0
6 12595050 0xc02f6a 0
0 12423441 0xbd9111 0
2 12569783 0xbfccb7 0
4 12511522 0xbee922 0

110
output/COUNT:1#NN:4#PPN:6#API:MPIIO#T:10485760.txt Normal file
View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 24 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 26843545600 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:4#PPN:6#API:MPIIO#T:10485760.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Tue Oct 23 12:06:24 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 26843545600 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 12:06:24 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 24 (6 per node)
repetitions = 3
xfersize = 10 MiB
blocksize = 25 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644213637120.
WARNING: Using actual aggregate bytes moved = 292940677120.
write 2320.29 26214400 10240 0.016737 120.39 0.001899 120.40 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644224122880.
WARNING: Using actual aggregate bytes moved = 298770759680.
write 2273.79 26214400 10240 4.97 120.34 0.001379 125.31 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644224122880.
WARNING: Using actual aggregate bytes moved = 296694579200.
write 2240.13 26214400 10240 6.05 120.26 0.001419 126.31 2
Max Write: 2320.29 MiB/sec (2433.01 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 2320.29 2240.13 2278.07 32.87 124.00765 0 24 6 3 0 0 1 0 0 1 26843545600 10485760 292940677120 MPIIO 0
Finished: Tue Oct 23 12:12:48 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 24 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 24 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 26843545600 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
+ tee -a ./output/COUNT:1#NN:4#PPN:6#API:MPIIO#T:10485760.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Tue Oct 23 12:13:01 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 26843545600 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 12:13:01 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 24 (6 per node)
repetitions = 3
xfersize = 10 MiB
blocksize = 25 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 323275980800.
read 2564.22 26214400 10240 0.148252 120.08 0.001063 120.23 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 328193802240.
read 2606.40 26214400 10240 0.002098 120.08 0.000950 120.09 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 330972528640.
read 2628.57 26214400 10240 0.001938 120.08 0.000912 120.08 2
Max Read: 2628.57 MiB/sec (2756.26 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 2628.57 2564.22 2599.73 26.69 120.13230 0 24 6 3 0 0 1 0 0 1 26843545600 10485760 323275980800 MPIIO 0
Finished: Tue Oct 23 12:19:01 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 8 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12511253 0xbee815 0
7 12459885 0xbe1f6d 0
1 12641163 0xc0e38b 0
3 12489465 0xbe92f9 0
5 12381527 0xbced57 0
6 12594784 0xc02e60 0
0 12423176 0xbd9008 0
2 12569517 0xbfcbad 0

110
output/COUNT:1#NN:4#PPN:6#API:MPIIO#T:16384.txt Normal file
View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 24 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 26843545600 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:4#PPN:6#API:MPIIO#T:16384.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Tue Oct 23 18:25:02 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 26843545600 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 18:25:02 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 24 (6 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 25 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644244193280.
WARNING: Using actual aggregate bytes moved = 6384123904.
write 49.78 26214400 16.00 0.016694 122.29 0.002429 122.31 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644241162240.
WARNING: Using actual aggregate bytes moved = 6403342336.
write 49.95 26214400 16.00 0.003267 122.27 0.001918 122.27 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644242046976.
WARNING: Using actual aggregate bytes moved = 6401703936.
write 49.93 26214400 16.00 0.003189 122.28 0.001912 122.28 2
Max Write: 49.95 MiB/sec (52.37 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 49.95 49.78 49.88 0.07 122.28574 0 24 6 3 0 0 1 0 0 1 26843545600 16384 6384123904 MPIIO 0
Finished: Tue Oct 23 18:31:10 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 24 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 24 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 26843545600 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
+ tee -a ./output/COUNT:1#NN:4#PPN:6#API:MPIIO#T:16384.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Tue Oct 23 18:31:17 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 26843545600 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 18:31:17 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 24 (6 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 25 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 5903941632.
read 46.09 26214400 16.00 0.012321 122.14 0.001101 122.15 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 5968314368.
read 46.60 26214400 16.00 0.002034 122.15 0.001100 122.15 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 5990383616.
read 46.77 26214400 16.00 0.001884 122.14 0.001095 122.14 2
Max Read: 46.77 MiB/sec (49.05 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 46.77 46.09 46.49 0.29 122.14669 0 24 6 3 0 0 1 0 0 1 26843545600 16384 5903941632 MPIIO 0
Finished: Tue Oct 23 18:37:23 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 8 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12382328 0xbcf078 0
6 12595584 0xc03180 0
0 12423973 0xbd9325 0
2 12570317 0xbfcecd 0
4 12512055 0xbeeb37 0
7 12460685 0xbe228d 0
1 12641962 0xc0e6aa 0
3 12490264 0xbe9618 0

476
output/COUNT:1#NN:4#PPN:6#API:POSIX#T:102400.txt Normal file
View File

@ -0,0 +1,476 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 24 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 26843545600 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:4#PPN:6#API:POSIX#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Tue Oct 23 16:03:44 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 26843545600 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 16:03:44 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 24 (6 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 25 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644220518400.
WARNING: Using actual aggregate bytes moved = 119719116800.
write 835.26 26214400 100.00 0.002545 136.69 0.001349 136.69 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644235264000.
WARNING: Using actual aggregate bytes moved = 118915072000.
write 834.13 26214400 100.00 0.003326 135.95 0.001452 135.96 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644208230400.
WARNING: Using actual aggregate bytes moved = 118739763200.
write 831.94 26214400 100.00 0.012565 136.10 0.001568 136.11 2
Max Write: 835.26 MiB/sec (875.83 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 835.26 831.94 833.78 1.38 136.25458 0 24 6 3 1 0 1 0 0 1 26843545600 102400 119719116800 POSIX 0
Finished: Tue Oct 23 16:10:35 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 24 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 24 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 26843545600 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread4/file -r
+ tee -a ./output/COUNT:1#NN:4#PPN:6#API:POSIX#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Tue Oct 23 16:10:47 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 26843545600 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread4/file -r
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 16:10:47 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/indread4/file
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 24 (6 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 25 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 3865470566400.
WARNING: Using actual aggregate bytes moved = 43632332800.
read 346.67 26214400 100.00 0.005849 120.03 0.005038 120.03 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 3865470566400.
WARNING: Using actual aggregate bytes moved = 45587558400.
read 362.22 26214400 100.00 0.005043 120.02 0.004908 120.03 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 3865470566400.
WARNING: Using actual aggregate bytes moved = 48580300800.
read 386.00 26214400 100.00 0.004915 120.03 0.004847 120.03 2
Max Read: 386.00 MiB/sec (404.75 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 386.00 346.67 364.96 16.17 120.02753 0 24 6 3 1 0 1 0 0 1 26843545600 102400 43632332800 POSIX 0
Finished: Tue Oct 23 16:16:47 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 8 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write.00000006
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12460404 0xbe2174 0
1 12641680 0xc0e590 0
3 12489983 0xbe94ff 0
5 12382046 0xbcef5e 0
6 12595301 0xc03065 0
0 12423692 0xbd920c 0
2 12570037 0xbfcdb5 0
4 12511774 0xbeea1e 0
/esfs/jtacquaviva/ioperf/file_write.00000023
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12489993 0xbe9509 0
7 12460415 0xbe217f 0
5 12382057 0xbcef69 0
6 12595312 0xc03070 0
0 12423703 0xbd9217 0
2 12570048 0xbfcdc0 0
4 12511785 0xbeea29 0
1 12641691 0xc0e59b 0
/esfs/jtacquaviva/ioperf/file_write.00000002
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12595298 0xc03062 0
0 12423689 0xbd9209 0
3 12489981 0xbe94fd 0
5 12382044 0xbcef5c 0
2 12570034 0xbfcdb2 0
4 12511773 0xbeea1d 0
7 12460403 0xbe2173 0
1 12641679 0xc0e58f 0
/esfs/jtacquaviva/ioperf/file_write.00000000
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12382039 0xbcef57 0
2 12570030 0xbfcdae 0
4 12511767 0xbeea17 0
1 12641674 0xc0e58a 0
6 12595295 0xc0305f 0
7 12460398 0xbe216e 0
0 12423687 0xbd9207 0
3 12489978 0xbe94fa 0
/esfs/jtacquaviva/ioperf/file_write.00000020
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12511771 0xbeea1b 0
0 12423690 0xbd920a 0
7 12460402 0xbe2172 0
6 12595300 0xc03064 0
1 12641678 0xc0e58e 0
3 12489982 0xbe94fe 0
5 12382045 0xbcef5d 0
2 12570036 0xbfcdb4 0
/esfs/jtacquaviva/ioperf/file_write.00000004
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12511765 0xbeea15 0
3 12489974 0xbe94f6 0
6 12595292 0xc0305c 0
2 12570028 0xbfcdac 0
1 12641672 0xc0e588 0
5 12382038 0xbcef56 0
7 12460396 0xbe216c 0
0 12423684 0xbd9204 0
/esfs/jtacquaviva/ioperf/file_write.00000007
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12423702 0xbd9216 0
2 12570047 0xbfcdbf 0
4 12511784 0xbeea28 0
7 12460414 0xbe217e 0
5 12382056 0xbcef68 0
6 12595311 0xc0306f 0
1 12641690 0xc0e59a 0
3 12489994 0xbe950a 0
/esfs/jtacquaviva/ioperf/file_write.00000018
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12641671 0xc0e587 0
5 12382037 0xbcef55 0
0 12423683 0xbd9203 0
7 12460395 0xbe216b 0
6 12595293 0xc0305d 0
2 12570029 0xbfcdad 0
3 12489975 0xbe94f7 0
4 12511766 0xbeea16 0
/esfs/jtacquaviva/ioperf/file_write.00000012
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12489976 0xbe94f8 0
5 12382040 0xbcef58 0
6 12595296 0xc03060 0
0 12423686 0xbd9206 0
2 12570032 0xbfcdb0 0
4 12511769 0xbeea19 0
7 12460399 0xbe216f 0
1 12641675 0xc0e58b 0
/esfs/jtacquaviva/ioperf/file_write.00000013
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12641677 0xc0e58d 0
6 12595299 0xc03063 0
4 12511772 0xbeea1c 0
3 12489980 0xbe94fc 0
7 12460401 0xbe2171 0
5 12382043 0xbcef5b 0
0 12423691 0xbd920b 0
2 12570035 0xbfcdb3 0
/esfs/jtacquaviva/ioperf/file_write.00000022
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12595303 0xc03067 0
2 12570039 0xbfcdb7 0
7 12460406 0xbe2176 0
3 12489986 0xbe9502 0
0 12423695 0xbd920f 0
4 12511777 0xbeea21 0
1 12641683 0xc0e593 0
5 12382049 0xbcef61 0
/esfs/jtacquaviva/ioperf/file_write.00000003
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12382053 0xbcef65 0
6 12595309 0xc0306d 0
0 12423699 0xbd9213 0
2 12570045 0xbfcdbd 0
4 12511782 0xbeea26 0
7 12460412 0xbe217c 0
1 12641688 0xc0e598 0
3 12489991 0xbe9507 0
/esfs/jtacquaviva/ioperf/file_write.00000008
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12423696 0xbd9210 0
2 12570041 0xbfcdb9 0
4 12511778 0xbeea22 0
7 12460408 0xbe2178 0
1 12641684 0xc0e594 0
3 12489987 0xbe9503 0
5 12382050 0xbcef62 0
6 12595305 0xc03069 0
/esfs/jtacquaviva/ioperf/file_write.00000010
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12423681 0xbd9201 0
2 12570026 0xbfcdaa 0
4 12511763 0xbeea13 0
7 12460393 0xbe2169 0
1 12641669 0xc0e585 0
3 12489972 0xbe94f4 0
5 12382035 0xbcef53 0
6 12595290 0xc0305a 0
/esfs/jtacquaviva/ioperf/file_write.00000017
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12595310 0xc0306e 0
0 12423701 0xbd9215 0
2 12570046 0xbfcdbe 0
4 12511783 0xbeea27 0
7 12460413 0xbe217d 0
1 12641689 0xc0e599 0
3 12489992 0xbe9508 0
5 12382055 0xbcef67 0
/esfs/jtacquaviva/ioperf/file_write.00000016
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12570027 0xbfcdab 0
4 12511764 0xbeea14 0
7 12460394 0xbe216a 0
1 12641670 0xc0e586 0
3 12489973 0xbe94f5 0
5 12382036 0xbcef54 0
6 12595291 0xc0305b 0
0 12423682 0xbd9202 0
/esfs/jtacquaviva/ioperf/file_write.00000005
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12489985 0xbe9501 0
0 12423694 0xbd920e 0
4 12511776 0xbeea20 0
1 12641682 0xc0e592 0
5 12382048 0xbcef60 0
2 12570040 0xbfcdb8 0
7 12460407 0xbe2177 0
6 12595304 0xc03068 0
/esfs/jtacquaviva/ioperf/file_write.00000014
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12570042 0xbfcdba 0
4 12511779 0xbeea23 0
7 12460409 0xbe2179 0
1 12641685 0xc0e595 0
3 12489988 0xbe9504 0
5 12382051 0xbcef63 0
6 12595306 0xc0306a 0
0 12423697 0xbd9211 0
/esfs/jtacquaviva/ioperf/file_write.00000021
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12511780 0xbeea24 0
7 12460410 0xbe217a 0
5 12382052 0xbcef64 0
6 12595307 0xc0306b 0
0 12423698 0xbd9212 0
1 12641686 0xc0e596 0
3 12489990 0xbe9506 0
2 12570044 0xbfcdbc 0
/esfs/jtacquaviva/ioperf/file_write.00000019
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12641673 0xc0e589 0
6 12595294 0xc0305e 0
0 12423685 0xbd9205 0
7 12460397 0xbe216d 0
3 12489977 0xbe94f9 0
2 12570031 0xbfcdaf 0
5 12382041 0xbcef59 0
4 12511768 0xbeea18 0
/esfs/jtacquaviva/ioperf/file_write.00000009
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12382042 0xbcef5a 0
6 12595297 0xc03061 0
0 12423688 0xbd9208 0
2 12570033 0xbfcdb1 0
4 12511770 0xbeea1a 0
7 12460400 0xbe2170 0
1 12641676 0xc0e58c 0
3 12489979 0xbe94fb 0
/esfs/jtacquaviva/ioperf/file_write.00000015
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12641681 0xc0e591 0
3 12489984 0xbe9500 0
5 12382047 0xbcef5f 0
6 12595302 0xc03066 0
0 12423693 0xbd920d 0
2 12570038 0xbfcdb6 0
4 12511775 0xbeea1f 0
7 12460405 0xbe2175 0
/esfs/jtacquaviva/ioperf/file_write.00000001
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12382058 0xbcef6a 0
6 12595313 0xc03071 0
0 12423704 0xbd9218 0
2 12570049 0xbfcdc1 0
4 12511786 0xbeea2a 0
7 12460416 0xbe2180 0
1 12641692 0xc0e59c 0
3 12489995 0xbe950b 0
/esfs/jtacquaviva/ioperf/file_write.00000011
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12489989 0xbe9505 0
2 12570043 0xbfcdbb 0
5 12382054 0xbcef66 0
6 12595308 0xc0306c 0
4 12511781 0xbeea25 0
7 12460411 0xbe217b 0
0 12423700 0xbd9214 0
1 12641687 0xc0e597 0

474
output/COUNT:1#NN:4#PPN:6#API:POSIX#T:1048576.txt Normal file
View File

@ -0,0 +1,474 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 24 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 26843545600 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:4#PPN:6#API:POSIX#T:1048576.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Tue Oct 23 13:58:06 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 26843545600 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 13:58:06 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 24 (6 per node)
repetitions = 3
xfersize = 1 MiB
blocksize = 25 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644245094400.
WARNING: Using actual aggregate bytes moved = 626325979136.
write 4887 26214400 1024.00 0.002933 122.23 0.001729 122.23 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644245094400.
WARNING: Using actual aggregate bytes moved = 619299471360.
write 4732 26214400 1024.00 0.005241 124.82 0.001683 124.82 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644245094400.
WARNING: Using actual aggregate bytes moved = 601632014336.
write 4557 26214400 1024.00 0.006001 125.91 0.001670 125.92 2
Max Write: 4886.84 MiB/sec (5124.23 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 4886.84 4556.71 4725.04 134.86 124.32239 0 24 6 3 1 0 1 0 0 1 26843545600 1048576 626325979136 POSIX 0
Finished: Tue Oct 23 14:04:23 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 24 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 24 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 26843545600 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread4/file -r
+ tee -a ./output/COUNT:1#NN:4#PPN:6#API:POSIX#T:1048576.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Tue Oct 23 14:04:35 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 26843545600 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread4/file -r
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 14:04:35 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/indread4/file
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 24 (6 per node)
repetitions = 3
xfersize = 1 MiB
blocksize = 25 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 3865470566400.
WARNING: Using actual aggregate bytes moved = 269722058752.
read 2143.03 26214400 1024.00 0.001394 120.03 0.000856 120.03 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 3865470566400.
WARNING: Using actual aggregate bytes moved = 322788392960.
read 2564.60 26214400 1024.00 0.000829 120.03 0.000856 120.03 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 3865470566400.
WARNING: Using actual aggregate bytes moved = 329560096768.
read 2618.62 26214400 1024.00 0.001044 120.02 0.000911 120.02 2
Max Read: 2618.62 MiB/sec (2745.83 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 2618.62 2143.03 2442.09 212.61 120.02798 0 24 6 3 1 0 1 0 0 1 26843545600 1048576 269722058752 POSIX 0
Finished: Tue Oct 23 14:10:35 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 8 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write.00000006
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12460139 0xbe206b 0
3 12489718 0xbe93f6 0
6 12595037 0xc02f5d 0
4 12511511 0xbee917 0
5 12381786 0xbcee5a 0
1 12641418 0xc0e48a 0
0 12423431 0xbd9107 0
2 12569773 0xbfccad 0
/esfs/jtacquaviva/ioperf/file_write.00000023
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12460142 0xbe206e 0
1 12641419 0xc0e48b 0
3 12489720 0xbe93f8 0
5 12381787 0xbcee5b 0
6 12595040 0xc02f60 0
0 12423432 0xbd9108 0
2 12569774 0xbfccae 0
4 12511513 0xbee919 0
/esfs/jtacquaviva/ioperf/file_write.00000002
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12489717 0xbe93f5 0
4 12511510 0xbee916 0
7 12460140 0xbe206c 0
1 12641417 0xc0e489 0
5 12381785 0xbcee59 0
6 12595038 0xc02f5e 0
0 12423430 0xbd9106 0
2 12569772 0xbfccac 0
/esfs/jtacquaviva/ioperf/file_write.00000000
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12381772 0xbcee4c 0
2 12569759 0xbfcc9f 0
7 12460127 0xbe205f 0
3 12489706 0xbe93ea 0
0 12423418 0xbd90fa 0
4 12511499 0xbee90b 0
6 12595026 0xc02f52 0
1 12641405 0xc0e47d 0
/esfs/jtacquaviva/ioperf/file_write.00000020
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12423417 0xbd90f9 0
4 12511498 0xbee90a 0
1 12641404 0xc0e47c 0
5 12381773 0xbcee4d 0
6 12595025 0xc02f51 0
2 12569760 0xbfcca0 0
7 12460128 0xbe2060 0
/esfs/jtacquaviva/ioperf/file_write.00000004
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12595034 0xc02f5a 0
0 12423427 0xbd9103 0
2 12569769 0xbfcca9 0
4 12511508 0xbee914 0
7 12460137 0xbe2069 0
1 12641414 0xc0e486 0
3 12489715 0xbe93f3 0
5 12381782 0xbcee56 0
/esfs/jtacquaviva/ioperf/file_write.00000007
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12460130 0xbe2062 0
2 12569764 0xbfcca4 0
1 12641409 0xc0e481 0
4 12511503 0xbee90f 0
5 12381777 0xbcee51 0
3 12489711 0xbe93ef 0
6 12595031 0xc02f57 0
0 12423423 0xbd90ff 0
/esfs/jtacquaviva/ioperf/file_write.00000018
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12511505 0xbee911 0
7 12460134 0xbe2066 0
1 12641411 0xc0e483 0
3 12489712 0xbe93f0 0
5 12381779 0xbcee53 0
6 12595032 0xc02f58 0
0 12423424 0xbd9100 0
2 12569766 0xbfcca6 0
/esfs/jtacquaviva/ioperf/file_write.00000012
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12489722 0xbe93fa 0
0 12423434 0xbd910a 0
7 12460144 0xbe2070 0
5 12381789 0xbcee5d 0
2 12569777 0xbfccb1 0
1 12641422 0xc0e48e 0
6 12595043 0xc02f63 0
4 12511516 0xbee91c 0
/esfs/jtacquaviva/ioperf/file_write.00000013
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12569771 0xbfccab 0
1 12641416 0xc0e488 0
5 12381784 0xbcee58 0
0 12423429 0xbd9105 0
7 12460141 0xbe206d 0
3 12489719 0xbe93f7 0
4 12511512 0xbee918 0
6 12595039 0xc02f5f 0
/esfs/jtacquaviva/ioperf/file_write.00000022
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12569762 0xbfcca2 0
3 12489708 0xbe93ec 0
5 12381775 0xbcee4f 0
4 12511502 0xbee90e 0
0 12423421 0xbd90fd 0
1 12641410 0xc0e482 0
7 12460133 0xbe2065 0
6 12595029 0xc02f55 0
/esfs/jtacquaviva/ioperf/file_write.00000003
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12641420 0xc0e48c 0
3 12489721 0xbe93f9 0
5 12381788 0xbcee5c 0
6 12595041 0xc02f61 0
0 12423433 0xbd9109 0
2 12569775 0xbfccaf 0
4 12511514 0xbee91a 0
7 12460143 0xbe206f 0
/esfs/jtacquaviva/ioperf/file_write.00000008
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12595042 0xc02f62 0
2 12569776 0xbfccb0 0
4 12511515 0xbee91b 0
1 12641421 0xc0e48d 0
0 12423435 0xbd910b 0
7 12460145 0xbe2071 0
3 12489723 0xbe93fb 0
/esfs/jtacquaviva/ioperf/file_write.00000010
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12489713 0xbe93f1 0
6 12595033 0xc02f59 0
2 12569767 0xbfcca7 0
7 12460136 0xbe2068 0
5 12381781 0xbcee55 0
0 12423426 0xbd9102 0
4 12511507 0xbee913 0
1 12641413 0xc0e485 0
/esfs/jtacquaviva/ioperf/file_write.00000017
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12381774 0xbcee4e 0
6 12595027 0xc02f53 0
0 12423419 0xbd90fb 0
2 12569761 0xbfcca1 0
4 12511500 0xbee90c 0
7 12460129 0xbe2061 0
1 12641406 0xc0e47e 0
3 12489707 0xbe93eb 0
/esfs/jtacquaviva/ioperf/file_write.00000016
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12489704 0xbe93e8 0
6 12595023 0xc02f4f 0
2 12569757 0xbfcc9d 0
7 12460126 0xbe205e 0
5 12381771 0xbcee4b 0
0 12423416 0xbd90f8 0
4 12511497 0xbee909 0
1 12641403 0xc0e47b 0
/esfs/jtacquaviva/ioperf/file_write.00000005
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12595028 0xc02f54 0
1 12641407 0xc0e47f 0
0 12423420 0xbd90fc 0
7 12460132 0xbe2064 0
2 12569765 0xbfcca5 0
3 12489710 0xbe93ee 0
4 12511504 0xbee910 0
5 12381778 0xbcee52 0
/esfs/jtacquaviva/ioperf/file_write.00000014
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12460135 0xbe2067 0
5 12381780 0xbcee54 0
0 12423425 0xbd9101 0
4 12511506 0xbee912 0
1 12641412 0xc0e484 0
6 12595035 0xc02f5b 0
2 12569768 0xbfcca8 0
3 12489714 0xbe93f2 0
/esfs/jtacquaviva/ioperf/file_write.00000021
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12489724 0xbe93fc 0
5 12381790 0xbcee5e 0
6 12595044 0xc02f64 0
0 12423436 0xbd910c 0
2 12569778 0xbfccb2 0
4 12511517 0xbee91d 0
7 12460146 0xbe2072 0
1 12641423 0xc0e48f 0
/esfs/jtacquaviva/ioperf/file_write.00000019
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12423428 0xbd9104 0
2 12569770 0xbfccaa 0
4 12511509 0xbee915 0
7 12460138 0xbe206a 0
1 12641415 0xc0e487 0
3 12489716 0xbe93f4 0
5 12381783 0xbcee57 0
6 12595036 0xc02f5c 0
/esfs/jtacquaviva/ioperf/file_write.00000009
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12460125 0xbe205d 0
5 12381770 0xbcee4a 0
0 12423415 0xbd90f7 0
4 12511496 0xbee908 0
1 12641402 0xc0e47a 0
6 12595024 0xc02f50 0
2 12569758 0xbfcc9e 0
3 12489705 0xbe93e9 0
/esfs/jtacquaviva/ioperf/file_write.00000015
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12569763 0xbfcca3 0
4 12511501 0xbee90d 0
7 12460131 0xbe2063 0
1 12641408 0xc0e480 0
3 12489709 0xbe93ed 0
5 12381776 0xbcee50 0
6 12595030 0xc02f56 0
0 12423422 0xbd90fe 0
/esfs/jtacquaviva/ioperf/file_write.00000001
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12595046 0xc02f66 0
0 12423438 0xbd910e 0
2 12569780 0xbfccb4 0
4 12511519 0xbee91f 0
7 12460148 0xbe2074 0
1 12641425 0xc0e491 0
3 12489726 0xbe93fe 0
5 12381792 0xbcee60 0
/esfs/jtacquaviva/ioperf/file_write.00000011
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12381791 0xbcee5f 0
6 12595045 0xc02f65 0
0 12423437 0xbd910d 0
2 12569779 0xbfccb3 0
4 12511518 0xbee91e 0
7 12460147 0xbe2073 0
1 12641424 0xc0e490 0
3 12489725 0xbe93fd 0

467
output/COUNT:1#NN:4#PPN:6#API:POSIX#T:10485760.txt Normal file
View File

@ -0,0 +1,467 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 24 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 26843545600 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:4#PPN:6#API:POSIX#T:10485760.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Tue Oct 23 11:54:39 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 26843545600 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 11:54:39 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 24 (6 per node)
repetitions = 3
xfersize = 10 MiB
blocksize = 25 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
write 5643 26214400 10240 0.002738 108.88 0.001553 108.88 0
write 5632 26214400 10240 0.004198 109.09 0.001529 109.09 1
write 5681 26214400 10240 0.003430 108.14 0.001607 108.14 2
Max Write: 5681.30 MiB/sec (5957.27 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 5681.30 5631.95 5652.05 21.16 108.70548 0 24 6 3 1 0 1 0 0 1 26843545600 10485760 644245094400 POSIX 0
Finished: Tue Oct 23 12:00:10 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 24 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 24 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 26843545600 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread4/file -r
+ tee -a ./output/COUNT:1#NN:4#PPN:6#API:POSIX#T:10485760.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Tue Oct 23 12:00:21 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 26843545600 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread4/file -r
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 12:00:21 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/indread4/file
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 24 (6 per node)
repetitions = 3
xfersize = 10 MiB
blocksize = 25 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 3865470566400.
WARNING: Using actual aggregate bytes moved = 271790899200.
read 2157.82 26214400 10240 0.001319 120.12 0.000687 120.12 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 3865470566400.
WARNING: Using actual aggregate bytes moved = 358371819520.
read 2844.33 26214400 10240 0.000622 120.16 0.000686 120.16 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 3865470566400.
WARNING: Using actual aggregate bytes moved = 355320463360.
read 2821.04 26214400 10240 0.000747 120.12 0.000900 120.12 2
Max Read: 2844.33 MiB/sec (2982.50 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 2844.33 2157.82 2607.73 318.28 120.13276 0 24 6 3 1 0 1 0 0 1 26843545600 10485760 271790899200 POSIX 0
Finished: Tue Oct 23 12:06:22 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 8 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write.00000006
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12459879 0xbe1f67 0
1 12641157 0xc0e385 0
3 12489459 0xbe92f3 0
5 12381520 0xbced50 0
6 12594778 0xc02e5a 0
0 12423170 0xbd9002 0
2 12569511 0xbfcba7 0
4 12511247 0xbee80f 0
/esfs/jtacquaviva/ioperf/file_write.00000023
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12594771 0xc02e53 0
2 12569504 0xbfcba0 0
1 12641150 0xc0e37e 0
0 12423164 0xbd8ffc 0
4 12511240 0xbee808 0
3 12489453 0xbe92ed 0
7 12459874 0xbe1f62 0
5 12381514 0xbced4a 0
/esfs/jtacquaviva/ioperf/file_write.00000002
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12459876 0xbe1f64 0
1 12641155 0xc0e383 0
3 12489457 0xbe92f1 0
5 12381518 0xbced4e 0
6 12594776 0xc02e58 0
0 12423168 0xbd9000 0
2 12569509 0xbfcba5 0
4 12511246 0xbee80e 0
/esfs/jtacquaviva/ioperf/file_write.00000000
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12489452 0xbe92ec 0
0 12423163 0xbd8ffb 0
5 12381513 0xbced49 0
7 12459873 0xbe1f61 0
6 12594772 0xc02e54 0
1 12641152 0xc0e380 0
2 12569505 0xbfcba1 0
4 12511242 0xbee80a 0
/esfs/jtacquaviva/ioperf/file_write.00000020
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12511236 0xbee804 0
7 12459868 0xbe1f5c 0
1 12641146 0xc0e37a 0
3 12489449 0xbe92e9 0
5 12381510 0xbced46 0
6 12594768 0xc02e50 0
0 12423159 0xbd8ff7 0
2 12569501 0xbfcb9d 0
/esfs/jtacquaviva/ioperf/file_write.00000004
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12489461 0xbe92f5 0
5 12381522 0xbced52 0
6 12594780 0xc02e5c 0
0 12423172 0xbd9004 0
2 12569513 0xbfcba9 0
4 12511249 0xbee811 0
7 12459881 0xbe1f69 0
1 12641159 0xc0e387 0
/esfs/jtacquaviva/ioperf/file_write.00000007
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12381504 0xbced40 0
7 12459864 0xbe1f58 0
6 12594763 0xc02e4b 0
0 12423155 0xbd8ff3 0
4 12511233 0xbee801 0
2 12569497 0xbfcb99 0
1 12641143 0xc0e377 0
3 12489446 0xbe92e6 0
/esfs/jtacquaviva/ioperf/file_write.00000018
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12489444 0xbe92e4 0
5 12381506 0xbced42 0
6 12594764 0xc02e4c 0
0 12423157 0xbd8ff5 0
2 12569498 0xbfcb9a 0
4 12511234 0xbee802 0
7 12459866 0xbe1f5a 0
1 12641144 0xc0e378 0
/esfs/jtacquaviva/ioperf/file_write.00000012
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12641154 0xc0e382 0
3 12489456 0xbe92f0 0
5 12381517 0xbced4d 0
6 12594775 0xc02e57 0
0 12423167 0xbd8fff 0
2 12569508 0xbfcba4 0
4 12511245 0xbee80d 0
7 12459878 0xbe1f66 0
/esfs/jtacquaviva/ioperf/file_write.00000013
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12381501 0xbced3d 0
6 12594759 0xc02e47 0
0 12423151 0xbd8fef 0
2 12569492 0xbfcb94 0
4 12511228 0xbee7fc 0
7 12459860 0xbe1f54 0
1 12641138 0xc0e372 0
3 12489440 0xbe92e0 0
/esfs/jtacquaviva/ioperf/file_write.00000022
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12423153 0xbd8ff1 0
2 12569494 0xbfcb96 0
4 12511230 0xbee7fe 0
7 12459862 0xbe1f56 0
1 12641140 0xc0e374 0
3 12489442 0xbe92e2 0
5 12381503 0xbced3f 0
6 12594761 0xc02e49 0
/esfs/jtacquaviva/ioperf/file_write.00000003
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12569499 0xbfcb9b 0
3 12489447 0xbe92e7 0
4 12511237 0xbee805 0
7 12459869 0xbe1f5d 0
5 12381509 0xbced45 0
6 12594767 0xc02e4f 0
1 12641148 0xc0e37c 0
0 12423160 0xbd8ff8 0
/esfs/jtacquaviva/ioperf/file_write.00000008
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12641149 0xc0e37d 0
3 12489451 0xbe92eb 0
5 12381512 0xbced48 0
6 12594770 0xc02e52 0
0 12423162 0xbd8ffa 0
2 12569503 0xbfcb9f 0
4 12511239 0xbee807 0
7 12459871 0xbe1f5f 0
/esfs/jtacquaviva/ioperf/file_write.00000010
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12381508 0xbced44 0
6 12594766 0xc02e4e 0
4 12511235 0xbee803 0
7 12459867 0xbe1f5b 0
1 12641145 0xc0e379 0
0 12423158 0xbd8ff6 0
2 12569500 0xbfcb9c 0
3 12489448 0xbe92e8 0
/esfs/jtacquaviva/ioperf/file_write.00000017
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12459870 0xbe1f5e 0
1 12641147 0xc0e37b 0
3 12489450 0xbe92ea 0
5 12381511 0xbced47 0
6 12594769 0xc02e51 0
0 12423161 0xbd8ff9 0
2 12569502 0xbfcb9e 0
4 12511238 0xbee806 0
/esfs/jtacquaviva/ioperf/file_write.00000016
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12641158 0xc0e386 0
3 12489460 0xbe92f4 0
5 12381521 0xbced51 0
6 12594779 0xc02e5b 0
0 12423171 0xbd9003 0
2 12569512 0xbfcba8 0
4 12511248 0xbee810 0
7 12459880 0xbe1f68 0
/esfs/jtacquaviva/ioperf/file_write.00000005
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12594760 0xc02e48 0
0 12423152 0xbd8ff0 0
2 12569493 0xbfcb95 0
4 12511229 0xbee7fd 0
7 12459861 0xbe1f55 0
1 12641139 0xc0e373 0
3 12489441 0xbe92e1 0
5 12381502 0xbced3e 0
/esfs/jtacquaviva/ioperf/file_write.00000014
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12511232 0xbee800 0
2 12569496 0xbfcb98 0
1 12641142 0xc0e376 0
3 12489445 0xbe92e5 0
0 12423156 0xbd8ff4 0
5 12381507 0xbced43 0
6 12594765 0xc02e4d 0
7 12459865 0xbe1f59 0
/esfs/jtacquaviva/ioperf/file_write.00000021
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12381523 0xbced53 0
6 12594781 0xc02e5d 0
0 12423173 0xbd9005 0
2 12569514 0xbfcbaa 0
4 12511250 0xbee812 0
7 12459882 0xbe1f6a 0
1 12641160 0xc0e388 0
3 12489462 0xbe92f6 0
/esfs/jtacquaviva/ioperf/file_write.00000019
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12511244 0xbee80c 0
7 12459877 0xbe1f65 0
1 12641156 0xc0e384 0
3 12489458 0xbe92f2 0
5 12381519 0xbced4f 0
6 12594777 0xc02e59 0
0 12423169 0xbd9001 0
2 12569510 0xbfcba6 0
/esfs/jtacquaviva/ioperf/file_write.00000009
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12489439 0xbe92df 0
5 12381500 0xbced3c 0
6 12594758 0xc02e46 0
0 12423150 0xbd8fee 0
2 12569491 0xbfcb93 0
4 12511227 0xbee7fb 0
7 12459859 0xbe1f53 0
1 12641137 0xc0e371 0
/esfs/jtacquaviva/ioperf/file_write.00000015
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12459872 0xbe1f60 0
1 12641151 0xc0e37f 0
0 12423165 0xbd8ffd 0
3 12489454 0xbe92ee 0
4 12511241 0xbee809 0
5 12381515 0xbced4b 0
6 12594773 0xc02e55 0
2 12569506 0xbfcba2 0
/esfs/jtacquaviva/ioperf/file_write.00000001
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12569495 0xbfcb97 0
4 12511231 0xbee7ff 0
7 12459863 0xbe1f57 0
1 12641141 0xc0e375 0
6 12594762 0xc02e4a 0
0 12423154 0xbd8ff2 0
3 12489443 0xbe92e3 0
5 12381505 0xbced41 0
/esfs/jtacquaviva/ioperf/file_write.00000011
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12459875 0xbe1f63 0
1 12641153 0xc0e381 0
3 12489455 0xbe92ef 0
5 12381516 0xbced4c 0
6 12594774 0xc02e56 0
0 12423166 0xbd8ffe 0
2 12569507 0xbfcba3 0
4 12511243 0xbee80b 0

476
output/COUNT:1#NN:4#PPN:6#API:POSIX#T:16384.txt Normal file
View File

@ -0,0 +1,476 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 24 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 26843545600 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:4#PPN:6#API:POSIX#T:16384.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Tue Oct 23 18:11:22 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 26843545600 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 18:11:22 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 24 (6 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 25 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644230332416.
WARNING: Using actual aggregate bytes moved = 40317681664.
write 260.19 26214400 16.00 0.002650 147.77 0.001447 147.77 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644241883136.
WARNING: Using actual aggregate bytes moved = 35624615936.
write 232.62 26214400 16.00 0.016533 146.04 0.001388 146.05 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644233265152.
WARNING: Using actual aggregate bytes moved = 35871342592.
write 231.20 26214400 16.00 0.003769 147.96 0.001395 147.96 2
Max Write: 260.19 MiB/sec (272.83 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 260.19 231.20 241.34 13.34 147.26318 0 24 6 3 1 0 1 0 0 1 26843545600 16384 40317681664 POSIX 0
Finished: Tue Oct 23 18:18:45 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 24 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 6 -np 24 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 26843545600 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread4/file -r
+ tee -a ./output/COUNT:1#NN:4#PPN:6#API:POSIX#T:16384.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Tue Oct 23 18:18:59 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 26843545600 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread4/file -r
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 18:18:59 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/indread4/file
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 24 (6 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 25 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 3865470566400.
WARNING: Using actual aggregate bytes moved = 8496349184.
read 67.43 26214400 16.00 0.001410 120.16 0.000621 120.16 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 3865470566400.
WARNING: Using actual aggregate bytes moved = 9008496640.
read 71.53 26214400 16.00 0.004323 120.10 0.001665 120.11 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 3865470566400.
WARNING: Using actual aggregate bytes moved = 9123708928.
read 72.44 26214400 16.00 0.000808 120.11 0.002333 120.11 2
Max Read: 72.44 MiB/sec (75.96 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 72.44 67.43 70.47 2.18 120.12622 0 24 6 3 1 0 1 0 0 1 26843545600 16384 8496349184 POSIX 0
Finished: Tue Oct 23 18:25:00 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 8 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write.00000006
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12460660 0xbe2274 0
1 12641937 0xc0e691 0
3 12490239 0xbe95ff 0
5 12382303 0xbcf05f 0
6 12595559 0xc03167 0
0 12423948 0xbd930c 0
2 12570292 0xbfceb4 0
4 12512029 0xbeeb1d 0
/esfs/jtacquaviva/ioperf/file_write.00000023
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12512034 0xbeeb22 0
5 12382310 0xbcf066 0
1 12641944 0xc0e698 0
6 12595566 0xc0316e 0
7 12460668 0xbe227c 0
3 12490247 0xbe9607 0
0 12423956 0xbd9314 0
2 12570300 0xbfcebc 0
/esfs/jtacquaviva/ioperf/file_write.00000002
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12570307 0xbfcec3 0
4 12512044 0xbeeb2c 0
7 12460675 0xbe2283 0
1 12641952 0xc0e6a0 0
3 12490254 0xbe960e 0
5 12382318 0xbcf06e 0
6 12595574 0xc03176 0
0 12423963 0xbd931b 0
/esfs/jtacquaviva/ioperf/file_write.00000000
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12382314 0xbcf06a 0
6 12595570 0xc03172 0
0 12423959 0xbd9317 0
2 12570303 0xbfcebf 0
4 12512040 0xbeeb28 0
7 12460671 0xbe227f 0
1 12641948 0xc0e69c 0
3 12490250 0xbe960a 0
/esfs/jtacquaviva/ioperf/file_write.00000020
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12512051 0xbeeb33 0
7 12460682 0xbe228a 0
1 12641959 0xc0e6a7 0
3 12490261 0xbe9615 0
5 12382325 0xbcf075 0
6 12595581 0xc0317d 0
0 12423970 0xbd9322 0
2 12570314 0xbfceca 0
/esfs/jtacquaviva/ioperf/file_write.00000004
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12641938 0xc0e692 0
6 12595560 0xc03168 0
2 12570293 0xbfceb5 0
7 12460661 0xbe2275 0
5 12382305 0xbcf061 0
0 12423950 0xbd930e 0
4 12512031 0xbeeb1f 0
3 12490241 0xbe9601 0
/esfs/jtacquaviva/ioperf/file_write.00000007
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12423952 0xbd9310 0
2 12570296 0xbfceb8 0
4 12512033 0xbeeb21 0
7 12460664 0xbe2278 0
1 12641941 0xc0e695 0
3 12490243 0xbe9603 0
5 12382307 0xbcf063 0
6 12595563 0xc0316b 0
/esfs/jtacquaviva/ioperf/file_write.00000018
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12595571 0xc03173 0
0 12423960 0xbd9318 0
2 12570304 0xbfcec0 0
4 12512041 0xbeeb29 0
7 12460672 0xbe2280 0
1 12641949 0xc0e69d 0
3 12490251 0xbe960b 0
5 12382315 0xbcf06b 0
/esfs/jtacquaviva/ioperf/file_write.00000012
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12460669 0xbe227d 0
5 12382312 0xbcf068 0
6 12595568 0xc03170 0
2 12570301 0xbfcebd 0
1 12641946 0xc0e69a 0
0 12423958 0xbd9316 0
3 12490249 0xbe9609 0
4 12512039 0xbeeb27 0
/esfs/jtacquaviva/ioperf/file_write.00000013
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12490256 0xbe9610 0
6 12595576 0xc03178 0
2 12570309 0xbfcec5 0
4 12512046 0xbeeb2e 0
5 12382321 0xbcf071 0
0 12423966 0xbd931e 0
7 12460678 0xbe2286 0
1 12641955 0xc0e6a3 0
/esfs/jtacquaviva/ioperf/file_write.00000022
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12382304 0xbcf060 0
0 12423949 0xbd930d 0
4 12512030 0xbeeb1e 0
1 12641939 0xc0e693 0
3 12490240 0xbe9600 0
6 12595561 0xc03169 0
2 12570294 0xbfceb6 0
7 12460662 0xbe2276 0
/esfs/jtacquaviva/ioperf/file_write.00000003
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12595578 0xc0317a 0
0 12423967 0xbd931f 0
2 12570311 0xbfcec7 0
4 12512048 0xbeeb30 0
7 12460679 0xbe2287 0
1 12641956 0xc0e6a4 0
3 12490258 0xbe9612 0
5 12382322 0xbcf072 0
/esfs/jtacquaviva/ioperf/file_write.00000008
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12460677 0xbe2285 0
5 12382320 0xbcf070 0
0 12423965 0xbd931d 0
1 12641954 0xc0e6a2 0
6 12595577 0xc03179 0
2 12570310 0xbfcec6 0
4 12512047 0xbeeb2f 0
3 12490257 0xbe9611 0
/esfs/jtacquaviva/ioperf/file_write.00000010
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12570297 0xbfceb9 0
1 12641942 0xc0e696 0
5 12382308 0xbcf064 0
4 12512035 0xbeeb23 0
3 12490246 0xbe9606 0
6 12595565 0xc0316d 0
7 12460667 0xbe227b 0
0 12423955 0xbd9313 0
/esfs/jtacquaviva/ioperf/file_write.00000017
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12512042 0xbeeb2a 0
3 12490252 0xbe960c 0
6 12595572 0xc03174 0
7 12460674 0xbe2282 0
5 12382317 0xbcf06d 0
0 12423962 0xbd931a 0
2 12570306 0xbfcec2 0
1 12641951 0xc0e69f 0
/esfs/jtacquaviva/ioperf/file_write.00000016
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12512028 0xbeeb1c 0
7 12460659 0xbe2273 0
1 12641936 0xc0e690 0
3 12490238 0xbe95fe 0
5 12382302 0xbcf05e 0
6 12595558 0xc03166 0
0 12423947 0xbd930b 0
2 12570291 0xbfceb3 0
/esfs/jtacquaviva/ioperf/file_write.00000005
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12570313 0xbfcec9 0
4 12512050 0xbeeb32 0
7 12460681 0xbe2289 0
1 12641958 0xc0e6a6 0
3 12490260 0xbe9614 0
5 12382324 0xbcf074 0
6 12595580 0xc0317c 0
0 12423969 0xbd9321 0
/esfs/jtacquaviva/ioperf/file_write.00000014
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12595562 0xc0316a 0
0 12423951 0xbd930f 0
2 12570295 0xbfceb7 0
4 12512032 0xbeeb20 0
7 12460663 0xbe2277 0
1 12641940 0xc0e694 0
3 12490242 0xbe9602 0
5 12382306 0xbcf062 0
/esfs/jtacquaviva/ioperf/file_write.00000021
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12512045 0xbeeb2d 0
7 12460676 0xbe2284 0
1 12641953 0xc0e6a1 0
3 12490255 0xbe960f 0
5 12382319 0xbcf06f 0
6 12595575 0xc03177 0
0 12423964 0xbd931c 0
2 12570308 0xbfcec4 0
/esfs/jtacquaviva/ioperf/file_write.00000019
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12423968 0xbd9320 0
2 12570312 0xbfcec8 0
4 12512049 0xbeeb31 0
7 12460680 0xbe2288 0
1 12641957 0xc0e6a5 0
3 12490259 0xbe9613 0
5 12382323 0xbcf073 0
6 12595579 0xc0317b 0
/esfs/jtacquaviva/ioperf/file_write.00000009
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12490248 0xbe9608 0
0 12423957 0xbd9315 0
4 12512038 0xbeeb26 0
5 12382313 0xbcf069 0
2 12570302 0xbfcebe 0
6 12595569 0xc03171 0
7 12460670 0xbe227e 0
1 12641947 0xc0e69b 0
/esfs/jtacquaviva/ioperf/file_write.00000015
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12460665 0xbe2279 0
3 12490244 0xbe9604 0
6 12595564 0xc0316c 0
0 12423953 0xbd9311 0
1 12641943 0xc0e697 0
5 12382309 0xbcf065 0
2 12570298 0xbfceba 0
4 12512036 0xbeeb24 0
/esfs/jtacquaviva/ioperf/file_write.00000001
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12490245 0xbe9605 0
7 12460666 0xbe227a 0
5 12382311 0xbcf067 0
0 12423954 0xbd9312 0
2 12570299 0xbfcebb 0
1 12641945 0xc0e699 0
6 12595567 0xc0316f 0
4 12512037 0xbeeb25 0
/esfs/jtacquaviva/ioperf/file_write.00000011
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12423961 0xbd9319 0
7 12460673 0xbe2281 0
1 12641950 0xc0e69e 0
5 12382316 0xbcf06c 0
2 12570305 0xbfcec1 0
3 12490253 0xbe960d 0
4 12512043 0xbeeb2b 0
6 12595573 0xc03175 0

110
output/COUNT:1#NN:4#PPN:8#API:MPIIO#T:102400.txt Normal file
View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 8 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 20132659200 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:4#PPN:8#API:MPIIO#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Tue Oct 23 15:51:21 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 20132659200 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 15:51:21 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 32 (8 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 18.75 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644243865600.
WARNING: Using actual aggregate bytes moved = 40235110400.
write 318.37 19660800 100.00 0.011896 120.51 0.000560 120.52 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644240896000.
WARNING: Using actual aggregate bytes moved = 39003648000.
write 308.67 19660800 100.00 0.007467 120.50 0.000531 120.51 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644243353600.
WARNING: Using actual aggregate bytes moved = 39161446400.
write 309.89 19660800 100.00 0.013143 120.51 0.000584 120.52 2
Max Write: 318.37 MiB/sec (333.83 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 318.37 308.67 312.31 4.31 120.51676 0 32 8 3 0 0 1 0 0 1 20132659200 102400 40235110400 MPIIO 0
Finished: Tue Oct 23 15:57:25 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 8 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 8 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 20132659200 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
+ tee -a ./output/COUNT:1#NN:4#PPN:8#API:MPIIO#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Tue Oct 23 15:57:40 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 20132659200 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 15:57:40 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 32 (8 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 18.75 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 35536179200.
read 281.53 19660800 100.00 0.020202 120.36 0.008444 120.38 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 36893388800.
read 292.32 19660800 100.00 0.005742 120.36 0.004209 120.36 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 37524582400.
read 297.33 19660800 100.00 0.005775 120.36 0.004299 120.36 2
Max Read: 297.33 MiB/sec (311.77 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 297.33 281.53 290.39 6.59 120.36648 0 32 8 3 0 0 1 0 0 1 20132659200 102400 35536179200 MPIIO 0
Finished: Tue Oct 23 16:03:41 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 8 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12641620 0xc0e554 0
3 12489923 0xbe94c3 0
5 12381987 0xbcef23 0
6 12595241 0xc03029 0
0 12423632 0xbd91d0 0
2 12569977 0xbfcd79 0
4 12511714 0xbee9e2 0
7 12460344 0xbe2138 0

110
output/COUNT:1#NN:4#PPN:8#API:MPIIO#T:1048576.txt Normal file
View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 8 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 20132659200 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:4#PPN:8#API:MPIIO#T:1048576.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Tue Oct 23 13:45:24 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 20132659200 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 13:45:24 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 32 (8 per node)
repetitions = 3
xfersize = 1 MiB
blocksize = 18.75 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644244045824.
WARNING: Using actual aggregate bytes moved = 434761629696.
write 3453.61 19660800 1024.00 0.010298 120.04 0.002244 120.05 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644244045824.
WARNING: Using actual aggregate bytes moved = 489523511296.
write 3727 19660800 1024.00 5.18 120.07 0.000567 125.24 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644244045824.
WARNING: Using actual aggregate bytes moved = 494812528640.
write 3742 19660800 1024.00 6.04 120.06 0.000624 126.10 2
Max Write: 3742.18 MiB/sec (3923.96 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 3742.18 3453.61 3641.08 132.70 123.79989 0 32 8 3 0 0 1 0 0 1 20132659200 1048576 434761629696 MPIIO 0
Finished: Tue Oct 23 13:51:47 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 8 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ tee -a ./output/COUNT:1#NN:4#PPN:8#API:MPIIO#T:1048576.txt
+ /opt/ddn/mvapich/bin/mpiexec -ppn 8 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 20132659200 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Tue Oct 23 13:52:04 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 20132659200 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 13:52:04 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 32 (8 per node)
repetitions = 3
xfersize = 1 MiB
blocksize = 18.75 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 157790765056.
read 1251.43 19660800 1024.00 0.181740 120.07 0.002955 120.25 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 162078392320.
read 1287.40 19660800 1024.00 0.004644 120.06 0.003104 120.06 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 161980874752.
read 1286.56 19660800 1024.00 0.004534 120.06 0.003093 120.07 2
Max Read: 1287.40 MiB/sec (1349.94 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 1287.40 1251.43 1275.13 16.76 120.12680 0 32 8 3 0 0 1 0 0 1 20132659200 1048576 157790765056 MPIIO 0
Finished: Tue Oct 23 13:58:04 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 8 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12381720 0xbcee18 0
6 12594974 0xc02f1e 0
0 12423367 0xbd90c7 0
2 12569708 0xbfcc6c 0
4 12511448 0xbee8d8 0
7 12460076 0xbe202c 0
1 12641354 0xc0e44a 0
3 12489657 0xbe93b9 0

110
output/COUNT:1#NN:4#PPN:8#API:MPIIO#T:10485760.txt Normal file
View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 8 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 20132659200 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:4#PPN:8#API:MPIIO#T:10485760.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Tue Oct 23 11:41:58 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 20132659200 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 11:41:58 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 32 (8 per node)
repetitions = 3
xfersize = 10 MiB
blocksize = 18.75 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644245094400.
WARNING: Using actual aggregate bytes moved = 337914101760.
write 2677.29 19660800 10240 0.008235 120.36 0.000516 120.37 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644234608640.
WARNING: Using actual aggregate bytes moved = 341500231680.
write 2585.91 19660800 10240 5.69 120.25 0.000565 125.94 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644245094400.
WARNING: Using actual aggregate bytes moved = 335837921280.
write 2546.56 19660800 10240 5.54 120.23 0.000936 125.77 2
Max Write: 2677.29 MiB/sec (2807.34 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 2677.29 2546.56 2603.25 54.76 124.02722 0 32 8 3 0 0 1 0 0 1 20132659200 10485760 337914101760 MPIIO 0
Finished: Tue Oct 23 11:48:22 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 8 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 8 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 20132659200 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
+ tee -a ./output/COUNT:1#NN:4#PPN:8#API:MPIIO#T:10485760.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Tue Oct 23 11:48:37 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 20132659200 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 11:48:37 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 32 (8 per node)
repetitions = 3
xfersize = 10 MiB
blocksize = 18.75 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 352678051840.
read 2798.38 19660800 10240 0.070351 120.12 0.001189 120.19 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 358046760960.
read 2843.07 19660800 10240 0.002883 120.10 0.001274 120.10 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 358738821120.
read 2848.64 19660800 10240 0.002712 120.10 0.001250 120.10 2
Max Read: 2848.64 MiB/sec (2987.01 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 2848.64 2798.38 2830.03 22.49 120.13099 0 32 8 3 0 0 1 0 0 1 20132659200 10485760 352678051840 MPIIO 0
Finished: Tue Oct 23 11:54:37 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 8 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12489389 0xbe92ad 0
5 12381451 0xbced0b 0
6 12594709 0xc02e15 0
0 12423102 0xbd8fbe 0
2 12569442 0xbfcb62 0
4 12511178 0xbee7ca 0
7 12459810 0xbe1f22 0
1 12641088 0xc0e340 0

110
output/COUNT:1#NN:4#PPN:8#API:MPIIO#T:16384.txt Normal file
View File

@ -0,0 +1,110 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 8 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 20132659200 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:4#PPN:8#API:MPIIO#T:16384.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Tue Oct 23 17:58:56 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 20132659200 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 17:58:56 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/ioperf/file_write
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 32 (8 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 18.75 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644243587072.
WARNING: Using actual aggregate bytes moved = 6858686464.
write 53.46 19660800 16.00 0.010894 122.35 0.000400 122.36 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644243652608.
WARNING: Using actual aggregate bytes moved = 6952583168.
write 54.20 19660800 16.00 0.006475 122.32 0.000468 122.33 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644244602880.
WARNING: Using actual aggregate bytes moved = 6846709760.
write 53.45 19660800 16.00 0.002086 122.16 0.000515 122.16 2
Max Write: 54.20 MiB/sec (56.83 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 54.20 53.45 53.70 0.35 122.28277 0 32 8 3 0 0 1 0 0 1 20132659200 16384 6858686464 MPIIO 0
Finished: Tue Oct 23 18:05:04 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 8 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 8 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 20132659200 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
+ tee -a ./output/COUNT:1#NN:4#PPN:8#API:MPIIO#T:16384.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
ior WARNING: fsync() only available in POSIX. Using value of 0.
Began: Tue Oct 23 18:05:13 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 20132659200 -D 120 -a MPIIO -e -g -z -k -o /esfs/jtacquaviva/file_read -r
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 18:05:13 2018
Summary:
api = MPIIO (version=3, subversion=0)
test filename = /esfs/jtacquaviva/file_read
access = single-shared-file
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 32 (8 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 18.75 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 7268204544.
read 56.75 19660800 16.00 0.010304 122.13 0.002476 122.14 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 7386972160.
read 57.68 19660800 16.00 0.002667 122.13 0.001126 122.13 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 4517768724480.
WARNING: Using actual aggregate bytes moved = 7405846528.
read 57.83 19660800 16.00 0.002670 122.12 0.001154 122.13 2
Max Read: 57.83 MiB/sec (60.64 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 57.83 56.75 57.42 0.48 122.13088 0 32 8 3 0 0 1 0 0 1 20132659200 16384 7268204544 MPIIO 0
Finished: Tue Oct 23 18:11:20 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 8 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12460610 0xbe2242 0
1 12641888 0xc0e660 0
3 12490189 0xbe95cd 0
5 12382253 0xbcf02d 0
6 12595509 0xc03135 0
0 12423899 0xbd92db 0
2 12570243 0xbfce83 0
4 12511981 0xbeeaed 0

604
output/COUNT:1#NN:4#PPN:8#API:POSIX#T:102400.txt Normal file
View File

@ -0,0 +1,604 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 8 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 20132659200 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:4#PPN:8#API:POSIX#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Tue Oct 23 15:38:08 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 20132659200 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 15:38:08 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 32 (8 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 18.75 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644230553600.
WARNING: Using actual aggregate bytes moved = 129645056000.
write 913.55 19660800 100.00 0.002680 135.34 0.000347 135.34 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644245094400.
WARNING: Using actual aggregate bytes moved = 111926988800.
write 765.87 19660800 100.00 0.016071 139.36 0.000441 139.37 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644222156800.
WARNING: Using actual aggregate bytes moved = 114604748800.
write 795.73 19660800 100.00 0.009010 137.34 0.000454 137.35 2
Max Write: 913.55 MiB/sec (957.93 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 913.55 765.87 825.05 63.76 137.35544 0 32 8 3 1 0 1 0 0 1 20132659200 102400 129645056000 POSIX 0
Finished: Tue Oct 23 15:45:05 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 8 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 8 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 20132659200 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread4/file -r
+ tee -a ./output/COUNT:1#NN:4#PPN:8#API:POSIX#T:102400.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Tue Oct 23 15:45:17 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 102400 -b 20132659200 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread4/file -r
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 15:45:17 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/indread4/file
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 32 (8 per node)
repetitions = 3
xfersize = 102400 bytes
blocksize = 18.75 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 5153960755200.
WARNING: Using actual aggregate bytes moved = 53493350400.
read 425.02 19660800 100.00 0.006524 120.03 0.005546 120.03 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 5153960755200.
WARNING: Using actual aggregate bytes moved = 58569728000.
read 465.37 19660800 100.00 0.005566 120.03 0.005577 120.03 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 5153960755200.
WARNING: Using actual aggregate bytes moved = 64302899200.
read 510.91 19660800 100.00 0.005641 120.02 0.011653 120.03 2
Max Read: 510.91 MiB/sec (535.73 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 510.91 425.02 467.10 35.08 120.02880 0 32 8 3 1 0 1 0 0 1 20132659200 102400 53493350400 POSIX 0
Finished: Tue Oct 23 15:51:18 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 8 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write.00000006
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12641613 0xc0e54d 0
3 12489916 0xbe94bc 0
5 12381980 0xbcef1c 0
6 12595234 0xc03022 0
0 12423624 0xbd91c8 0
2 12569970 0xbfcd72 0
4 12511707 0xbee9db 0
7 12460337 0xbe2131 0
/esfs/jtacquaviva/ioperf/file_write.00000023
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12595229 0xc0301d 0
0 12423619 0xbd91c3 0
2 12569965 0xbfcd6d 0
4 12511702 0xbee9d6 0
7 12460332 0xbe212c 0
1 12641608 0xc0e548 0
3 12489911 0xbe94b7 0
5 12381975 0xbcef17 0
/esfs/jtacquaviva/ioperf/file_write.00000002
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12423628 0xbd91cc 0
2 12569974 0xbfcd76 0
4 12511711 0xbee9df 0
7 12460341 0xbe2135 0
1 12641617 0xc0e551 0
3 12489920 0xbe94c0 0
5 12381984 0xbcef20 0
6 12595238 0xc03026 0
/esfs/jtacquaviva/ioperf/file_write.00000026
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12569967 0xbfcd6f 0
4 12511704 0xbee9d8 0
7 12460334 0xbe212e 0
1 12641610 0xc0e54a 0
3 12489913 0xbe94b9 0
5 12381977 0xbcef19 0
6 12595231 0xc0301f 0
0 12423621 0xbd91c5 0
/esfs/jtacquaviva/ioperf/file_write.00000000
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12460336 0xbe2130 0
1 12641612 0xc0e54c 0
3 12489915 0xbe94bb 0
5 12381979 0xbcef1b 0
6 12595233 0xc03021 0
0 12423623 0xbd91c7 0
2 12569969 0xbfcd71 0
4 12511706 0xbee9da 0
/esfs/jtacquaviva/ioperf/file_write.00000020
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12423603 0xbd91b3 0
2 12569949 0xbfcd5d 0
4 12511686 0xbee9c6 0
7 12460316 0xbe211c 0
1 12641592 0xc0e538 0
3 12489895 0xbe94a7 0
5 12381959 0xbcef07 0
6 12595213 0xc0300d 0
/esfs/jtacquaviva/ioperf/file_write.00000030
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12423620 0xbd91c4 0
2 12569966 0xbfcd6e 0
4 12511703 0xbee9d7 0
7 12460333 0xbe212d 0
1 12641609 0xc0e549 0
3 12489912 0xbe94b8 0
5 12381976 0xbcef18 0
6 12595230 0xc0301e 0
/esfs/jtacquaviva/ioperf/file_write.00000004
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12489893 0xbe94a5 0
0 12423601 0xbd91b1 0
4 12511684 0xbee9c4 0
1 12641590 0xc0e536 0
6 12595212 0xc0300c 0
2 12569948 0xbfcd5c 0
7 12460315 0xbe211b 0
5 12381958 0xbcef06 0
/esfs/jtacquaviva/ioperf/file_write.00000007
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12489917 0xbe94bd 0
5 12381981 0xbcef1d 0
6 12595235 0xc03023 0
0 12423625 0xbd91c9 0
2 12569971 0xbfcd73 0
4 12511708 0xbee9dc 0
7 12460338 0xbe2132 0
1 12641614 0xc0e54e 0
/esfs/jtacquaviva/ioperf/file_write.00000018
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12569959 0xbfcd67 0
4 12511696 0xbee9d0 0
7 12460326 0xbe2126 0
1 12641602 0xc0e542 0
3 12489905 0xbe94b1 0
5 12381969 0xbcef11 0
6 12595223 0xc03017 0
0 12423613 0xbd91bd 0
/esfs/jtacquaviva/ioperf/file_write.00000031
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12381974 0xbcef16 0
6 12595228 0xc0301c 0
0 12423618 0xbd91c2 0
2 12569964 0xbfcd6c 0
4 12511701 0xbee9d5 0
7 12460331 0xbe212b 0
1 12641607 0xc0e547 0
3 12489910 0xbe94b6 0
/esfs/jtacquaviva/ioperf/file_write.00000012
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12569943 0xbfcd57 0
4 12511680 0xbee9c0 0
7 12460310 0xbe2116 0
1 12641586 0xc0e532 0
3 12489889 0xbe94a1 0
5 12381953 0xbcef01 0
6 12595207 0xc03007 0
0 12423597 0xbd91ad 0
/esfs/jtacquaviva/ioperf/file_write.00000013
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12489899 0xbe94ab 0
0 12423607 0xbd91b7 0
4 12511690 0xbee9ca 0
1 12641596 0xc0e53c 0
5 12381963 0xbcef0b 0
2 12569954 0xbfcd62 0
7 12460321 0xbe2121 0
6 12595218 0xc03012 0
/esfs/jtacquaviva/ioperf/file_write.00000025
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12460318 0xbe211e 0
3 12489898 0xbe94aa 0
1 12641595 0xc0e53b 0
6 12595215 0xc0300f 0
2 12569951 0xbfcd5f 0
5 12381962 0xbcef0a 0
0 12423606 0xbd91b6 0
4 12511689 0xbee9c9 0
/esfs/jtacquaviva/ioperf/file_write.00000022
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12641605 0xc0e545 0
3 12489908 0xbe94b4 0
5 12381972 0xbcef14 0
6 12595226 0xc0301a 0
0 12423616 0xbd91c0 0
2 12569962 0xbfcd6a 0
4 12511699 0xbee9d3 0
7 12460329 0xbe2129 0
/esfs/jtacquaviva/ioperf/file_write.00000003
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12595237 0xc03025 0
0 12423627 0xbd91cb 0
2 12569973 0xbfcd75 0
4 12511710 0xbee9de 0
7 12460340 0xbe2134 0
1 12641616 0xc0e550 0
3 12489919 0xbe94bf 0
5 12381983 0xbcef1f 0
/esfs/jtacquaviva/ioperf/file_write.00000008
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12489909 0xbe94b5 0
5 12381973 0xbcef15 0
6 12595227 0xc0301b 0
0 12423617 0xbd91c1 0
2 12569963 0xbfcd6b 0
4 12511700 0xbee9d4 0
7 12460330 0xbe212a 0
1 12641606 0xc0e546 0
/esfs/jtacquaviva/ioperf/file_write.00000028
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12511705 0xbee9d9 0
7 12460335 0xbe212f 0
1 12641611 0xc0e54b 0
3 12489914 0xbe94ba 0
5 12381978 0xbcef1a 0
6 12595232 0xc03020 0
0 12423622 0xbd91c6 0
2 12569968 0xbfcd70 0
/esfs/jtacquaviva/ioperf/file_write.00000010
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12569950 0xbfcd5e 0
5 12381960 0xbcef08 0
6 12595214 0xc0300e 0
0 12423604 0xbd91b4 0
4 12511687 0xbee9c7 0
7 12460317 0xbe211d 0
3 12489897 0xbe94a9 0
1 12641593 0xc0e539 0
/esfs/jtacquaviva/ioperf/file_write.00000017
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12511681 0xbee9c1 0
7 12460311 0xbe2117 0
1 12641587 0xc0e533 0
3 12489890 0xbe94a2 0
5 12381954 0xbcef02 0
6 12595208 0xc03008 0
0 12423598 0xbd91ae 0
2 12569944 0xbfcd58 0
/esfs/jtacquaviva/ioperf/file_write.00000029
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12511697 0xbee9d1 0
7 12460327 0xbe2127 0
1 12641603 0xc0e543 0
3 12489906 0xbe94b2 0
5 12381970 0xbcef12 0
6 12595224 0xc03018 0
0 12423614 0xbd91be 0
2 12569960 0xbfcd68 0
/esfs/jtacquaviva/ioperf/file_write.00000016
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12595217 0xc03011 0
2 12569953 0xbfcd61 0
7 12460320 0xbe2120 0
3 12489900 0xbe94ac 0
0 12423608 0xbd91b8 0
4 12511691 0xbee9cb 0
1 12641597 0xc0e53d 0
5 12381965 0xbcef0d 0
/esfs/jtacquaviva/ioperf/file_write.00000005
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12489896 0xbe94a8 0
1 12641594 0xc0e53a 0
5 12381961 0xbcef09 0
0 12423605 0xbd91b5 0
4 12511688 0xbee9c8 0
6 12595216 0xc03010 0
2 12569952 0xbfcd60 0
7 12460319 0xbe211f 0
/esfs/jtacquaviva/ioperf/file_write.00000014
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12460328 0xbe2128 0
1 12641604 0xc0e544 0
3 12489907 0xbe94b3 0
5 12381971 0xbcef13 0
6 12595225 0xc03019 0
0 12423615 0xbd91bf 0
2 12569961 0xbfcd69 0
4 12511698 0xbee9d2 0
/esfs/jtacquaviva/ioperf/file_write.00000021
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12595220 0xc03014 0
0 12423610 0xbd91ba 0
2 12569956 0xbfcd64 0
4 12511693 0xbee9cd 0
7 12460323 0xbe2123 0
1 12641599 0xc0e53f 0
3 12489902 0xbe94ae 0
5 12381966 0xbcef0e 0
/esfs/jtacquaviva/ioperf/file_write.00000019
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12641589 0xc0e535 0
3 12489892 0xbe94a4 0
5 12381956 0xbcef04 0
6 12595210 0xc0300a 0
0 12423600 0xbd91b0 0
2 12569946 0xbfcd5a 0
4 12511683 0xbee9c3 0
7 12460313 0xbe2119 0
/esfs/jtacquaviva/ioperf/file_write.00000009
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12423612 0xbd91bc 0
2 12569958 0xbfcd66 0
4 12511695 0xbee9cf 0
7 12460325 0xbe2125 0
1 12641601 0xc0e541 0
3 12489904 0xbe94b0 0
5 12381968 0xbcef10 0
6 12595222 0xc03016 0
/esfs/jtacquaviva/ioperf/file_write.00000015
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12460312 0xbe2118 0
1 12641588 0xc0e534 0
3 12489891 0xbe94a3 0
5 12381955 0xbcef03 0
6 12595209 0xc03009 0
0 12423599 0xbd91af 0
2 12569945 0xbfcd59 0
4 12511682 0xbee9c2 0
/esfs/jtacquaviva/ioperf/file_write.00000027
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12595221 0xc03015 0
0 12423611 0xbd91bb 0
2 12569957 0xbfcd65 0
4 12511694 0xbee9ce 0
7 12460324 0xbe2124 0
1 12641600 0xc0e540 0
3 12489903 0xbe94af 0
5 12381967 0xbcef0f 0
/esfs/jtacquaviva/ioperf/file_write.00000024
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12381964 0xbcef0c 0
6 12595219 0xc03013 0
0 12423609 0xbd91b9 0
2 12569955 0xbfcd63 0
4 12511692 0xbee9cc 0
7 12460322 0xbe2122 0
1 12641598 0xc0e53e 0
3 12489901 0xbe94ad 0
/esfs/jtacquaviva/ioperf/file_write.00000001
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12381982 0xbcef1e 0
6 12595236 0xc03024 0
0 12423626 0xbd91ca 0
2 12569972 0xbfcd74 0
4 12511709 0xbee9dd 0
7 12460339 0xbe2133 0
1 12641615 0xc0e54f 0
3 12489918 0xbe94be 0
/esfs/jtacquaviva/ioperf/file_write.00000011
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12595211 0xc0300b 0
2 12569947 0xbfcd5b 0
7 12460314 0xbe211a 0
3 12489894 0xbe94a6 0
5 12381957 0xbcef05 0
0 12423602 0xbd91b2 0
4 12511685 0xbee9c5 0
1 12641591 0xc0e537 0

604
output/COUNT:1#NN:4#PPN:8#API:POSIX#T:1048576.txt Normal file
View File

@ -0,0 +1,604 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 8 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 20132659200 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:4#PPN:8#API:POSIX#T:1048576.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Tue Oct 23 13:32:48 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 20132659200 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 13:32:48 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 32 (8 per node)
repetitions = 3
xfersize = 1 MiB
blocksize = 18.75 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644245094400.
WARNING: Using actual aggregate bytes moved = 596751941632.
write 4509 19660800 1024.00 0.002501 126.20 0.000397 126.20 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644245094400.
WARNING: Using actual aggregate bytes moved = 637083320320.
write 4858 19660800 1024.00 0.004593 125.06 0.003708 125.07 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644245094400.
WARNING: Using actual aggregate bytes moved = 638399283200.
write 4957 19660800 1024.00 0.003583 122.82 0.000592 122.82 2
Max Write: 4956.96 MiB/sec (5197.75 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 4956.96 4509.40 4774.75 191.94 124.69848 0 32 8 3 1 0 1 0 0 1 20132659200 1048576 596751941632 POSIX 0
Finished: Tue Oct 23 13:39:07 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 8 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 8 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 20132659200 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread4/file -r
+ tee -a ./output/COUNT:1#NN:4#PPN:8#API:POSIX#T:1048576.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Tue Oct 23 13:39:21 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 1048576 -b 20132659200 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread4/file -r
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 13:39:21 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/indread4/file
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 32 (8 per node)
repetitions = 3
xfersize = 1 MiB
blocksize = 18.75 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 5153960755200.
WARNING: Using actual aggregate bytes moved = 336569827328.
read 2674.04 19660800 1024.00 0.001528 120.03 0.000366 120.03 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 5153960755200.
WARNING: Using actual aggregate bytes moved = 390937444352.
read 3106.42 19660800 1024.00 0.000512 120.02 0.000520 120.02 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 5153960755200.
WARNING: Using actual aggregate bytes moved = 400710172672.
read 3183.98 19660800 1024.00 0.000660 120.02 0.000799 120.02 2
Max Read: 3183.98 MiB/sec (3338.65 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 3183.98 2674.04 2988.15 224.35 120.02490 0 32 8 3 1 0 1 0 0 1 20132659200 1048576 336569827328 POSIX 0
Finished: Tue Oct 23 13:45:21 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 8 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write.00000006
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12381692 0xbcedfc 0
2 12569680 0xbfcc50 0
7 12460048 0xbe2010 0
1 12641327 0xc0e42f 0
6 12594947 0xc02f03 0
4 12511420 0xbee8bc 0
0 12423340 0xbd90ac 0
3 12489630 0xbe939e 0
/esfs/jtacquaviva/ioperf/file_write.00000023
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12569682 0xbfcc52 0
7 12460050 0xbe2012 0
3 12489631 0xbe939f 0
0 12423341 0xbd90ad 0
1 12641329 0xc0e431 0
4 12511422 0xbee8be 0
5 12381695 0xbcedff 0
6 12594949 0xc02f05 0
/esfs/jtacquaviva/ioperf/file_write.00000002
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12423343 0xbd90af 0
4 12511424 0xbee8c0 0
2 12569685 0xbfcc55 0
6 12594952 0xc02f08 0
7 12460054 0xbe2016 0
3 12489635 0xbe93a3 0
1 12641332 0xc0e434 0
5 12381699 0xbcee03 0
/esfs/jtacquaviva/ioperf/file_write.00000026
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12569690 0xbfcc5a 0
4 12511429 0xbee8c5 0
1 12641336 0xc0e438 0
5 12381702 0xbcee06 0
7 12460059 0xbe201b 0
0 12423350 0xbd90b6 0
3 12489640 0xbe93a8 0
6 12594957 0xc02f0d 0
/esfs/jtacquaviva/ioperf/file_write.00000000
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12594956 0xc02f0c 0
0 12423349 0xbd90b5 0
7 12460058 0xbe201a 0
3 12489639 0xbe93a7 0
2 12569691 0xbfcc5b 0
4 12511430 0xbee8c6 0
1 12641337 0xc0e439 0
5 12381703 0xbcee07 0
/esfs/jtacquaviva/ioperf/file_write.00000020
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12594955 0xc02f0b 0
0 12423348 0xbd90b4 0
2 12569689 0xbfcc59 0
4 12511428 0xbee8c4 0
7 12460057 0xbe2019 0
1 12641335 0xc0e437 0
3 12489638 0xbe93a6 0
5 12381701 0xbcee05 0
/esfs/jtacquaviva/ioperf/file_write.00000030
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12381688 0xbcedf8 0
0 12423335 0xbd90a7 0
4 12511415 0xbee8b7 0
1 12641323 0xc0e42b 0
6 12594943 0xc02eff 0
2 12569677 0xbfcc4d 0
7 12460045 0xbe200d 0
3 12489626 0xbe939a 0
/esfs/jtacquaviva/ioperf/file_write.00000004
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12641322 0xc0e42a 0
6 12594942 0xc02efe 0
2 12569676 0xbfcc4c 0
7 12460044 0xbe200c 0
3 12489625 0xbe9399 0
0 12423336 0xbd90a8 0
4 12511416 0xbee8b8 0
5 12381689 0xbcedf9 0
/esfs/jtacquaviva/ioperf/file_write.00000007
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12594968 0xc02f18 0
0 12423361 0xbd90c1 0
2 12569702 0xbfcc66 0
4 12511441 0xbee8d1 0
7 12460070 0xbe2026 0
1 12641348 0xc0e444 0
3 12489651 0xbe93b3 0
5 12381714 0xbcee12 0
/esfs/jtacquaviva/ioperf/file_write.00000018
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12381697 0xbcee01 0
4 12511425 0xbee8c1 0
2 12569686 0xbfcc56 0
6 12594953 0xc02f09 0
3 12489634 0xbe93a2 0
0 12423346 0xbd90b2 0
7 12460055 0xbe2017 0
1 12641333 0xc0e435 0
/esfs/jtacquaviva/ioperf/file_write.00000031
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12460066 0xbe2022 0
1 12641344 0xc0e440 0
3 12489647 0xbe93af 0
5 12381710 0xbcee0e 0
6 12594964 0xc02f14 0
0 12423357 0xbd90bd 0
2 12569698 0xbfcc62 0
4 12511437 0xbee8cd 0
/esfs/jtacquaviva/ioperf/file_write.00000012
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12460046 0xbe200e 0
6 12594944 0xc02f00 0
1 12641325 0xc0e42d 0
3 12489628 0xbe939c 0
0 12423338 0xbd90aa 0
2 12569679 0xbfcc4f 0
5 12381691 0xbcedfb 0
4 12511418 0xbee8ba 0
/esfs/jtacquaviva/ioperf/file_write.00000013
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12569692 0xbfcc5c 0
4 12511431 0xbee8c7 0
7 12460060 0xbe201c 0
1 12641338 0xc0e43a 0
3 12489641 0xbe93a9 0
5 12381704 0xbcee08 0
6 12594958 0xc02f0e 0
0 12423351 0xbd90b7 0
/esfs/jtacquaviva/ioperf/file_write.00000025
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12381696 0xbcee00 0
6 12594950 0xc02f06 0
1 12641331 0xc0e433 0
0 12423344 0xbd90b0 0
7 12460053 0xbe2015 0
2 12569687 0xbfcc57 0
3 12489636 0xbe93a4 0
4 12511426 0xbee8c2 0
/esfs/jtacquaviva/ioperf/file_write.00000022
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12511432 0xbee8c8 0
7 12460061 0xbe201d 0
1 12641339 0xc0e43b 0
3 12489642 0xbe93aa 0
5 12381705 0xbcee09 0
6 12594959 0xc02f0f 0
0 12423352 0xbd90b8 0
2 12569693 0xbfcc5d 0
/esfs/jtacquaviva/ioperf/file_write.00000003
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12423353 0xbd90b9 0
7 12460063 0xbe201f 0
2 12569695 0xbfcc5f 0
4 12511434 0xbee8ca 0
1 12641341 0xc0e43d 0
3 12489644 0xbe93ac 0
5 12381707 0xbcee0b 0
6 12594961 0xc02f11 0
/esfs/jtacquaviva/ioperf/file_write.00000008
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12489649 0xbe93b1 0
5 12381712 0xbcee10 0
6 12594966 0xc02f16 0
0 12423359 0xbd90bf 0
2 12569700 0xbfcc64 0
4 12511439 0xbee8cf 0
7 12460068 0xbe2024 0
1 12641346 0xc0e442 0
/esfs/jtacquaviva/ioperf/file_write.00000028
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12641326 0xc0e42e 0
6 12594946 0xc02f02 0
0 12423339 0xbd90ab 0
4 12511419 0xbee8bb 0
3 12489629 0xbe939d 0
2 12569681 0xbfcc51 0
7 12460049 0xbe2011 0
5 12381693 0xbcedfd 0
/esfs/jtacquaviva/ioperf/file_write.00000010
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12569684 0xbfcc54 0
4 12511423 0xbee8bf 0
7 12460052 0xbe2014 0
1 12641330 0xc0e432 0
3 12489633 0xbe93a1 0
5 12381698 0xbcee02 0
6 12594951 0xc02f07 0
0 12423345 0xbd90b1 0
/esfs/jtacquaviva/ioperf/file_write.00000017
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12511436 0xbee8cc 0
7 12460065 0xbe2021 0
1 12641343 0xc0e43f 0
3 12489646 0xbe93ae 0
5 12381709 0xbcee0d 0
6 12594963 0xc02f13 0
0 12423356 0xbd90bc 0
2 12569697 0xbfcc61 0
/esfs/jtacquaviva/ioperf/file_write.00000029
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12569704 0xbfcc68 0
4 12511443 0xbee8d3 0
7 12460072 0xbe2028 0
1 12641350 0xc0e446 0
3 12489653 0xbe93b5 0
5 12381716 0xbcee14 0
6 12594970 0xc02f1a 0
0 12423363 0xbd90c3 0
/esfs/jtacquaviva/ioperf/file_write.00000016
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12381713 0xbcee11 0
6 12594967 0xc02f17 0
0 12423360 0xbd90c0 0
2 12569701 0xbfcc65 0
4 12511440 0xbee8d0 0
7 12460069 0xbe2025 0
1 12641347 0xc0e443 0
3 12489650 0xbe93b2 0
/esfs/jtacquaviva/ioperf/file_write.00000005
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12423362 0xbd90c2 0
2 12569703 0xbfcc67 0
4 12511442 0xbee8d2 0
7 12460071 0xbe2027 0
1 12641349 0xc0e445 0
3 12489652 0xbe93b4 0
5 12381715 0xbcee13 0
6 12594969 0xc02f19 0
/esfs/jtacquaviva/ioperf/file_write.00000014
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12594948 0xc02f04 0
4 12511421 0xbee8bd 0
1 12641328 0xc0e430 0
5 12381694 0xbcedfe 0
2 12569683 0xbfcc53 0
7 12460051 0xbe2013 0
0 12423342 0xbd90ae 0
3 12489632 0xbe93a0 0
/esfs/jtacquaviva/ioperf/file_write.00000021
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12460043 0xbe200b 0
1 12641321 0xc0e429 0
3 12489624 0xbe9398 0
5 12381687 0xbcedf7 0
6 12594941 0xc02efd 0
0 12423334 0xbd90a6 0
2 12569675 0xbfcc4b 0
4 12511414 0xbee8b6 0
/esfs/jtacquaviva/ioperf/file_write.00000019
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12423337 0xbd90a9 0
2 12569678 0xbfcc4e 0
1 12641324 0xc0e42c 0
3 12489627 0xbe939b 0
5 12381690 0xbcedfa 0
4 12511417 0xbee8b9 0
7 12460047 0xbe200f 0
6 12594945 0xc02f01 0
/esfs/jtacquaviva/ioperf/file_write.00000009
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12511413 0xbee8b5 0
7 12460042 0xbe200a 0
1 12641320 0xc0e428 0
3 12489623 0xbe9397 0
5 12381686 0xbcedf6 0
6 12594940 0xc02efc 0
0 12423333 0xbd90a5 0
2 12569674 0xbfcc4a 0
/esfs/jtacquaviva/ioperf/file_write.00000015
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12381700 0xbcee04 0
6 12594954 0xc02f0a 0
0 12423347 0xbd90b3 0
2 12569688 0xbfcc58 0
4 12511427 0xbee8c3 0
7 12460056 0xbe2018 0
1 12641334 0xc0e436 0
3 12489637 0xbe93a5 0
/esfs/jtacquaviva/ioperf/file_write.00000027
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12511444 0xbee8d4 0
7 12460073 0xbe2029 0
1 12641351 0xc0e447 0
3 12489654 0xbe93b6 0
5 12381717 0xbcee15 0
6 12594971 0xc02f1b 0
0 12423364 0xbd90c4 0
2 12569705 0xbfcc69 0
/esfs/jtacquaviva/ioperf/file_write.00000024
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12460062 0xbe201e 0
1 12641340 0xc0e43c 0
3 12489643 0xbe93ab 0
5 12381706 0xbcee0a 0
2 12569694 0xbfcc5e 0
4 12511433 0xbee8c9 0
6 12594960 0xc02f10 0
0 12423354 0xbd90ba 0
/esfs/jtacquaviva/ioperf/file_write.00000001
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12641345 0xc0e441 0
3 12489648 0xbe93b0 0
5 12381711 0xbcee0f 0
6 12594965 0xc02f15 0
0 12423358 0xbd90be 0
2 12569699 0xbfcc63 0
4 12511438 0xbee8ce 0
7 12460067 0xbe2023 0
/esfs/jtacquaviva/ioperf/file_write.00000011
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12569696 0xbfcc60 0
4 12511435 0xbee8cb 0
7 12460064 0xbe2020 0
1 12641342 0xc0e43e 0
3 12489645 0xbe93ad 0
5 12381708 0xbcee0c 0
6 12594962 0xc02f12 0
0 12423355 0xbd90bb 0

594
output/COUNT:1#NN:4#PPN:8#API:POSIX#T:10485760.txt Normal file
View File

@ -0,0 +1,594 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 8 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 20132659200 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:4#PPN:8#API:POSIX#T:10485760.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Tue Oct 23 11:30:08 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 20132659200 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 11:30:08 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 32 (8 per node)
repetitions = 3
xfersize = 10 MiB
blocksize = 18.75 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
write 5671 19660800 10240 0.021417 108.32 0.000967 108.34 0
write 5638 19660800 10240 0.003743 108.98 0.000972 108.98 1
write 5656 19660800 10240 0.003467 108.62 0.001086 108.63 2
Max Write: 5670.84 MiB/sec (5946.31 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 5670.84 5637.75 5654.90 13.54 108.64968 0 32 8 3 1 0 1 0 0 1 20132659200 10485760 644245094400 POSIX 0
Finished: Tue Oct 23 11:35:39 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 8 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ tee -a ./output/COUNT:1#NN:4#PPN:8#API:POSIX#T:10485760.txt
+ /opt/ddn/mvapich/bin/mpiexec -ppn 8 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 20132659200 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread4/file -r
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Tue Oct 23 11:35:54 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 10485760 -b 20132659200 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread4/file -r
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 11:35:54 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/indread4/file
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 32 (8 per node)
repetitions = 3
xfersize = 10 MiB
blocksize = 18.75 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 5153960755200.
WARNING: Using actual aggregate bytes moved = 347173027840.
read 2755.72 19660800 10240 0.011325 120.14 0.010177 120.15 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 5153960755200.
WARNING: Using actual aggregate bytes moved = 449964933120.
read 3571.20 19660800 10240 0.010228 120.15 0.010208 120.16 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 5153960755200.
WARNING: Using actual aggregate bytes moved = 440234147840.
read 3485.20 19660800 10240 0.010245 120.46 0.010226 120.46 2
Max Read: 3571.20 MiB/sec (3744.68 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 3571.20 2755.72 3270.71 365.84 120.25715 0 32 8 3 1 0 1 0 0 1 20132659200 10485760 347173027840 POSIX 0
Finished: Tue Oct 23 11:41:55 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 8 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write.00000006
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12594681 0xc02df9 0
4 12511150 0xbee7ae 0
7 12459782 0xbe1f06 0
3 12489361 0xbe9291 0
0 12423074 0xbd8fa2 0
1 12641061 0xc0e325 0
5 12381424 0xbcecf0 0
2 12569415 0xbfcb47 0
/esfs/jtacquaviva/ioperf/file_write.00000023
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12569428 0xbfcb54 0
1 12641074 0xc0e332 0
6 12594695 0xc02e07 0
4 12511164 0xbee7bc 0
3 12489376 0xbe92a0 0
0 12423089 0xbd8fb1 0
7 12459797 0xbe1f15 0
5 12381438 0xbcecfe 0
/esfs/jtacquaviva/ioperf/file_write.00000002
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12511170 0xbee7c2 0
7 12459802 0xbe1f1a 0
5 12381443 0xbced03 0
6 12594701 0xc02e0d 0
0 12423094 0xbd8fb6 0
1 12641080 0xc0e338 0
3 12489382 0xbe92a6 0
2 12569434 0xbfcb5a 0
/esfs/jtacquaviva/ioperf/file_write.00000026
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12594693 0xc02e05 0
4 12511162 0xbee7ba 0
1 12641072 0xc0e330 0
3 12489373 0xbe929d 0
2 12569427 0xbfcb53 0
0 12423087 0xbd8faf 0
7 12459795 0xbe1f13 0
5 12381436 0xbcecfc 0
/esfs/jtacquaviva/ioperf/file_write.00000000
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12423098 0xbd8fba 0
2 12569437 0xbfcb5d 0
4 12511174 0xbee7c6 0
7 12459806 0xbe1f1e 0
1 12641084 0xc0e33c 0
3 12489385 0xbe92a9 0
5 12381447 0xbced07 0
6 12594705 0xc02e11 0
/esfs/jtacquaviva/ioperf/file_write.00000020
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12459800 0xbe1f18 0
5 12381441 0xbced01 0
6 12594699 0xc02e0b 0
0 12423092 0xbd8fb4 0
4 12511168 0xbee7c0 0
3 12489380 0xbe92a4 0
1 12641079 0xc0e337 0
2 12569432 0xbfcb58 0
/esfs/jtacquaviva/ioperf/file_write.00000030
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12569410 0xbfcb42 0
1 12641056 0xc0e320 0
6 12594678 0xc02df6 0
4 12511147 0xbee7ab 0
3 12489358 0xbe928e 0
7 12459779 0xbe1f03 0
5 12381420 0xbcecec 0
0 12423071 0xbd8f9f 0
/esfs/jtacquaviva/ioperf/file_write.00000004
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12511167 0xbee7bf 0
7 12459799 0xbe1f17 0
1 12641077 0xc0e335 0
3 12489378 0xbe92a2 0
5 12381440 0xbced00 0
6 12594698 0xc02e0a 0
0 12423091 0xbd8fb3 0
2 12569430 0xbfcb56 0
/esfs/jtacquaviva/ioperf/file_write.00000007
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12489365 0xbe9295 0
0 12423078 0xbd8fa6 0
4 12511154 0xbee7b2 0
5 12381427 0xbcecf3 0
2 12569419 0xbfcb4b 0
7 12459787 0xbe1f0b 0
1 12641065 0xc0e329 0
6 12594686 0xc02dfe 0
/esfs/jtacquaviva/ioperf/file_write.00000018
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12381445 0xbced05 0
6 12594703 0xc02e0f 0
0 12423096 0xbd8fb8 0
2 12569435 0xbfcb5b 0
4 12511172 0xbee7c4 0
7 12459804 0xbe1f1c 0
1 12641082 0xc0e33a 0
3 12489383 0xbe92a7 0
/esfs/jtacquaviva/ioperf/file_write.00000031
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12489381 0xbe92a5 0
2 12569433 0xbfcb59 0
5 12381444 0xbced04 0
6 12594702 0xc02e0e 0
4 12511171 0xbee7c3 0
7 12459803 0xbe1f1b 0
0 12423095 0xbd8fb7 0
1 12641081 0xc0e339 0
/esfs/jtacquaviva/ioperf/file_write.00000012
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12489355 0xbe928b 0
5 12381417 0xbcece9 0
6 12594675 0xc02df3 0
0 12423068 0xbd8f9c 0
2 12569408 0xbfcb40 0
4 12511144 0xbee7a8 0
7 12459776 0xbe1f00 0
1 12641054 0xc0e31e 0
/esfs/jtacquaviva/ioperf/file_write.00000013
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12641070 0xc0e32e 0
6 12594691 0xc02e03 0
0 12423084 0xbd8fac 0
4 12511160 0xbee7b8 0
3 12489371 0xbe929b 0
2 12569425 0xbfcb51 0
7 12459793 0xbe1f11 0
5 12381434 0xbcecfa 0
/esfs/jtacquaviva/ioperf/file_write.00000025
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12569438 0xbfcb5e 0
4 12511175 0xbee7c7 0
7 12459807 0xbe1f1f 0
1 12641085 0xc0e33d 0
3 12489386 0xbe92aa 0
5 12381448 0xbced08 0
6 12594706 0xc02e12 0
0 12423099 0xbd8fbb 0
/esfs/jtacquaviva/ioperf/file_write.00000022
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12511148 0xbee7ac 0
3 12489359 0xbe928f 0
6 12594679 0xc02df7 0
2 12569412 0xbfcb44 0
1 12641059 0xc0e323 0
5 12381422 0xbcecee 0
7 12459781 0xbe1f05 0
0 12423073 0xbd8fa1 0
/esfs/jtacquaviva/ioperf/file_write.00000003
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12594677 0xc02df5 0
4 12511146 0xbee7aa 0
7 12459778 0xbe1f02 0
3 12489357 0xbe928d 0
5 12381419 0xbceceb 0
0 12423070 0xbd8f9e 0
1 12641057 0xc0e321 0
2 12569411 0xbfcb43 0
/esfs/jtacquaviva/ioperf/file_write.00000008
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12594704 0xc02e10 0
0 12423097 0xbd8fb9 0
2 12569436 0xbfcb5c 0
4 12511173 0xbee7c5 0
7 12459805 0xbe1f1d 0
1 12641083 0xc0e33b 0
3 12489384 0xbe92a8 0
5 12381446 0xbced06 0
/esfs/jtacquaviva/ioperf/file_write.00000028
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12569429 0xbfcb55 0
4 12511166 0xbee7be 0
7 12459798 0xbe1f16 0
1 12641076 0xc0e334 0
3 12489377 0xbe92a1 0
5 12381439 0xbcecff 0
6 12594697 0xc02e09 0
0 12423090 0xbd8fb2 0
/esfs/jtacquaviva/ioperf/file_write.00000010
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12641058 0xc0e322 0
5 12381421 0xbceced 0
0 12423072 0xbd8fa0 0
7 12459780 0xbe1f04 0
6 12594680 0xc02df8 0
4 12511149 0xbee7ad 0
2 12569413 0xbfcb45 0
3 12489360 0xbe9290 0
/esfs/jtacquaviva/ioperf/file_write.00000017
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12569420 0xbfcb4c 0
4 12511156 0xbee7b4 0
7 12459788 0xbe1f0c 0
1 12641066 0xc0e32a 0
3 12489367 0xbe9297 0
5 12381429 0xbcecf5 0
6 12594687 0xc02dff 0
0 12423080 0xbd8fa8 0
/esfs/jtacquaviva/ioperf/file_write.00000029
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12594685 0xc02dfd 0
2 12569418 0xbfcb4a 0
7 12459786 0xbe1f0a 0
1 12641064 0xc0e328 0
0 12423079 0xbd8fa7 0
4 12511155 0xbee7b3 0
3 12489366 0xbe9296 0
5 12381428 0xbcecf4 0
/esfs/jtacquaviva/ioperf/file_write.00000016
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12641062 0xc0e326 0
6 12594683 0xc02dfb 0
2 12569416 0xbfcb48 0
3 12489364 0xbe9294 0
0 12423077 0xbd8fa5 0
5 12381426 0xbcecf2 0
4 12511153 0xbee7b1 0
7 12459785 0xbe1f09 0
/esfs/jtacquaviva/ioperf/file_write.00000005
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12641068 0xc0e32c 0
5 12381431 0xbcecf7 0
2 12569422 0xbfcb4e 0
7 12459790 0xbe1f0e 0
6 12594690 0xc02e02 0
3 12489370 0xbe929a 0
0 12423083 0xbd8fab 0
4 12511159 0xbee7b7 0
/esfs/jtacquaviva/ioperf/file_write.00000014
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12569421 0xbfcb4d 0
4 12511157 0xbee7b5 0
7 12459789 0xbe1f0d 0
1 12641067 0xc0e32b 0
3 12489368 0xbe9298 0
5 12381430 0xbcecf6 0
6 12594688 0xc02e00 0
0 12423081 0xbd8fa9 0
/esfs/jtacquaviva/ioperf/file_write.00000021
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12381418 0xbcecea 0
6 12594676 0xc02df4 0
0 12423069 0xbd8f9d 0
2 12569409 0xbfcb41 0
4 12511145 0xbee7a9 0
7 12459777 0xbe1f01 0
1 12641055 0xc0e31f 0
3 12489356 0xbe928c 0
/esfs/jtacquaviva/ioperf/file_write.00000019
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12381433 0xbcecf9 0
2 12569424 0xbfcb50 0
7 12459792 0xbe1f10 0
6 12594692 0xc02e04 0
0 12423085 0xbd8fad 0
4 12511161 0xbee7b9 0
1 12641071 0xc0e32f 0
3 12489372 0xbe929c 0
/esfs/jtacquaviva/ioperf/file_write.00000009
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12459796 0xbe1f14 0
3 12489375 0xbe929f 0
5 12381437 0xbcecfd 0
0 12423088 0xbd8fb0 0
1 12641075 0xc0e333 0
6 12594696 0xc02e08 0
4 12511165 0xbee7bd 0
/esfs/jtacquaviva/ioperf/file_write.00000015
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12489379 0xbe92a3 0
2 12569431 0xbfcb57 0
1 12641078 0xc0e336 0
5 12381442 0xbced02 0
4 12511169 0xbee7c1 0
7 12459801 0xbe1f19 0
6 12594700 0xc02e0c 0
0 12423093 0xbd8fb5 0
/esfs/jtacquaviva/ioperf/file_write.00000027
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12569414 0xbfcb46 0
1 12641060 0xc0e324 0
5 12381423 0xbcecef 0
4 12511151 0xbee7af 0
3 12489362 0xbe9292 0
7 12459783 0xbe1f07 0
6 12594682 0xc02dfa 0
0 12423075 0xbd8fa3 0
/esfs/jtacquaviva/ioperf/file_write.00000024
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12511158 0xbee7b6 0
3 12489369 0xbe9299 0
6 12594689 0xc02e01 0
0 12423082 0xbd8faa 0
1 12641069 0xc0e32d 0
7 12459791 0xbe1f0f 0
5 12381432 0xbcecf8 0
2 12569423 0xbfcb4f 0
/esfs/jtacquaviva/ioperf/file_write.00000001
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12569426 0xbfcb52 0
7 12459794 0xbe1f12 0
5 12381435 0xbcecfb 0
0 12423086 0xbd8fae 0
1 12641073 0xc0e331 0
4 12511163 0xbee7bb 0
3 12489374 0xbe929e 0
6 12594694 0xc02e06 0
/esfs/jtacquaviva/ioperf/file_write.00000011
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12511152 0xbee7b0 0
3 12489363 0xbe9293 0
5 12381425 0xbcecf1 0
0 12423076 0xbd8fa4 0
7 12459784 0xbe1f08 0
6 12594684 0xc02dfc 0
2 12569417 0xbfcb49 0
1 12641063 0xc0e327 0

602
output/COUNT:1#NN:4#PPN:8#API:POSIX#T:16384.txt Normal file
View File

@ -0,0 +1,602 @@
+ /opt/ddn/mvapich/bin/mpiexec -ppn 8 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 20132659200 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
+ tee -a ./output/COUNT:1#NN:4#PPN:8#API:POSIX#T:16384.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Tue Oct 23 17:45:18 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 20132659200 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/ioperf/file_write -w
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 17:45:18 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/ioperf/file_write
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 32 (8 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 18.75 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644218880000.
WARNING: Using actual aggregate bytes moved = 45198721024.
write 290.06 19660800 16.00 0.002192 148.60 0.000331 148.60 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644233822208.
WARNING: Using actual aggregate bytes moved = 36566794240.
write 236.11 19660800 16.00 0.016415 147.68 0.000472 147.70 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 644232511488.
WARNING: Using actual aggregate bytes moved = 37753257984.
write 243.22 19660800 16.00 0.002755 148.03 0.000515 148.03 2
Max Write: 290.06 MiB/sec (304.15 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write 290.06 236.11 256.47 23.93 148.11075 0 32 8 3 1 0 1 0 0 1 20132659200 16384 45198721024 POSIX 0
Finished: Tue Oct 23 17:52:44 2018
+ /opt/ddn/mvapich/bin/mpiexec -ppn 8 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/git/ime-evaluation/drop_caches.sh
+ /opt/ddn/mvapich/bin/mpiexec -ppn 8 -np 32 -genv MV2_NUM_HCAS 1 -genv MV2_CPU_BINDING_LEVEL core -genv MV2_CPU_BINDING_POLICY scatter --hosts isc17-c04,isc17-c05,isc17-c06,isc17-c07 /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 20132659200 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread4/file -r
+ tee -a ./output/COUNT:1#NN:4#PPN:8#API:POSIX#T:16384.txt
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Tue Oct 23 17:52:52 2018
Command line used: /esfs/jtacquaviva/software/install/ior/git-ddn/bin/ior -i 3 -s 1 -t 16384 -b 20132659200 -D 120 -a POSIX -F -e -g -z -k -o /esfs/jtacquaviva/indread4/file -r
Machine: Linux isc17-c04
Test 0 started: Tue Oct 23 17:52:52 2018
Summary:
api = POSIX
test filename = /esfs/jtacquaviva/indread4/file
access = file-per-process
ordering in a file = random offsets
ordering inter file= no tasks offsets
clients = 32 (8 per node)
repetitions = 3
xfersize = 16384 bytes
blocksize = 18.75 GiB
aggregate filesize = 600 GiB
Using stonewalling = 120 second(s)
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
------ --------- ---------- --------- -------- -------- -------- -------- ----
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 5153960755200.
WARNING: Using actual aggregate bytes moved = 10084745216.
read 80.09 19660800 16.00 0.004060 120.09 0.002986 120.09 0
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 5153960755200.
WARNING: Using actual aggregate bytes moved = 11453759488.
read 90.96 19660800 16.00 0.003118 120.09 0.003032 120.09 1
WARNING: Expected aggregate file size = 644245094400.
WARNING: Stat() of aggregate file size = 5153960755200.
WARNING: Using actual aggregate bytes moved = 11651366912.
read 92.53 19660800 16.00 0.003079 120.08 0.007151 120.09 2
Max Read: 92.53 MiB/sec (97.02 MB/sec)
Summary of all tests:
Operation Max(MiB) Min(MiB) Mean(MiB) StdDev Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
read 92.53 80.09 87.86 5.53 120.08946 0 32 8 3 1 0 1 0 0 1 20132659200 16384 10084745216 POSIX 0
Finished: Tue Oct 23 17:58:53 2018
+ set +x
/esfs/jtacquaviva/ioperf
stripe_count: 8 stripe_size: 1048576 stripe_offset: -1
/esfs/jtacquaviva/ioperf/file_write.00000006
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12460606 0xbe223e 0
1 12641883 0xc0e65b 0
3 12490184 0xbe95c8 0
5 12382248 0xbcf028 0
6 12595504 0xc03130 0
0 12423895 0xbd92d7 0
2 12570239 0xbfce7f 0
4 12511977 0xbeeae9 0
/esfs/jtacquaviva/ioperf/file_write.00000023
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12490176 0xbe95c0 0
5 12382240 0xbcf020 0
6 12595496 0xc03128 0
0 12423887 0xbd92cf 0
2 12570231 0xbfce77 0
4 12511969 0xbeeae1 0
7 12460598 0xbe2236 0
1 12641875 0xc0e653 0
/esfs/jtacquaviva/ioperf/file_write.00000002
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12382242 0xbcf022 0
6 12595499 0xc0312b 0
0 12423890 0xbd92d2 0
2 12570234 0xbfce7a 0
4 12511972 0xbeeae4 0
7 12460601 0xbe2239 0
1 12641878 0xc0e656 0
3 12490179 0xbe95c3 0
/esfs/jtacquaviva/ioperf/file_write.00000026
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12641884 0xc0e65c 0
3 12490185 0xbe95c9 0
5 12382249 0xbcf029 0
6 12595505 0xc03131 0
0 12423896 0xbd92d8 0
2 12570240 0xbfce80 0
4 12511978 0xbeeaea 0
7 12460607 0xbe223f 0
/esfs/jtacquaviva/ioperf/file_write.00000000
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12460596 0xbe2234 0
1 12641873 0xc0e651 0
3 12490174 0xbe95be 0
5 12382238 0xbcf01e 0
6 12595494 0xc03126 0
0 12423885 0xbd92cd 0
2 12570229 0xbfce75 0
4 12511967 0xbeeadf 0
/esfs/jtacquaviva/ioperf/file_write.00000020
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12511965 0xbeeadd 0
7 12460594 0xbe2232 0
1 12641871 0xc0e64f 0
3 12490172 0xbe95bc 0
5 12382236 0xbcf01c 0
6 12595492 0xc03124 0
0 12423883 0xbd92cb 0
2 12570227 0xbfce73 0
/esfs/jtacquaviva/ioperf/file_write.00000030
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12570211 0xbfce63 0
1 12641855 0xc0e63f 0
6 12595478 0xc03116 0
4 12511949 0xbeeacd 0
0 12423869 0xbd92bd 0
7 12460580 0xbe2224 0
3 12490160 0xbe95b0 0
5 12382223 0xbcf00f 0
/esfs/jtacquaviva/ioperf/file_write.00000004
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12570237 0xbfce7d 0
4 12511975 0xbeeae7 0
7 12460604 0xbe223c 0
1 12641881 0xc0e659 0
3 12490182 0xbe95c6 0
5 12382246 0xbcf026 0
6 12595502 0xc0312e 0
0 12423893 0xbd92d5 0
/esfs/jtacquaviva/ioperf/file_write.00000007
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12641859 0xc0e643 0
6 12595481 0xc03119 0
4 12511953 0xbeead1 0
5 12382224 0xbcf010 0
7 12460582 0xbe2226 0
0 12423871 0xbd92bf 0
2 12570215 0xbfce67 0
/esfs/jtacquaviva/ioperf/file_write.00000018
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12423880 0xbd92c8 0
2 12570224 0xbfce70 0
4 12511962 0xbeeada 0
7 12460591 0xbe222f 0
1 12641869 0xc0e64d 0
3 12490170 0xbe95ba 0
5 12382234 0xbcf01a 0
6 12595490 0xc03122 0
/esfs/jtacquaviva/ioperf/file_write.00000031
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12641866 0xc0e64a 0
3 12490167 0xbe95b7 0
5 12382231 0xbcf017 0
6 12595487 0xc0311f 0
0 12423878 0xbd92c6 0
2 12570222 0xbfce6e 0
4 12511960 0xbeead8 0
7 12460589 0xbe222d 0
/esfs/jtacquaviva/ioperf/file_write.00000012
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12460579 0xbe2223 0
3 12490158 0xbe95ae 0
1 12641858 0xc0e642 0
5 12382222 0xbcf00e 0
6 12595480 0xc03118 0
0 12423870 0xbd92be 0
2 12570214 0xbfce66 0
4 12511952 0xbeead0 0
/esfs/jtacquaviva/ioperf/file_write.00000013
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12382241 0xbcf021 0
6 12595497 0xc03129 0
0 12423888 0xbd92d0 0
2 12570232 0xbfce78 0
4 12511970 0xbeeae2 0
7 12460599 0xbe2237 0
1 12641876 0xc0e654 0
3 12490177 0xbe95c1 0
/esfs/jtacquaviva/ioperf/file_write.00000025
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 1
obdidx objid objid group
1 12641874 0xc0e652 0
3 12490175 0xbe95bf 0
5 12382239 0xbcf01f 0
6 12595495 0xc03127 0
0 12423886 0xbd92ce 0
2 12570230 0xbfce76 0
4 12511968 0xbeeae0 0
7 12460597 0xbe2235 0
/esfs/jtacquaviva/ioperf/file_write.00000022
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12382220 0xbcf00c 0
6 12595477 0xc03115 0
0 12423867 0xbd92bb 0
2 12570212 0xbfce64 0
1 12641857 0xc0e641 0
3 12490159 0xbe95af 0
4 12511950 0xbeeace 0
7 12460581 0xbe2225 0
/esfs/jtacquaviva/ioperf/file_write.00000003
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12460578 0xbe2222 0
3 12490157 0xbe95ad 0
5 12382221 0xbcf00d 0
0 12423868 0xbd92bc 0
1 12641856 0xc0e640 0
6 12595479 0xc03117 0
2 12570213 0xbfce65 0
4 12511951 0xbeeacf 0
/esfs/jtacquaviva/ioperf/file_write.00000008
lmm_stripe_count: 7
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12382225 0xbcf011 0
7 12460583 0xbe2227 0
2 12570217 0xbfce69 0
3 12490162 0xbe95b2 0
4 12511955 0xbeead3 0
0 12423873 0xbd92c1 0
1 12641861 0xc0e645 0
/esfs/jtacquaviva/ioperf/file_write.00000028
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12595498 0xc0312a 0
0 12423889 0xbd92d1 0
2 12570233 0xbfce79 0
4 12511971 0xbeeae3 0
7 12460600 0xbe2238 0
1 12641877 0xc0e655 0
3 12490178 0xbe95c2 0
5 12382243 0xbcf023 0
/esfs/jtacquaviva/ioperf/file_write.00000010
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12511958 0xbeead6 0
7 12460587 0xbe222b 0
1 12641864 0xc0e648 0
3 12490165 0xbe95b5 0
5 12382229 0xbcf015 0
6 12595485 0xc0311d 0
0 12423876 0xbd92c4 0
2 12570220 0xbfce6c 0
/esfs/jtacquaviva/ioperf/file_write.00000017
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12423872 0xbd92c0 0
7 12460584 0xbe2228 0
3 12490163 0xbe95b3 0
5 12382226 0xbcf012 0
6 12595482 0xc0311a 0
1 12641862 0xc0e646 0
4 12511956 0xbeead4 0
2 12570218 0xbfce6a 0
/esfs/jtacquaviva/ioperf/file_write.00000029
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12570226 0xbfce72 0
4 12511964 0xbeeadc 0
7 12460593 0xbe2231 0
1 12641870 0xc0e64e 0
3 12490171 0xbe95bb 0
5 12382235 0xbcf01b 0
6 12595491 0xc03123 0
0 12423882 0xbd92ca 0
/esfs/jtacquaviva/ioperf/file_write.00000016
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12595500 0xc0312c 0
0 12423891 0xbd92d3 0
2 12570235 0xbfce7b 0
4 12511973 0xbeeae5 0
7 12460602 0xbe223a 0
1 12641879 0xc0e657 0
3 12490180 0xbe95c4 0
5 12382244 0xbcf024 0
/esfs/jtacquaviva/ioperf/file_write.00000005
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12511957 0xbeead5 0
7 12460586 0xbe222a 0
1 12641863 0xc0e647 0
3 12490164 0xbe95b4 0
5 12382228 0xbcf014 0
6 12595484 0xc0311c 0
0 12423875 0xbd92c3 0
2 12570219 0xbfce6b 0
/esfs/jtacquaviva/ioperf/file_write.00000014
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12423892 0xbd92d4 0
2 12570236 0xbfce7c 0
4 12511974 0xbeeae6 0
7 12460603 0xbe223b 0
1 12641880 0xc0e658 0
3 12490181 0xbe95c5 0
5 12382245 0xbcf025 0
6 12595501 0xc0312d 0
/esfs/jtacquaviva/ioperf/file_write.00000021
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 0
obdidx objid objid group
0 12423865 0xbd92b9 0
4 12511947 0xbeeacb 0
3 12490155 0xbe95ab 0
1 12641854 0xc0e63e 0
7 12460577 0xbe2221 0
5 12382219 0xbcf00b 0
2 12570210 0xbfce62 0
6 12595476 0xc03114 0
/esfs/jtacquaviva/ioperf/file_write.00000019
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 7
obdidx objid objid group
7 12460588 0xbe222c 0
1 12641865 0xc0e649 0
3 12490166 0xbe95b6 0
5 12382230 0xbcf016 0
6 12595486 0xc0311e 0
0 12423877 0xbd92c5 0
2 12570221 0xbfce6d 0
4 12511959 0xbeead7 0
/esfs/jtacquaviva/ioperf/file_write.00000009
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 5
obdidx objid objid group
5 12382218 0xbcf00a 0
2 12570209 0xbfce61 0
7 12460576 0xbe2220 0
1 12641853 0xc0e63d 0
6 12595475 0xc03113 0
0 12423866 0xbd92ba 0
3 12490156 0xbe95ac 0
4 12511948 0xbeeacc 0
/esfs/jtacquaviva/ioperf/file_write.00000015
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 6
obdidx objid objid group
6 12595488 0xc03120 0
7 12460590 0xbe222e 0
3 12490169 0xbe95b9 0
0 12423881 0xbd92c9 0
2 12570225 0xbfce71 0
1 12641868 0xc0e64c 0
4 12511963 0xbeeadb 0
5 12382233 0xbcf019 0
/esfs/jtacquaviva/ioperf/file_write.00000027
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12511976 0xbeeae8 0
7 12460605 0xbe223d 0
1 12641882 0xc0e65a 0
3 12490183 0xbe95c7 0
5 12382247 0xbcf027 0
6 12595503 0xc0312f 0
0 12423894 0xbd92d6 0
2 12570238 0xbfce7e 0
/esfs/jtacquaviva/ioperf/file_write.00000024
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 2
obdidx objid objid group
2 12570216 0xbfce68 0
3 12490161 0xbe95b1 0
4 12511954 0xbeead2 0
1 12641860 0xc0e644 0
0 12423874 0xbd92c2 0
7 12460585 0xbe2229 0
5 12382227 0xbcf013 0
6 12595483 0xc0311b 0
/esfs/jtacquaviva/ioperf/file_write.00000001
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 3
obdidx objid objid group
3 12490168 0xbe95b8 0
0 12423879 0xbd92c7 0
2 12570223 0xbfce6f 0
4 12511961 0xbeead9 0
1 12641867 0xc0e64b 0
5 12382232 0xbcf018 0
6 12595489 0xc03121 0
7 12460592 0xbe2230 0
/esfs/jtacquaviva/ioperf/file_write.00000011
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: 1
lmm_layout_gen: 0
lmm_stripe_offset: 4
obdidx objid objid group
4 12511966 0xbeeade 0
7 12460595 0xbe2233 0
1 12641872 0xc0e650 0
3 12490173 0xbe95bd 0
5 12382237 0xbcf01d 0
6 12595493 0xc03125 0
0 12423884 0xbd92cc 0
2 12570228 0xbfce74 0