fio

最後更新: 2019-10-18

 

介紹

HomePage: https://github.com/axboe/fio

支持許多種不同的I/O引擎, 包括: mmap, sync ... libaio, rdma, posixaio ...

( 列出 engine: fio --enghelp )

(--enghelp=[ioengine[,command]])

engine

sync

Basic read(2) or write(2) I/O.

lseek(2) is used to position  the  I/O location.  

See fsync and fdata-sync for syncing write I/Os.

libaio

Linux native asynchronous I/O.

Note that Linux may only support queued behavior with non-buffered I/O
(set `direct=1' or `buffered=0')

(buffered=bool    # Default: 1; use buffered I/O. This is the opposite of the direct option.)

posixaio

POSIX  asynchronous  I/O  using  aio_read(3) and aio_write(3).

mmap  

File is memory mapped with mmap(2) and data copied to/from using memcpy(3)


benchmarking using fio

 

Install

Linux

apt-get install fio

Windows

https://ci.appveyor.com/project/axboe/fio

Job -> Artifacts

準備

  • echo mq-deadline > /sys/block/sdb/queue/scheduler
  • echo mq-deadline > /sys/block/sdc/queue/scheduler
  • echo 1 > /proc/sys/vm/dirty_background_ratio
  • echo 50 > /proc/sys/vm/dirty_ratio
  • echo 500 > /proc/sys/vm/dirty_expire_centisecs
  • /sbin/blockdev --setra 256 /dev/sdb

設定

direct=1              # Default: 0; use non-buffered I/O. This is usually O_DIRECT.

readwrite=str       # read, write, randread, randwrite, randrw (--rwmixread=50)

loops=int             # Run  the specified number of iterations of this job.

iodepth=int          # Default: 1; Number of I/O units to keep in flight against  the file.

                           # iodepth beyond 1 will not affect synchronous ioengines

                           # Even async engines may impose OS restrictions

                           # Keep an eye on the I/O depth distribution in the fio output to verify that the achieved depth is as expected.

bs=int                 # block  size. Default: 4096.

size=int               # The total size of file I/O for each thread of this job.

                           # Fio will run  until  this many bytes has been transferred,

                           # unless runtime is limited by other options (runtime)

runtime=sec         # Tell  fio  to terminate processing after the specified period of  time.

numjobs=int         # Default: 1; Create the specified number of clones of this job.

                           # Each clone of job  is spawned as an independent thread or process.

group_reporting    # groups of jobs as a whole instead of for each individual job

iodepth                 # If the I/O engine is async, how large a queuing depth do we want to maintain

Performing a Random Read Test

fio --name=MyTest --ioengine=libaio --direct=1 --loops=3 \
--readwrite=randread --iodepth=16 --bs=4k --size=512M \
--numjobs=4 --runtime=240 --group_reporting

P.S.

會在當前目錄建立 MyTest.0.0  MyTest.1.0  MyTest.2.0  MyTest.3.0, 每個 512M

filename=str

Fio normally makes up a filename based on the job name, thread number

If you want  to share  files between threads in a job or several jobs with fixed file paths

you can specify a number of files by separating the names with a ':'

The size of regular files specified by this  option  will be  "size"  divided  by number of files

Read Write Performance Test

fio --name=MyTest --ioengine=libaio --direct=1 --loops=3 \
--readwrite=randwrite --iodepth=64 --bs=4k  --size=1G \
--numjobs=4 --runtime=240 --group_reporting \
--filename=test.fio

Output

過程

fio-3.12
Starting 4 processes
Jobs: 4 (f=4): [W(4)][44.6%][w=9537KiB/s][w=2384 IOPS][eta 02m:13s]

Jobs: 4

number of threads currently running and doing I/O

(f=?)

the  number  of  currently open  files

[W(4)]: current status of each thread

W      Running, doing sequential writes.

w      Running, doing random writes.

[44.6%]

the estimated completion percentage

[w=9.81MiB/s]

last check bandwidth

[w=2512 IOPS]

last check IOPS

[eta 02m:13s]

time to completion for the current running  group

(1298kB/s)(297MiB/240017msec)

(total I/O performed / runtime of that thread)

randread test

MyTest: (groupid=0, jobs=4): err= 0: pid=16939: Thu Oct 17 14:56:54 2019
  read: IOPS=316, BW=1268KiB/s (1298kB/s)(297MiB/240017msec)
    slat (usec): min=4, max=662603, avg=12611.44, stdev=9067.83
    clat (msec): min=9, max=1093, avg=189.23, stdev=51.90
     lat (msec): min=19, max=1128, avg=201.84, stdev=54.38
....

說明

slat   Submission latency

  This is the time it took to submit  the  I/O.

  For sync I/O this row is not displayed as the slat is really the completion latency

  (since queue/complete is  one operation  there)

clat   Completion latency

  time from submission to completion of the I/O pieces.

  For sync  I/O,  clat will usually be equal (or very close) to 0

  (submit to complete is basically  just CPU  time)

lat     Total latency

  the time from when fio created the I/O unit to completion of the I/O operation

randwrite test

MyTest: (groupid=0, jobs=4): err= 0: pid=17719: Thu Oct 17 15:18:37 2019
  write: IOPS=45, BW=182KiB/s (187kB/s)(42.7MiB/240081msec); 0 zone resets
    slat (usec): min=24, max=1189.7k, avg=87792.58, stdev=93402.38
    clat (usec): min=449, max=7698.9k, avg=5457754.26, stdev=782456.18
     lat (msec): min=76, max=7757, avg=5545.55, stdev=785.56
....

write test

MyTest: (groupid=0, jobs=4): err= 0: pid=29176: Fri Oct 18 11:15:40 2019
  write: IOPS=2027, BW=8110KiB/s (8305kB/s)(1901MiB/240070msec); 0 zone resets
    slat (usec): min=4, max=1158.7k, avg=1827.57, stdev=14772.15
    clat (usec): min=165, max=1305.5k, avg=124341.19, stdev=120815.20
     lat (usec): min=804, max=1377.5k, avg=126169.08, stdev=121318.38
....