Skip to content

Instantly share code, notes, and snippets.

@jwasinger
Last active May 15, 2026 12:06
Show Gist options
  • Select an option

  • Save jwasinger/cb6a3ba548e4d93869b7ed5d54179d28 to your computer and use it in GitHub Desktop.

Select an option

Save jwasinger/cb6a3ba548e4d93869b7ed5d54179d28 to your computer and use it in GitHub Desktop.
IO Benchmarks: external disk (`/chad`) vs root mount (`/`)

on /chad

fio --name=seq --filename=test --direct=1 --ioengine=libaio --iodepth=64 --bs=1M --size=4G --rw=readwrite

seq: (g=0): rw=rw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=libaio, iodepth=64
fio-3.39
Starting 1 process
Jobs: 1 (f=1): [M(1)][100.0%][r=215MiB/s,w=235MiB/s][r=215,w=235 IOPS][eta 00m:00s]
seq: (groupid=0, jobs=1): err= 0: pid=483105: Thu May 14 21:23:31 2026
  read: IOPS=214, BW=214MiB/s (225MB/s)(1992MiB/9295msec)
    slat (usec): min=1236, max=18539, avg=2911.41, stdev=996.86
    clat (usec): min=4, max=332680, avg=140450.65, stdev=20997.03
     lat (msec): min=2, max=334, avg=143.36, stdev=21.13
    clat percentiles (msec):
     |  1.00th=[  102],  5.00th=[  127], 10.00th=[  131], 20.00th=[  136],
     | 30.00th=[  138], 40.00th=[  138], 50.00th=[  140], 60.00th=[  140],
     | 70.00th=[  142], 80.00th=[  144], 90.00th=[  148], 95.00th=[  155],
     | 99.00th=[  194], 99.50th=[  317], 99.90th=[  334], 99.95th=[  334],
     | 99.99th=[  334]
   bw (  KiB/s): min=67584, max=243712, per=98.82%, avg=216860.44, stdev=39711.50, samples=18
   iops        : min=   66, max=  238, avg=211.78, stdev=38.78, samples=18
  write: IOPS=226, BW=226MiB/s (237MB/s)(2104MiB/9295msec); 0 zone resets
    slat (usec): min=984, max=20510, avg=1647.62, stdev=1083.85
    clat (msec): min=2, max=333, avg=140.78, stdev=22.53
     lat (msec): min=4, max=335, avg=142.43, stdev=22.80
    clat percentiles (msec):
     |  1.00th=[   77],  5.00th=[  127], 10.00th=[  131], 20.00th=[  136],
     | 30.00th=[  138], 40.00th=[  140], 50.00th=[  140], 60.00th=[  142],
     | 70.00th=[  142], 80.00th=[  144], 90.00th=[  148], 95.00th=[  155],
     | 99.00th=[  197], 99.50th=[  321], 99.90th=[  334], 99.95th=[  334],
     | 99.99th=[  334]
   bw (  KiB/s): min=79872, max=284672, per=97.93%, avg=226986.67, stdev=42492.92, samples=18
   iops        : min=   78, max=  278, avg=221.67, stdev=41.50, samples=18
  lat (usec)   : 10=0.02%
  lat (msec)   : 4=0.02%, 10=0.10%, 20=0.12%, 50=0.37%, 100=0.59%
  lat (msec)   : 250=97.90%, 500=0.88%
  cpu          : usr=0.93%, sys=33.52%, ctx=4105, majf=0, minf=12
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.2%, 16=0.4%, 32=0.8%, >=64=98.5%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued rwts: total=1992,2104,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
   READ: bw=214MiB/s (225MB/s), 214MiB/s-214MiB/s (225MB/s-225MB/s), io=1992MiB (2089MB), run=9295-9295msec
  WRITE: bw=226MiB/s (237MB/s), 226MiB/s-226MiB/s (237MB/s-237MB/s), io=2104MiB (2206MB), run=9295-9295msec

fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=64 --size=4G --readwrite=randrw --rwmixread=75

test: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=64
fio-3.39
Starting 1 process
Jobs: 1 (f=1): [m(1)][99.6%][r=10.7MiB/s,w=3811KiB/s][r=2750,w=952 IOPS][eta 00m:02s]
test: (groupid=0, jobs=1): err= 0: pid=483178: Thu May 14 21:33:57 2026
  read: IOPS=1411, BW=5646KiB/s (5782kB/s)(3070MiB/556773msec)
   bw (  KiB/s): min= 1432, max=14080, per=99.98%, avg=5645.68, stdev=1542.72, samples=1113
   iops        : min=  358, max= 3520, avg=1411.40, stdev=385.67, samples=1113
  write: IOPS=471, BW=1887KiB/s (1932kB/s)(1026MiB/556773msec); 0 zone resets
   bw (  KiB/s): min=  400, max= 4681, per=99.95%, avg=1886.89, stdev=523.58, samples=1113
   iops        : min=  100, max= 1170, avg=471.70, stdev=130.89, samples=1113
  cpu          : usr=0.28%, sys=4.01%, ctx=773878, majf=0, minf=5
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued rwts: total=785920,262656,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
   READ: bw=5646KiB/s (5782kB/s), 5646KiB/s-5646KiB/s (5782kB/s-5782kB/s), io=3070MiB (3219MB), run=556773-556773msec
  WRITE: bw=1887KiB/s (1932kB/s), 1887KiB/s-1887KiB/s (1932kB/s-1932kB/s), io=1026MiB (1076MB), run=556773-556773msec

on /:

fio --name=seq --filename=test --direct=1 --ioengine=libaio --iodepth=64 --bs=1M --size=4G --rw=readwrite

seq: (g=0): rw=rw, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=libaio, iodepth=64
fio-3.39
Starting 1 process
Jobs: 1 (f=1)
seq: (groupid=0, jobs=1): err= 0: pid=483654: Thu May 14 21:42:47 2026
  read: IOPS=1445, BW=1446MiB/s (1516MB/s)(1992MiB/1378msec)
    slat (nsec): min=3364, max=89463, avg=6350.31, stdev=5691.15
    clat (usec): min=1990, max=45499, avg=13929.42, stdev=6514.46
     lat (usec): min=1995, max=45504, avg=13935.77, stdev=6514.36
    clat percentiles (usec):
     |  1.00th=[ 3261],  5.00th=[ 5080], 10.00th=[ 6849], 20.00th=[ 8979],
     | 30.00th=[10552], 40.00th=[11469], 50.00th=[12518], 60.00th=[13960],
     | 70.00th=[16057], 80.00th=[18220], 90.00th=[22676], 95.00th=[26608],
     | 99.00th=[34341], 99.50th=[37487], 99.90th=[45351], 99.95th=[45351],
     | 99.99th=[45351]
   bw (  MiB/s): min= 1330, max= 1560, per=99.96%, avg=1445.00, stdev=162.63, samples=2
   iops        : min= 1330, max= 1560, avg=1445.00, stdev=162.63, samples=2
  write: IOPS=1526, BW=1527MiB/s (1601MB/s)(2104MiB/1378msec); 0 zone resets
    slat (usec): min=16, max=230, avg=49.33, stdev=14.52
    clat (usec): min=2199, max=65876, avg=28598.71, stdev=10691.31
     lat (usec): min=2250, max=65932, avg=28648.04, stdev=10689.68
    clat percentiles (usec):
     |  1.00th=[ 7767],  5.00th=[12387], 10.00th=[15270], 20.00th=[19792],
     | 30.00th=[22938], 40.00th=[24773], 50.00th=[27132], 60.00th=[29492],
     | 70.00th=[33817], 80.00th=[37487], 90.00th=[43779], 95.00th=[47449],
     | 99.00th=[56886], 99.50th=[60031], 99.90th=[65799], 99.95th=[65799],
     | 99.99th=[65799]
   bw (  MiB/s): min= 1416, max= 1564, per=97.59%, avg=1490.00, stdev=104.65, samples=2
   iops        : min= 1416, max= 1564, avg=1490.00, stdev=104.65, samples=2
  lat (msec)   : 2=0.02%, 4=1.56%, 10=12.30%, 20=38.40%, 50=46.07%
  lat (msec)   : 100=1.64%
  cpu          : usr=8.71%, sys=0.29%, ctx=256, majf=0, minf=10
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.2%, 16=0.4%, 32=0.8%, >=64=98.5%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued rwts: total=1992,2104,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
   READ: bw=1446MiB/s (1516MB/s), 1446MiB/s-1446MiB/s (1516MB/s-1516MB/s), io=1992MiB (2089MB), run=1378-1378msec
  WRITE: bw=1527MiB/s (1601MB/s), 1527MiB/s-1527MiB/s (1601MB/s-1601MB/s), io=2104MiB (2206MB), run=1378-1378msec

Disk stats (read/write):
  vda: ios=1724/1770, sectors=3407872/3524608, merge=0/0, ticks=22796/49562, in_queue=72358, util=81.86%

fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=64 --size=4G --readwrite=randrw --rwmixread=75

test: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=64
fio-3.39
Starting 1 process
Jobs: 1 (f=1): [m(1)][100.0%][r=497MiB/s,w=166MiB/s][r=127k,w=42.5k IOPS][eta 00m:00s]
test: (groupid=0, jobs=1): err= 0: pid=483658: Thu May 14 21:43:47 2026
  read: IOPS=117k, BW=458MiB/s (481MB/s)(3070MiB/6699msec)
   bw (  KiB/s): min=262560, max=529424, per=99.75%, avg=468101.54, stdev=82616.03, samples=13
   iops        : min=65640, max=132356, avg=117025.69, stdev=20653.95, samples=13
  write: IOPS=39.2k, BW=153MiB/s (161MB/s)(1026MiB/6699msec); 0 zone resets
   bw (  KiB/s): min=87248, max=177128, per=99.80%, avg=156527.38, stdev=27933.09, samples=13
   iops        : min=21812, max=44282, avg=39131.85, stdev=6983.27, samples=13
  cpu          : usr=10.66%, sys=41.85%, ctx=75694, majf=0, minf=8
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued rwts: total=785920,262656,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
   READ: bw=458MiB/s (481MB/s), 458MiB/s-458MiB/s (481MB/s-481MB/s), io=3070MiB (3219MB), run=6699-6699msec
  WRITE: bw=153MiB/s (161MB/s), 153MiB/s-153MiB/s (161MB/s-161MB/s), io=1026MiB (1076MB), run=6699-6699msec

Disk stats (read/write):
  vda: ios=778707/260205, sectors=6229656/2081648, merge=0/1, ticks=296172/76996, in_queue=373169, util=61.57%
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment