Fix for bad write performance with LUKS + RAID5

After setting up a new fileserver with an LUKS encrypted dataset on top of Linux software RAID5 I did some benchmarks and reliability testing before filling it up with “real” data… but it turns out write performance was quite horrible. Only a fraction of what the “raw” RAID5 could do, all while having the CPU mostly idle. So what’s going on?

Apparently only small blocks of data get written to the disks when using LUKS encrypted volumes (mostly default settings) therefore dramatically reducing write performance (figured it out thanks to this) on my RAID 5 volume. Setting theĀ stripe_cache_size from 256 to 8192 increased throughput from roughly 50-70 MB/sec to well above 200 MB/sec.

echo 8192 > /sys/block/md0/md/stripe_cache_size

To permanently set it, so you don’t have to manually adjust it after every reboot, consider setting up a udev rule like this:

/etc/udev/rules.d/md-stripe-cache.rules

SUBSYSTEM=="block", KERNEL=="md*", ACTION=="change", TEST=="md/stripe_cache_size", ATTR{md/stripe_cache_size}="8192"