Improving Read Performance with ZFS on Linux

Examines unexpectedly low drive utilization during reads from an eight-disk RAIDZ2 pool and describes improving throughput by adjusting ZFS’s zfetch_max_distance prefetch parameter.

My ZFS system is a raidz2 array of 8 × 10 TB HDDs, and its read performance has consistently been around 190 MB/s. That is not terribly slow, but at that speed iostat shows disk utilization of only about 25%, while CPU usage is also low, leaving plenty of room for improvement.

After quite a bit of searching online, I found that the zfetch_max_distance parameter might affect read performance. It controls the maximum stream prefetch size, or, you could say, the size of the read buffer (see this documentation).

Using the command echo 67108864 > /sys/module/zfs/parameters/zfetch_max_distance to set this parameter to 64 MB can significantly improve read speed.

Discussion

View discussion on GitHub

Built with Hugo
Theme Stack designed by Jimmy