Troubleshooting Sustained 100% CPU Usage by zfs zed After an Upgrade

Investigates sustained 100% CPU usage by zed after a kernel and ZFS upgrade, tracing the problem to mismatched userspace and kernel-module versions and resolving it by rebuilding the module.

To use BBR, I upgraded my NAS kernel from 4.4 to 4.13. Ubuntu 16.04 LTS shipped with zfs 0.6.5, which did not support kernel 4.13, so I upgraded zfs to 0.7.6 using a PPA. After rebooting, however, I found that the zed daemon stayed at 100% CPU usage for long periods. Trying zpool upgrade -a to enable the new features also produced an error.

There are many accounts of this behavior online. Apparently, a bug in some version long ago caused similar symptoms. I tried updating zfs, but things still did not return to normal.

Later, I read an article saying that a version mismatch between zfsutils-linux and the kernel module could cause this problem. I ran modinfo zfs and found that the version was still 0.6.5. I then followed the zfsonlinux project's GitHub wiki, whose Custom Packages section explains how to compile and install it. This let me update the kernel module to 0.7.6, and after a reboot the problem disappeared.

Discussion

View discussion on GitHub

Built with Hugo
Theme Stack designed by Jimmy