fsflags monitoring does not detect when fs became read-only on Linux

On linux fsflags monitoring does not detect if kernel remounted filesystem read-only.

Context: For example, on vmware environment, storage sometimes may get delayed and, because of timeouts, kernel interprets these delays as errors and remount filesystem read-only. Monit is unable to detect this condition with changed fsflags monitoring. This is important condition for server health but monit is unexpectedly quiet about it.

It could be reproduced with echo u > /proc/sysrq-trigger, after that observe that monit status still shows filesystem flags 0x1000. In the same time /proc/mounts shows that filesystem is actually changed flags to ro.

Example on Centos 7.3.1611 system:

root@test:/# echo u > /proc/sysrq-trigger
-- dmesg --
[222022.193510] SysRq : Emergency Remount R/O
[222022.274584] EXT4-fs (sdb): re-mounted. Opts: (null)
[222022.567667] EXT4-fs (loop0): re-mounted. Opts: (null)
[222022.567695] Emergency Remount complete

root@test:/# grep -w ro /proc/mounts
tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0
/dev/mapper/centos_test2-root / xfs ro,relatime,attr2,inode64,noquota 0 0
/dev/sda1 /boot xfs ro,relatime,attr2,inode64,noquota 0 0
/dev/sdb /space ext4 ro,relatime 0 0

root@test:/# monit status|grep flags
  filesystem flags                  0x1000
  filesystem flags                  0x1000

Some thoughts: Not necessarily statvfs() syscall is working incorrectly, because, f_flag value is defined as /* mount flags */, i.e. it could be interpreted as mount time flags, and not as current filesystem state flags.

You state in documentation:

This test is useful for detecting changes of filesystem flags such as if the filesystem become read-only (on disk error) or mount flags were changed (such as nosuid).

"on disk error" is exactly incorrect statement.

Comments (4)