ZFS space usage irritations

[Sebastian Gabbert]

I am trying to set some alerts on zfs datasets / zfs pools and am puzzled about the given percentages.

[root@gorleben /usr/local/etc/monit.d] zfs list
NAME                 USED  AVAIL  REFER  MOUNTPOINT
zroot                202G   244G    88K  /zroot
zroot/pgsql          197G   244G   159G  /usr/local/pgsql

[root@gorleben /usr/local/etc/monit.d] zpool list
NAME    SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
zroot   460G   202G   258G        -         -    70%    43%  1.00x  ONLINE  -

Monit config

check filesystem pgsql with path /usr/local/pgsql
  if space usage > 1% then exec "/usr/local/bin/monitbridge --level warning --debug"

monitbridge is my bridge to slack. The monit variable say:

MONIT_DESCRIPTION=space usage 39.5% matches resource limit [space usage > 1.0%]

When I calculate usage based on the 197G shown by zfs list I expect ~42% and if I use the 202G used by the entire pool I expect ~44%.

So I looked further:

[root@gorleben /usr/local/etc/monit.d] df -h
Filesystem            Size    Used   Avail Capacity  Mounted on
zroot/pgsql           403G    159G    244G    40%    /usr/local/pgsql

Here we find the 39%. I got the strong feeling, monit should use the values reported by zfs list / zpool list rather than the df values.

[Tildeslash]

Monit reads the statistics directly via statvfs() system API - it doesn't read the statistics from "df", but "df" most probably uses the same data source.

Please can you attach full output of "monit status pgsql"?

[Sebastian Gabbert]

Thanks for the reply. I didn't want to imply you read data from df :)

This oracle link states The statvfs() function returns correct results for the total number of blocks in the entire pool (total) and the number of available blocks within a given file system (free). The equation “used = total - free” will not work for ZFS file systems, due to the fact that in pooled storage many file systems share the total available from the entire storage pool.

Is it maybe possible to specify the filesystem or use another check that specifies that the checked fs is zfs and use another datasource?

Monit 5.25.2 uptime: 6d 3h 16m

Filesystem 'pgsql'
  status                       OK
  monitoring status            Monitored
  monitoring mode              active
  on reboot                    start
  filesystem type              zfs
  filesystem flags             nfs4acls, noatime, local
  permission                   700
  uid                          770
  gid                          770
  block size                   512 B
  space total                  397.3 GB (of which 0.0% is reserved for root user)
  space free for non superuser 237.4 GB [59.8%]
  space free total             237.4 GB [59.8%]
  inodes total                 497965932
  inodes free                  497953672 [100.0%]
  read                         0 B/s [0 B total], 0.0 reads/s [0 reads total]
  write                        0 B/s [0 B total], 0.0 writes/s [0 writes total]
  service time                 0.000ms/operation (of which read 0.000ms, write 0.000ms)
  data collected               Wed, 19 Sep 2018 13:57:06

zpool list

NAME    SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
zroot   460G   208G   252G        -         -    70%    45%  1.00x  ONLINE  -

zfs list

zroot/pgsql          201G   237G   160G  /usr/local/pgsql

[Tildeslash]

Thanks for update, we'll look on it (it will take some time, we're busy with other stuff now).

[Christos Chatzaras]

Is any way to monitor total zpool space usage instead of seperate datasets?

[Christos Chatzaras]

Do I have to use a script like this?

https://gist.github.com/Rychu-Pawel/4419344dc420f58e17fdf51fef6df9d9