I'm putting this on hold until upstream fixes removing rules in a predictable fashion. It’s too easy to delete the wrong rule when trying to guess the correct handle to delete on someone else's system.
the solution is to use the correct function of nft. firstly, just like iptables, separate chains are supported so that one can be assigned to a firewall generator process like sshguard. secondly, for cases like this, instead of adding individual rules which is very inefficient at evaluation time, a set should be used, ipset or nft set.
edit: it seems I was misled by the (very?) out-of-date documentation. the point is that nft (now?) supports sets.
fwiw the proposed solution doesn't work if the user chooses to use "inet" family. I think the best solution is just to document an example script and have the user adjust it for their own configuration.