Accelerate black list loading with nftables backend

Create issue
Issue #151 open
Former user created an issue

I have a black list of tens of thousands addresses on my server. Every time it reboots, sshguard configures them one by one via nft add element comand, which takes up to half an hour.

I tried switching to batch configuration with nft -f, and it takes less than a minute to complete. My approach is to cache block commands in a tmpfile, and flush to nftables after a timeout.

I think it would be better to use batch config only during black list loading, and use nft add element afterwards. This may need to introduce new backend commands to indicate the start and end of initialization.

Some changes to sshg-fw-nft-sets in test:

fw_block() {
    prot="$(proto $2)"
    file="/tmp/sshguard-nft-${prot}.rule"
    if [ ! -s ${file} ]; then
        cat > ${file} << EOF
table ${prot} ${NFT_TABLE} {
    set ${NFT_SET} {
        type ipv${2}_addr
        flags interval
        elements = {
EOF
    fi
    echo "            $1/$3," >> ${file}
}

fw_block_flush() {
    for prot in ip ip6; do
        file="/tmp/sshguard-nft-${prot}.rule"
        if [ -s ${file} ]; then
            echo -e "        }\n    }\n}" >> ${file}
            ${CMD_NFT} -f ${file}
            rm ${file}
        fi
    done
}
while :; do
    read -t 3 -r cmd address addrtype cidr
    ret=$?
    if [ "$ret" -gt 128 ]; then
        fw_block_flush
    elif [ $ret -eq 0 ]; then
        if [ $cmd != "block" ]; then
            fw_block_flush
        fi
        case $cmd in
            block)
                fw_block "$address" "$addrtype" "$cidr";;
            release)
                fw_release "$address" "$addrtype" "$cidr";;
            flush)
                fw_flush;;
            flushonexit)
                flushonexit=YES;;
            *)
                die 65 "Invalid command";;
        esac
    else
        break
    fi
done

Comments (2)

  1. Kevin Zheng
    • changed status to open

    Would it help if all the backends gained a "batch add" command? That way SSHGuard can issue a single "batch add" command at startup when it re-blacklists everything, then goes back to blocking addresses one by one.

    Alternatively, do you know if there's a better way to control nft so that it isn't so slow?

  2. Xiao Liang

    Do you mean a single batch add command with huge arguments? It could be fine, but I would prefer enclosing block commands with a batch-start and batch-end pair. A benefit is that backends which don’t (or not necessary to) support batch can simply ignore them.

    It seems that nft add element reads the set from kernel to determine its type, while the batch form can provide this information. I don’t know if there’re other ways to fix it.

  3. Log in to comment