heavy load of hashtable may cause C level assert

[Takashi Kato]

MQTT broker is using hashtable and following scripts causes C level assert of hashtable.

broker.scm

(import (rnrs) (net mq mqtt broker))

(define config (make-mqtt-broker-config
        :exception-handler (lambda (s so e) (print e))))
(define broker (make-mqtt-broker "5000" :config config))

(mqtt-broker-start! broker)

mqtt-pub.scm

(import (rnrs) (sagittarius control) (net mq mqtt client))

(let loop ()
  (let ((conn (open-mqtt-connection "localhost" "5000")))
    (unwind-protect
     (let lp ((count 0))
       (mqtt-publish conn "topic" 
             (string->utf8 (format "Hello MQTT ~a" count))
             :qos +qos-at-least-once+)
       (display count) (display "\r") (flush-output-port)
       (lp (if (= count 1000) 0 (+ count 1))))
     (close-mqtt-connection! conn)))
  (loop))

mqtt-sub.scm

(import (rnrs) (net mq mqtt client))

(let ((conn (open-mqtt-connection "localhost" "5000")))
  (mqtt-subscribe conn "topic" +qos-exactly-once+
                  (lambda (topic payload)
                    (get-bytevector-all payload)))
  (let loop ()
    (let ((r (mqtt-receive-message conn)))
      ;;(set! count (+ count 1))
      (display (utf8->string r))
      (display "\r")
      (flush-output-port)
      (loop)))
  (close-mqtt-connection! conn))

this may be the result

ASSERT failure /home/takashi/project/sagittarius/src/hashtable.c:305: table->entryCount >= 0

It happened only twice.

[Takashi Kato]

It seems timing issue on multi thread environment. The line 305 on hashtable.c is checking entry count after deletion.

There is no way to make this happen constantly for now.

[Takashi Kato]

C level assert is not a good behaviour because it can be eliminated by build option. So what we can do is

• Remove remaining entries, since entry count is zero anyway, and raise a Scheme error
• Make this panic condition (at least always kills it)

The latter one would be more consistent but the first one seems also fine. Need sometime to consider.

[Takashi Kato]

This is more or less the issue of concurrency. It's better to fix MQTT broker implementation instead of making built-in hashtable to concurrent hashtable.

I'll keep this issue for future reference but probably won't fix it.