Huge design problem in wsgi server
Running cherrypy wsgi server as a wrapper for flask, noticed that i got no responses from my clients (they were stuck), even though there were only 2 req/sec. Tried increasing worker threads to 100s with no luck.
Some debugging led me to understand that there were many timeouts with the ssl handshakes. In your design, the ssl handshakes are done on the main threads and not on worker threads, so other client's are 'stuck'.
Solved this by moving cherrypy behind nginx which does the SSLing and it works.
Should be done on different threads.