Mysql error when the connection pool is cleaned up by a new thread.

[Ralf Ramsauer]

Hi,

it took me hours to create a minimal reconstruction of my problem. I get a error from libmysqlclient which is used by libzdb:

#!

Error in my_thread_global_end(): 1 threads didn't exit

This message appears when a thread different to the one that created the connection pool tries to do the connectionpool cleanup.

Here's a minimal reconstruction of the problem.

If the same thread cleans up the pool, everything is fine.

Thank you very much

[Ralf Ramsauer]

Some further random analysis: ConnectionPool_stop calls Connection_onstop calls MysqlConnection_onstop calls mysql_server_end.

Manpage of mysql_server_end says:

#!

 This function finalizes the MySQL library. You should call it when you are done using the library. 

So this function must not be called when stopping a single connectionPool as a further connectionPool to a different server might still be existing.

[Tildeslash]

This is the root cause then. Could you verify? If so, the solution is either to never call mysql_server_end or or only do it if one connection pool is used. I have to think about it.

[Ralf Ramsauer]

Yes i can verify. Commenting out mysql_server_end in the library solved my issue.

Maybe it is better to create a global pool-independent function like void zdb_init() and void zdb_cleanup() that do global pool-wide cleanups. Let's say you are connecting to a Postgre AND Mysql at the same time. When shutting down, the zdb_cleanup() would do the postgre AND mysql cleanup. Internally, zdb could store which libraries were in use during its runtime. Other libraries do similar cleanups.

[Tildeslash]

Sure, but I prefer the API to be as simple as possible for the end-user. I can think of two solutions, either simply remove Connection_onstop or use reference counting at ConnectionPool_new and only call Connection_onstop when ConnectionPool_stop is called and there are only one Connection Pool in use for the specific database. I think I prefer the first option, both because it is simplest and because cleaning up at program shutdown is not critical as the process will stop and the OS will do the cleanup.

[Ralf Ramsauer]

That's true, but i bet you will get bug reports when people are confused when running valgrind on their software :-)

I think reference counting is a very straight forward solution as the API will not change.

[Tildeslash]

Fixed: #7 Removed onstop handler which would call the underlying third-party database library shutdown method. In practice, explicitly calling a library shutdown function to cleanup resources at program exit is only "needed" for MySQL. The "onstop" handler was implemented for MySQL and SQLIte. The handler could cause a problem for MySQL and potentially for SQLite if more than one Connection Pool was used as the onstop handler was called in ConnectionPool_stop() and therefor would render other live Connection Pools invalid as the underlying library was shutdown. To solve the problem we first created a reference counting solution for Connection Pool to make sure that we only called "onstop" when reference count went to zero. The solution needed one refcount per database type supported and the overhead started to look unappealing compared to simply ignore third-party library cleanup at shutdown in which case the OS would cleanup the process resources in any case. We might revert to the reference counting model if it turns out that this decision will have side-effects which we haven't encounter during our testing.

→ <<cset a186816aa9b4>>