tcmalloc should be disabled in Linux (OSX?) builds

Issue #1827 resolved
Nicky Dasmijn
created an issue

The current linux builds still have tcmalloc on by default, this causes a few issues for people using the prebuild from cefbuilds.com/forcing them to build their own versions.

1) If someone uses dlopen or friends, this will often lead to crashes or memory corruption. https://github.com/cztomczak/cefpython/issues/155 https://github.com/cztomczak/cefpython/issues/73 and likely more

2) Performance benefits seem marginal http://www.magpcss.org/ceforum/viewtopic.php?f=10&t=11462 (granted this is a bit old)

3) If someone would like something as intrusive as tcmalloc in their application, it should be the main executable that dictates this, not some so/dll/dylib that gets loaded.

4) This goes along with 3, even if there would be a strong need for tcmalloc, it's quite bad to leak an implementation detail like it and bleed it over the whole process, rather than at least keeping it local to just the loaded module

Comments (16)

  1. Nicky Dasmijn reporter

    I understand your reluctance, I'd feel the same in your shoes :) What I do is use LD_PRELOAD in fact (it is a little more tricky, as the CEF host gets spawned from another process, but it works). I am not fond not rebuilding CEF, as it adds a big burden for each (security) update.

    I do not think everyone (maybe little exaggeration on my part ;) ) compiling their own CEF is a viable solution, mainly for 2 reasons: 1) The binary version should be a reasonable default, see my 4 points why I personally think the current version is not (of course all IMO) 2) Software maintainers will feel a lot more reluctant to update a package if it possibly involves downloading a few gib and then compiling for a few hours, in contrast to downloading a prebuild and just do a minimal compile.

    Back to your valid concerns, what are you afraid it breaks? Considering afaik Windows-CEF already does not even use tcmalloc anymore, so all those code paths that are shared should be 'safe'.

    Another idea, I do not know how much influence you have about the cefbuilds and how much cpu power/upload there is still to spare, could two flavors be doable, one tcmalloc and one 'no tcmalloc' set of builds (for Linux)?

  2. Tom Jakubowski

    The fix as described in the JCEF issue is to preload libcef.so via LD_PRELOAD.

    To clarify, you only need to be sure that libcef.so is loaded before libc is, so it can provide malloc/free symbols before libc. LD_PRELOAD is one way to do that.

    Back to your valid concerns, what are you afraid it breaks? Considering afaik Windows-CEF already does not even use tcmalloc anymore, so all those code paths that are shared should be 'safe'.

    I think we tried a non-tcmalloc build (maybe it was a build that didn't replace malloc/free, but still used tcmalloc internally) on Linux a few months ago and experienced strange issues (incl. crashes) in skia. I seem to remember problems in particular when resizing a browser.

    4) This goes along with 3, even if there would be a strong need for tcmalloc, it's quite bad to leak an implementation detail like it and bleed it over the whole process, rather than at least keeping it local to just the loaded module

    Chromium's source isn't perfect with regards to matching malloc calls with free calls and tc_malloc calls with tc_free calls. That means CEF builds which use tcmalloc internally but don't replace malloc may have issues (see above).

  3. Czarek Tomczak

    On Windows tcmalloc is certainly not being used.

    @Marshall Greenblatt I have been providing CEF Python releases for Linux with tcmalloc disabled for more than a year and I haven't noticed any issues nor reports from users about any issues. In fact I had many issue reports when tcmalloc was enabled and now it's quiet. What kind of issues should I look out for?

    You say that changing allocator is untested in Chromium. But how about looking at it the other way - is tcmalloc tested with thousands of existing applications that are going to be integrating CEF? The worst thing about tcmalloc is that issue can go unrevealed for many months and then one day you get crash in your app! This is because these crashes are random and can reveal itself at some point when memory layout changes in app.

    I can understand your reluctance to not make such a big change, because you mostly work with statically compiled languages like C++ and Java and in your programs CEF and most of other libraries are automatically preloaded at program startup, or at least you have more way to control that in a predictable manner. But in a dynamic language like Python most of the libraries are loaded dynamically at random points of the script running. I had to warn users in code comments and in documentation that you had to import the cefpython library at the very beginning of a python script. But that didn't look professional and make look for my library to be some kind of a hack. Still many users ignored this warning (because everything worked fine at that time) and then some weeks later they had unexpected crashes in their apps.

  4. Log in to comment