1.12 SEGVs when OpenMP is enabled
$ ART
(ART:37787): GLib-GObject-CRITICAL **: 13:45:43.319: g_object_unref: assertion 'old_ref > 0' failed
(ART:37787): GLib-GObject-CRITICAL **: 13:45:43.319: g_object_unref: assertion 'old_ref > 0' failed
(ART:37787): GLib-GObject-CRITICAL **: 13:45:43.326: g_object_unref: assertion 'old_ref > 0' failed
OMP: Error #13: Assertion failure at kmp_runtime.cpp(3689).
OMP: Hint Please submit a bug report with this message, compile and run commands used, and machine configuration info including native compiler and operating system versions. Faster response will be obtained by including all program sources. For information on submitting this issue, please see https://bugs.llvm.org/.
Abort trap
[yuri@yv /disk-samsung/freebsd-ports/graphics/art]$ ART
(ART:37789): GLib-GObject-CRITICAL **: 13:46:02.394: g_object_unref: assertion 'old_ref > 0' failed
Segmentation fault
Comments (25)
-
reporter -
reporter clang-12
OS: FreeBSD 13
-
repo owner Hi,
can you provide a GDB backtrace of a debug build? Thanks!
-
reporter (gdb) bt #0 thr_kill () at thr_kill.S:4 #1 0x00000008041b13d4 in __raise (s=s@entry=6) at /disk-samsung/freebsd-src/lib/libc/gen/raise.c:52 #2 0x0000000804265f39 in abort () at /disk-samsung/freebsd-src/lib/libc/stdlib/abort.c:67 #3 0x00000008031428c0 in __kmp_abort_process() () at /disk-samsung/freebsd-src/contrib/llvm-project/openmp/runtime/src/kmp_runtime.cpp:449 #4 0x000000080313b014 in __kmp_fatal(kmp_msg_t, ...) (message=...) at /disk-samsung/freebsd-src/contrib/llvm-project/openmp/runtime/src/kmp_i18n.cpp:868 #5 0x00000008031223a5 in __kmp_debug_assert(char const*, char const*, int) (msg=<optimized out>, file=<optimized out>, line=3689) at /disk-samsung/freebsd-src/contrib/llvm-project/openmp/runtime/src/kmp_debug.cpp:74 #6 0x0000000803141cba in __kmp_register_root(int) (initial_thread=<optimized out>, initial_thread@entry=0) at /disk-samsung/freebsd-src/contrib/llvm-project/openmp/runtime/src/kmp_runtime.cpp:3689 #7 0x0000000803141791 in __kmp_get_global_thread_id_reg() () at /disk-samsung/freebsd-src/contrib/llvm-project/openmp/runtime/src/kmp_runtime.cpp:253 #8 0x0000000000ce636f in rtengine::Imagefloat::multiply(float, bool) (this=0x110f310 <rtengine::ICCStore::getInstance()::instance>, factor=0, multithread=false) at /disk-samsung/freebsd-ports/graphics/art/work/ART-1.12/rtengine/imagefloat.cc:395 #9 0x0000000000cee24d in rtengine::Imagefloat::normalizeFloatTo1(bool) (this=0x814e09800, multithread=true) at /disk-samsung/freebsd-ports/graphics/art/work/ART-1.12/rtengine/imagefloat.cc:429 #10 0x0000000000f8c94a in rtengine::StdImageSource::colorSpaceConversion(rtengine::Imagefloat*, rtengine::procparams::ColorManagementParams const&, void*, rtengine::IIO_Sample_Format, rtengine::ProgressListener*) (im=0x814e09800, cmp=..., embedded=0x0, sampleFormat=rtengine::IIOSF_UNKNOWN, plistener=0x0) at /disk-samsung/freebsd-ports/graphics/art/work/ART-1.12/rtengine/stdimagesource.cc:301 #11 0x0000000000f6e1ff in rtengine::Thumbnail::processImage(rtengine::procparams::ProcParams const&, rtengine::SensorType, int, rtengine::TypeInterpolation, rtengine::FramesMetaData const*, double&, bool, bool) (this=0x814e6db80, params=..., sensorType=rtengine::ST_NONE, rheight=126, interp=rtengine::TI_Bilinear, metadata=0x808420028, myscale=@0x7fffdf5f8af0: 1, forMonitor=true, forHistogramMatching=false) at /disk-samsung/freebsd-ports/graphics/art/work/ART-1.12/rtengine/rtthumbnail.cc:1041 #12 0x0000000000b0d149 in Thumbnail::processThumbImage(rtengine::procparams::ProcParams const&, int, double&) (this=0x808420000, pparams=..., h=126, scale=@0x7fffdf5f8af0: 1) at /disk-samsung/freebsd-ports/graphics/art/work/ART-1.12/rtgui/thumbnail.cc:687 #13 0x0000000000b00b92 in ThumbImageUpdater::Impl::processNextJob() (this=0x81006ac80) at /disk-samsung/freebsd-ports/graphics/art/work/ART-1.12/rtgui/thumbimageupdater.cc:152 #14 0x0000000000b02442 in sigc::bound_mem_functor0<void, ThumbImageUpdater::Impl>::operator()() const (this=0x8122a1ab8) at /usr/local/include/sigc++-2.0/sigc++/functors/mem_fun.h:1991 #15 0x0000000000b023bd in std::__1::__invoke<sigc::bound_mem_functor0<void, ThumbImageUpdater::Impl>&>(sigc::bound_mem_functor0<void, ThumbImageUpdater::Impl>&) (__f=...) at /usr/include/c++/v1/type_traits:3694 #16 0x0000000000b0238d in std::__1::__apply_functor<sigc::bound_mem_functor0<void, ThumbImageUpdater::Impl>, std::__1::tuple<>, , std::__1::tuple<> >(sigc::bound_mem_functor0<void, ThumbImageUpdater::Impl>&, std::__1::tuple<>&, std::__1::__tuple_indices<>, std::__1::tuple<>&&) (__f=..., __bound_args=..., __args=...) at /usr/include/c++/v1/functional:2857 #17 0x0000000000b0234b in std::__1::__bind<sigc::bound_mem_functor0<void, ThumbImageUpdater::Impl>&>::operator()<>() (this=0x8122a1ab8) at /usr/include/c++/v1/functional:2890 #18 0x0000000000b022fd in std::__1::__invoke<std::__1::__bind<sigc::bound_mem_functor0<void, ThumbImageUpdater::Impl>&>&>(std::__1::__bind<sigc::bound_mem_functor0<void, ThumbImageUpdater::Impl>&>&) (__f=...) at /usr/include/c++/v1/type_traits:3694 #19 0x0000000000b02141 in std::__1::__packaged_task_func<std::__1::__bind<sigc::bound_mem_functor0<void, ThumbImageUpdater::Impl>&>, std::__1::allocator<std::__1::__bind<sigc::bound_mem_functor0<void, ThumbImageUpdater::Impl>&> >, void ()>::operator()() (this=0x8122a1ab0) at /usr/include/c++/v1/future:1684 #20 0x00000000006e87fa in std::__1::__packaged_task_function<void ()>::operator()() const (this=0x812293e40) at /usr/include/c++/v1/future:1866 #21 0x00000000006e8721 in std::__1::packaged_task<void ()>::operator()() (this=0x812293e40) at /usr/include/c++/v1/future:2086 #22 0x0000000000b0430d in rtengine::ThreadPool::enqueue<sigc::bound_mem_functor0<void, ThumbImageUpdater::Impl>&>(rtengine::ThreadPool::Priority, sigc::bound_mem_functor0<void, ThumbImageUpdater::Impl>&)::{lambda()#1}::operator()() const (this=0x814e00018) at /disk-samsung/freebsd-ports/graphics/art/work/ART-1.12/rtgui/../rtengine/threadpool.h:159 #23 0x0000000000b042cd in std::__1::__invoke<rtengine::ThreadPool::enqueue<sigc::bound_mem_functor0<void, ThumbImageUpdater::Impl>&>(rtengine::ThreadPool::Priority, sigc::bound_mem_functor0<void, ThumbImageUpdater::Impl>&)::{lambda()#1}&>(sigc::bound_mem_functor0<void, ThumbImageUpdater::Impl>&) (__f=...) at /usr/include/c++/v1/type_traits:3694 #24 0x0000000000b0427d in std::__1::__invoke_void_return_wrapper<void, true>::__call<rtengine::ThreadPool::enqueue<sigc::bound_mem_functor0<void, ThumbImageUpdater::Impl>&>(rtengine::ThreadPool::Priority, sigc::bound_mem_functor0<void, ThumbImageUpdater::Impl>&)::{lambda()#1}&>(rtengine::ThreadPool::enqueue<sigc::bound_mem_functor0<void, ThumbImageUpdater::Impl>&>(rtengine::ThreadPool::Priority, sigc::bound_mem_functor0<void, ThumbImageUpdater::Impl>&)::{lambda()#1}&) (__args=...) at /usr/include/c++/v1/__functional_base:348 #25 0x0000000000b0424d in std::__1::__function::__alloc_func<rtengine::ThreadPool::enqueue<sigc::bound_mem_functor0<void, ThumbImageUpdater::Impl>&>(rtengine::ThreadPool::Priority, sigc::bound_mem_functor0<void, ThumbImageUpdater::Impl>&)::{lambda()#1}, std::__1::allocator<rtengine::ThreadPool::enqueue<sigc::bound_mem_functor0<void, ThumbImageUpdater::Impl>&>(rtengine::ThreadPool::Priority, sigc::bound_mem_functor0<void, ThumbImageUpdater::Impl>&)::{lambda()#1}>, void ()>::operator()() (this=0x814e00018) at /usr/include/c++/v1/functional:1558 #26 0x0000000000b03129 in std::__1::__function::__func<rtengine::ThreadPool::enqueue<sigc::bound_mem_functor0<void, ThumbImageUpdater::Impl>&>(rtengine::ThreadPool::Priority, sigc::bound_me m_functor0<void, ThumbImageUpdater::Impl>&)::{lambda()#1}, std::__1::allocator<rtengine::ThreadPool::enqueue<sigc::bound_mem_functor0<void, ThumbImageUpdater::Impl>&>(rtengine::ThreadPool::Priority, sigc::bound_mem_functor0<void, ThumbImageUpdater::Impl>&)::{lambda()#1}>, void ()>::operator()() (this=0x814e00010) at /usr/include/c++/v1/functional:1732 #27 0x0000000000d34522 in std::__1::__function::__value_func<void ()>::operator()() const (this=0x814e00010) at /usr/include/c++/v1/functional:1885 #28 0x0000000000d31f95 in std::__1::function<void ()>::operator()() const (this=0x814e00010) at /usr/include/c++/v1/functional:2560 #29 0x0000000000d33a6d in std::__1::__invoke<rtengine::ThreadPool::Task&>(rtengine::ThreadPool::Task&) (__f=...) at /usr/include/c++/v1/type_traits:3694 #30 0x0000000000d33a1d in std::__1::__invoke_void_return_wrapper<void, true>::__call<rtengine::ThreadPool::Task&>(rtengine::ThreadPool::Task&) (__args=...) at /usr/include/c++/v1/__functional_base:348 #31 0x0000000000d339ed in std::__1::__function::__alloc_func<rtengine::ThreadPool::Task, std::__1::allocator<rtengine::ThreadPool::Task>, void ()>::operator()() (this=0x814e00010) at /usr/include/c++/v1/functional:1558 #32 0x0000000000d32ac9 in std::__1::__function::__func<rtengine::ThreadPool::Task, std::__1::allocator<rtengine::ThreadPool::Task>, void ()>::operator()() (this=0x814e00000) at /usr/include/c++/v1/functional:1732 #33 0x0000000000d34522 in std::__1::__function::__value_func<void ()>::operator()() const (this=0x7fffdf5f8f00) at /usr/include/c++/v1/functional:1885 #34 0x0000000000d31f95 in std::__1::function<void ()>::operator()() const (this=0x7fffdf5f8f00) at /usr/include/c++/v1/functional:2560 #35 0x0000000000d31cfb in rtengine::ThreadPool::ThreadPool(unsigned long)::{lambda()#1}::operator()() const (this=0x80790c378) at /disk-samsung/freebsd-ports/graphics/art/work/ART-1.12/rtengine/threadpool.h:131 #36 0x0000000000d31b6d in std::__1::__invoke<rtengine::ThreadPool::ThreadPool(unsigned long)::{lambda()#1}>(rtengine::ThreadPool::ThreadPool(unsigned long)::{lambda()#1}&&) (__f=...) at /usr/include/c++/v1/type_traits:3694 #37 0x0000000000d31b45 in std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, rtengine::ThreadPool::ThreadPool(unsigned long)::{lambda()#1}>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, rtengine::ThreadPool::ThreadPool(unsigned long)::{lambda()#1}>&, std::__1::__tuple_indices<>) (__t=...) at /usr/include/c++/v1/thread:280 #38 0x0000000000d318d0 in std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, rtengine::ThreadPool::ThreadPool(unsigned long)::{lambda()#1}> >(void*) (__vp=0x80790c370) at /usr/include/c++/v1/thread:291 #39 0x0000000801ed07ea in thread_start (curthread=0x807814a00) at /disk-samsung/freebsd-src/lib/libthr/thread/thr_create.c:292 #40 0x0000000000000000 in ()
-
reporter It appears to crash while processing this image.
-
reporter This assert causes failure:
KMP_ASSERT(gtid < __kmp_threads_capacity);
-
reporter Building with OPTION_OMP=OFF helps.
-
reporter - changed title to 1.12 SEGVs when OpenMP is enabled
-
reporter I added the OPENMP option to the FreeBSD port, making it OFF by default. This way there’s no SEGV.
-
repo owner thanks. Seems like a very costly workaround though
I have no experience with freebsd but I'll try getting a VM and see if I can find the culprit
-
repo owner did you try compiling with clang btw?
-
reporter Thanks!
I have instructions on how to install FreeBSD into a VM:
Is there an easy way to test under FreeBSD?
- Install the FreeBSD VM image into VirtualBox.
- Boot FreeBSD
- Install git: pkg install git
- Check out the ports tree: git clone https://git.FreeBSD.org/ports.git /usr/ports
- cd /usr/ports/graphics/art
- Install dependencies: pkg install -A
make missing
- Build: make
- Install: make install
-
reporter did you try compiling with clang btw?
Port normally builds with clang-12.
-
repo owner thanks for the instructions! Is there a way to compile with GCC as well?
-
reporter Is there a way to compile with GCC as well?
Add
USE_GCC= yes
in the middle of Makefile first.
-
repo owner Reading through https://reviews.llvm.org/D77609, it seems ART is not the only app that hits the assertion. Can you try setting LIBOMP_NUM_HIDDEN_HELPER_THREADS=0 (as suggested in one of the comments there) and see if that helps?
-
repo owner Hi,
did you have the chance to try and see if
LIBOMP_NUM_HIDDEN_HELPER_THREADS=0
helps? I’d like to release a version 1.12.1 to fix a few (minor) glitches, but I wanted to sort this out first…
Thanks!
-
reporter The problem is gone with
LIBOMP_NUM_HIDDEN_HELPER_THREADS=0
-
repo owner Thanks for checking. Would it be possible to tweak your port to use a wrapper script to run ART with the env var set?
Alternatively, it might also be that this is an LLVM-specific issue, and compiling with gcc would also solve it.
(FWIW I tried with clang-12 on Linux but I couldn’t reproduce)
-
reporter Would it be possible to tweak your port to use a wrapper script to run ART with the env var set?
I think it’s better to add this environment variable in the beginning on the main function. It can be if-defed only for clang if this is clang-specific.
I just verified - this works.
-
repo owner If this is an issue with LLVM/clang, it might be version and platform dependent. So, I would prefer this to be addressed at the distro level. For example, can you apply the patch when building the port? I haven’t been able to reproduce the bug on Linux, even when using clang
-
reporter Ok, I will add such patch.
-
reporter The patch was added to the port.
-
repo owner great, thanks! I'll close this for now then
-
repo owner - changed status to resolved
- Log in to comment