Running out of memory constantly when working on NEF files
When working on NEF files from a NIkon D800E, ART uses 2GB more memory every time a parameter is changed, for instance the color temperature. Once the system memory is used completely, the ART process is killed by the system. This happens as well when going through different pictures.
This does not happen with other file formats. With the later, the available memory goes done, but the free memory stabilizes at around 2GB, even when more changes are done.
Let me know where I can send the NEF file privately if it helps.
Reproducible with 1.15 and 1.17.2
Comments (14)
-
repo owner -
reporter Yes, I build ART myself, because otherwise it does not even start on my system, due to:
(ART.bin:71335): GLib-GIO-ERROR **: Settings schema 'org.gnome.settings-daemon.plugins.xsettings' does not contain a key named 'antialiasing'
Here is the build information:
Version: 1.17.2 Branch: 1.17.2 Commit: d8be8afed Commit date: 2022-11-30 Compiler: cc 11.3.0 Processor: x86_64 System: Linux Bit depth: 64 bits Gtkmm: 3.24.5 Lensfun: 0.3.2.0 Exiv2: 0.27.5 LCMS2: 2.12 LibRaw: N/A OpenColorIO: N/A Build type: Release Build flags: -std=c++11 -ffp-contract=off -march=native -Werror=unused-label -fno-math-errno -Wall -Wuninitialized -Wno-deprecated-declarations -Wno-unused-result -fopenmp -Werror=unknown-pragmas -O3 -DNDEBUG -ftree-vectorize Link flags: -march=native OpenMP support: ON Mi-malloc: N/A Build OS: Linux Build date: 2022-12-17T06:58:00Z
-
repo owner thanks for the info. Please try rebuilding with mi-malloc enabled, there's more than one chance that it will solve the issue (it is on by default now, and I'm considering making it mandatory). I would also strongly recommend to enable libraw, but this is probably not related to your issue here
-
reporter Enabling mi-malloc solves the issue. It was not so easy to build with it on Ubuntu 22.04 because the pkg libmimalloc-dev 2.0.5+ds-2 does not provide CMake integration. I applied to following hack for that:
commit 1fac608e2372d3c4cb0485c4c1db5372b718478d (HEAD) Author: Olivier Croquette <ocroquette@free.fr> Date: Sat Dec 17 10:39:41 2022 +0100 Patch CMakeLists.txt for mimalloc diff --git a/CMakeLists.txt b/CMakeLists.txt index 5cc6b4e2b..f1a1e42ea 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -569,7 +569,9 @@ int main() set(HAS_MIMALLOC) set(ART_MIMALLOC_VERSION_INFO "N/A") if(ENABLE_MIMALLOC) - find_package(mimalloc) + # find_package(mimalloc) + set(mimalloc_VERSION 2.0.5) + set(mimalloc_FOUND TRUE) if(mimalloc_FOUND) if ("${mimalloc_VERSION}" VERSION_LESS 1.4) message(STATUS "found mimalloc ${mimalloc_VERSION}, but at least 1.4 is required")
-
repo owner thanks for testing, I'll try to take care of the cmake integration on my side. And I'll make mi-malloc mandatory from now on
-
reporter Thanks for the quick support. It would be also great if you could provide exact instructions how releases are build (which system, what packages are installed, what command line is executed) so that users can reproduce them more easily. That would also reduce the number of variants in the field and the support effort.
-
repo owner I'll see what I can do.
AboutThisBuild.txt
does provide some of the info though -
I have detailed build instructions for Ubuntu 22.04 where all relevant packages (libraw, exiv2, lensfun, ocio, mimalloc, …) are built from source. I wrote them down for my own use, any interest?
-
You can find my build recipe here
-
repo owner Thanks! I’ll link it from the wiki
-
reporter Thank you Danny, I will try it soon.
-
@agriggio I see that “tools/build-art” in master still uses tcmalloc and does not use libraw and ocio, is that intentional? Or is it simply because you do not use it yourself to build ART?
-
repo owner Indeed I don’t use that. But I’ll update it, thanks for reminding me!
-
repo owner - changed status to resolved
made mimalloc a hard requirement
- Log in to comment
Hi, are you building art yourself? If so, which platform and compiler? Can you attach your
AboutThisBuild.txt
file?And yes, a raw file would be useful. You can upload to Dropbox or similar. Thanks!