Segmentation fault when pasting processing profile from the File Browser on multiple images
Issue #63
resolved
With
Git checkout information:
– Commit description: 1.2-108-g2310526cd
– Branch: master
– Commit: 2310526cd
– Commit date: 2020-04-22
– Commits since tag: 108
– Commits since branch: 108
– Version (unreliable): 1.2.108
I get a crash when I paste a profile to a lot of photos (for this case there were 256 RAF compressed files from an Fuji X-T2) from the File Browser. It does not matter what the profile contains (I can reproduce it with just opening one file, click on the rotate 90 right button, copy the profile and paste it from the File Browser).
BackBuffer::copyRGBCharData(BackBuffer * const this, const unsigned char * srcData, int srcX, int srcY, int srcW, int srcH, int srcRowStride, int dstX, int dstY) (m:\code\art\rtgui\guiutils.cc:1715)
ThumbBrowserEntryBase::updateBackBuffer(ThumbBrowserEntryBase * const this) (m:\code\art\rtgui\thumbbrowserentrybase.cc:250)
ThumbBrowserEntryBase::draw(ThumbBrowserEntryBase * const this, Cairo::RefPtrCairo::Context cc) (m:\code\art\rtgui\thumbbrowserentrybase.cc:625)
FileBrowserEntry::draw(FileBrowserEntry * const this, Cairo::RefPtrCairo::Context cc) (m:\code\art\rtgui\filebrowserentry.cc:533)
ThumbBrowserBase::Internal::on_draw(ThumbBrowserBase::Internal * const this, const Cairo::RefPtrCairo::Context & cr) (m:\code\art\rtgui\thumbbrowserbase.cc:924)
libgtkmm-3.0-1.dll!libgtkmm-3.0-1!_ZN3Gtk12Widget_Class13draw_callbackEP10_GtkWidgetP6_cairo (Unknown Source:0)
libgtk-3-0.dll!libgtk-3-0!gtk_widget_get_allocated_baseline (Unknown Source:0)
libgtk-3-0.dll!libgtk-3-0!gtk_container_propagate_draw (Unknown Source:0)
libgtk-3-0.dll!libgtk-3-0!gtk_container_propagate_draw (Unknown Source:0)
libgtk-3-0.dll!libgtk-3-0!gtk_gl_area_attach_buffers (Unknown Source:0)
libgtk-3-0.dll!libgtk-3-0!gtk_container_get_path_for_child (Unknown Source:0)
libgtk-3-0.dll!libgtk-3-0!gtk_style_properties_lookup_property (Unknown Source:0)
libgtk-3-0.dll!libgtk-3-0!gtk_gl_area_attach_buffers (Unknown Source:0)
libgtkmm-3.0-1.dll!libgtkmm-3.0-1!_ZN3Gtk6Widget7on_drawERKN5Cairo6RefPtrINS1_7ContextEEE (Unknown Source:0)
libgtkmm-3.0-1.dll!libgtkmm-3.0-1!_ZN3Gtk12Widget_Class13draw_callbackEP10_GtkWidgetP6_cairo (Unknown Source:0)
libgtk-3-0.dll!libgtk-3-0!gtk_widget_get_allocated_baseline (Unknown Source:0)
libgtk-3-0.dll!libgtk-3-0!gtk_container_propagate_draw (Unknown Source:0)
libgtk-3-0.dll!libgtk-3-0!gtk_container_propagate_draw (Unknown Source:0)
libgtk-3-0.dll!libgtk-3-0!gtk_border_get_type (Unknown Source:0)
libgtk-3-0.dll!libgtk-3-0!gtk_container_get_path_for_child (Unknown Source:0)
The debugger says it is a segmentation fault and at this point because ‘src’ '0x508ed001 <error: Cannot access memory at address 0x508ed001'.
Comments (47)
-
repo owner
-
reporter
Not yet, but I will make some tests.
-
reporter
It does not seem to have anything to do with files outside the screen. I am not able to find a number of files when it always crashes.
What I can say now additionally is that it crashes after the arp files are written to the disk. I can find them all in the folder after the crash.
I will continue to try to get a more reliable workflow to crash it.
Could be something related to concurrency again? The crash always happens here guiutils.cc at one of the lines 1714,1715 or 1716.
r = *(src++); g = *(src++); b = *(src++);
-
repo owner
Those calls should be protected by locks, but concurrency is tricky, so it’s totally possible that something is going on…
FWIW, I am unable to reproduce on Linux, I’ll try on Windows now
-
repo owner
- attached lock.patch
still no luck here... just to make sure, can you try the attached patch? (I'm not expecting it to solve anything, but I want to rule out simple things first...)
-
reporter
You were right, it does not help, it just makes the thumbnails load very slow.
I’ve noticed that the same crash appears if after you paste the profile over all files you then clear the profile.
The debugger stops in the same place, with the same message.
-
reporter
one thing to try (if you did not do it already) is tu use a debug build. While the release build seems to survive sometimes, the debug one seems to crash almost all the time.
It can be a memory problem, as I am using compressed RAF files which I guess are a little more taxing on the memory than other type of RAW files.
-
repo owner
still no luck on my side…
I’ll try with RAF files and see if that might give a clue
-
Tried with 248 RAF don’t know if they are compressed( about 6GO)
master_1.2-109-g79b668673 skylake release and generic debug versions on W10
cannot reproduce.
very small impact on memory use.
-
repo owner
@Daniel Catalina did you try with other builds perhaps? just to check whether it’s related to some compilation settings
-
reporter
i tried with at least two, the one mentioned in the report and the same baseline compiled with the lock patch attached.
I will try today also with the latest master.
-
reporter
I tried with
Version: 1.2-114-geefe3e808
Branch: master
Commit: eefe3e808
Commit date: 2020-04-25
Compiler: gcc 9.3.0
Processor: undefined
System: Windows
Bit depth: 64 bits
Gtkmm: V3.24.2
Lensfun: V0.3.2.0
Exiv2: V0.27.2
Build type: release
Build flags: -std=c++11 -march=native -Werror=unused-label -Wall -Wuninitialized -Wno-deprecated-declarations -Wno-unused-result -fopenmp -Werror=unknown-pragmas -O3 -DNDEBUG -ftree-vectorize
Link flags: -march=native
OpenMP support: ON
MMAP support: ON
Build OS: Windows
Build date: 2020-04-26T14:14:00Z
It stil crashes for the scenario mentioned above. Otherwise, it is quite stable and fast, only when a profile is copied over multiple files it crashes. The call stack is the same as in the report.
-
reporter
BTW, I am compiling as recommended for RT in Rawpedia:
cmake -G "MSYS Makefiles" -DLENSFUNDBDIR=share/lensfun -DCMAKE_BUILD_TYPE="release" -DPROC_TARGET_NUMBER="2" -DCACHE_NAME_SUFFIX="-12" ..
Can this be a problem?
-
repo owner
would you be able to test a build made by someone else (e.g. one of the nightly builds), just to see if it's related to the compiler/tool chain used? that might help in narrowing it down
-
reporter
sure, I will try the last nightly this evening.
-
reporter
I tried with:
Version: 1.2-114-geefe3e808
Branch: master
Commit: eefe3e808
Commit date: 2020-04-25
Compiler: gcc 9.3.0
Processor: Skylake
System: Windows
Bit depth: 64 bits
Gtkmm: V3.24.2
Lensfun: V0.3.95.0
Exiv2: V0.27.3.1
Build type: release
Build flags: -m64 -mwin32 -msse2 -mfpmath=sse -mthreads -Wno-aggressive-loop-optimizations -Wno-parentheses -std=c++11 -march=native -Werror=unused-label -Wall -Wuninitialized -Wno-deprecated-declarations -Wno-unused-result -fopenmp -Werror=unknown-pragmas -DNDEBUG -O3 -ftree-vectorize
Link flags: -m64 -mthreads -static-libgcc -march=native -s -O3 -fno-use-linker-plugin
OpenMP support: ON
MMAP support: ON
Build OS: Windows
Build date: 2020-04-25T17:00:13Zand it crashes. I do the following:
- open ART
- point to the folder where the files are
- open the first one
- click on rotate 90 right
- click on copy processing profile
- get back to the file browser
- ctrl+a to select all of them (258 RAF files)
- paste the processing profile (does not matter if it is with Ctrl+v or from the menu)
- try to scroll the page (this makes it crash all the time, if I do not touch anything and let it be sometimes it does not crash.
I will try again later with:
- a different set of files (although I copied them again today from the sdcard, maybe one of them it is corrupted?!?)
- on jpegs
- with a ‘not optimized’ build
-
repo owner
ok, I think the missing piece of info was that you are rotating 90 degrees. I think. you said that early on but I did not realize that could be the culprit. I will investigate more and see if I can finally reproduce
-
repo owner
- attached thumb_draw.patch
can you try the attached patch?
-
reporter
Sure. I applied it and recompiled.
It seems to help, in regards that the thumbnails are updated and it no longer crashes while scrolling between them. But it crashes immediately after ART finishes generating thumbnails if I try to:
- double click on a photo to work on it
- or to scroll in the file browser
- or with all photos selected click on clear processing profile
I will make a debug build to see if it crashes in the same place or somewhere else.
-
- attached log_file.txt
ART_master_1.2-120-gf8ec2b0ba_W64_generic debug build
After applying @Daniel Catalina procedure above, I catched a segfault while clearing the profiles of all the files
-
reporter
@Gaaned92 thanks for helping!
Unrelated to this bug, I noticed that there are some differences between the ‘AboutThisBuild.txt’ files from the build I make myself and the one from your repo. I am just following the procedure from Rawpedia, are you doing something different?
-
- attached KT18fstdKe.png
Using offending procedure, scrolling through all photo to generate Thumbnails, then select all
-
The windows build procedure in rawpedia was completely rewritten by me. I surely put the minimal list of flags.
I surely modified my build flags after, but I am not able to say if it is better! I am going to check and try to remember the reasons.
One main difference is when you use the march flag. It generates a lot of specific -m flags that are I documented in “ART-W64NightlyBuilds/Windows ART builds read before downloading.md”
-
repo owner
@Daniel Catalina @Gaaned92 can you try with the latest master when you have the chance? Thanks!
-
reporter
@agriggio I made a clean build and deleted the cache, but I still get the crash with:
Git checkout information:
-- Commit description: 1.2-122-gb6b354244
-- Branch: master
-- Commit: b6b354244
-- Commit date: 2020-04-27
-- Commits since tag: 122
-- Commits since branch: 122
-- Version (unreliable): 1.2.122
-
- attached ZKuYL1RR9q.mp4
sometimes, the display of thumnail gets borked. Still searching for a reproducible procedure.
-
repo owner
I still have troubles reproducing this… anyway, I found something else that smells fishy. I pushed a couple more changesets, let’s see if we get closer…
-
reporter
no worries, we will get it in the end. The good part is that it is not a ‘normal’ workflow, so I would not make it a ‘release blocker’, but more of a thing that you put in the ‘release notes’ :)
-
repo owner
no release until this is sorted out
-
repo owner
btw I understand that the current master still segfaults, right?
-
repo owner
- marked as blocker
-
reporter
@agriggio I am compiling now with the changes since the last report (Commit: b6b354244) and will comment as soon as I can test it.
-
reporter
also, note that I did not checked with a debug build yet if the crash after Commit: b6b354244 is the same … :(
I will do also this today.
-
reporter
OK, it is time for some good news!
With
Version: 1.2-126-g26ea554c3
Branch: master
Commit: 26ea554c3
Commit date: 2020-04-27
It no longer crashes.
I get however this once the processing profile finishes to apply:
If I am going to another folder and come back or go in the editor and come back to the file browser the thumbnails are showing correctly.It also takes some time to complete the paste profile action. Since it’s not possible anyway to do something else during the paste profile process isn’t it possible to start some sort of modal dialog with a progress bar? This should help in this case as there is no visual indication anyway that something is happening until the thumbnails appear changed.
-
repo owner
We can do whatever, after this is sorted out. This is about not crashing, UI improvements can definitely wait. I’m still working on in though, trying to understand what causes the different behaviour (apparently platform dependent).
BTW, does f5/refresh help in showing the thumbnails, or do you really need to change dir and come back?
Thanks for your help!
-
repo owner
btw how many cores does your CPU have?
-
reporter
F5 reacts quite strange.
- With no image selected when I press F5 nothing happens, still no thumbnails visible
- With all images selected when I press F5 nothing happens, still no thumbnails visible
- With one image selected when I press F5 a dialog appears saying: 'This command requires all selected images to be queue processed first'
According to Windows this is the CPU I have:
Processor Intel(R) Core(TM) i7-8650U CPU @ 1.90GHz, 2112 Mhz, 4 Core(s), 8 Logical Processor(s)
-
repo owner
thanks. F5 is indeed strage, but seems like a separate bug. What happens if you cliick on the refresh button instead? (the one with the two arrows in a circle, top-right, next to the location bar)
-
reporter
The refresh button does the job. The thumbnails are fine after clicking it.
-
- attached log_file.txt
ART_master_1.2-127-g10b3c43c3_W64_generic
A crash regretfully I edited one file with rotation. applied the profile to all files. edited one other file.
Behaviour is much better with exception of glitch described above: the thumbnails are not redisplayed unless you change directory. No more borked thumbnail.
edit: still trying to reproduce the crash
-
reporter
this time seems to crash in a different place.
Can you please detail what ‘edited one other file’ means? Just open one file and changing exposure for example did not triggered a crash for me. According to the log, it crashes somewhere in LABMasks…
-
repo owner
yes, this seems completely different. @Gaaned92 can you please open a separate issue and add some details there? Thanks!
-
reporter
@Gaaned92 and @agriggio I can also sometimes make it crash after using this scenario and doing different random things afterwards. I created https://bitbucket.org/agriggio/art/issues/68/segmentation-fault-in-stress-test to gather information there.
-
reporter
I played a little (about half of hour) also with:
Version: 1.2-129-gca64b559b
Branch: master
Commit: ca64b559b
Commit date: 2020-04-28Did not managed to make it crash until now.
-
repo owner
great. let's play with this a bit more, and if things don't change in a couple of days were might close it (and get ready for 1.3)
-
Made a lot of tests and did not get other crash
-
repo owner
- changed status to resolved
Seems fixed, closing
- Log in to comment
thanks for the report. Were you able to identify a minimum number of images for which this happens? E.g. is it enough that you are pasting on thumbnails that are not visible on screen (I mean because you would have to scrolll down)? Or do you really need hundreds?