FlatCAM hangs in the "plotting ..." state

[voidloop]

I have successfully started FlatCAM Beta 8.994 but after opening a gerber file, the interface remains in the "Plotting ..." state, nothing happens.

Running FlatCAM in debug mode on my IDE I found that the program stops working here

https://bitbucket.org/jpcgt/flatcam/src/ebf5cb9e3094362c4b0774a54cf119559c02211d/appGUI/VisPyVisuals.py#lines-553

My knowledge of vispy is limited and I don't understand why this happens.

Could you help me understand what the problem is?

Thanks.

‌

[Marius Stanciu]

Could you help me understand what the problem is?

Not without knowing what you have done (what Gerber you used, generated by what software etc), not without a short description on how to replicate the issue…

[voidloop]

  <div class="preview-container wiki-content"><!-- loaded via ajax --></div>
  <div class="mask"></div>
</div>

</div> </form>

[voidloop]

Ok sure, sorry.

I attached the gerber file (Kicad). I simple start the program and load the gerber in attachment.

This is the venv I used to start FlatCAM:

% pip list  
Package           Version
----------------- ---------
absl-py           1.0.0
affine            2.3.1
attrs             21.4.0
certifi           2021.10.8
click             8.1.3
click-plugins     1.1.1
cligj             0.7.2
cssselect2        0.6.0
cycler            0.11.0
dill              0.3.4
ezdxf             0.17.2
fonttools         4.33.3
freetype-py       2.3.0
GDAL              3.3.3
hsluv             5.0.2
kiwisolver        1.4.2
lxml              4.8.0
matplotlib        3.5.2
networkx          2.8
numpy             1.22.3
ortools           9.3.10497
packaging         21.3
Pillow            9.1.0
pip               21.3.1
protobuf          3.20.1
PyOpenGL          3.1.6
pyparsing         3.0.8
PyQt5             5.15.6
PyQt5-Qt5         5.15.2
PyQt5-sip         12.10.1
PyQt5-stubs       5.15.6.0
pyserial          3.5
python-dateutil   2.8.2
qrcode            7.3.1
rasterio          1.2.10
reportlab         3.6.9
Rtree             1.0.0
setuptools        60.2.0
Shapely           1.8.2
simplejson        3.17.6
six               1.16.0
snuggs            1.4.7
svg.path          6.0
svglib            1.2.1
tinycss2          1.1.1
typing_extensions 4.2.0
vispy             0.7.0
webencodings      0.5.1
wheel             0.37.1

I’m using Fedora 35.

Do you need more information?

Thank you

[Marius Stanciu]

Hello,
I just tried to load your file in FlatCAM beta 8.994 (Windows 10 x64) and it works as expected, But the packages used are those available at the time when I made the 8.994 release.
That means that something on your system, or the video card, or the video drivers or some of the Python packages is interfering. You could try to launch the program in the 2D Legacy graphic mode (set in Preferences).

Or you could try the version in my cooking pot (prone to bugs and some features are broken) and see if you can load your file.
https://bitbucket.org/marius_stanciu/flatcam_beta/get/Beta_8.995.zip

‌

[voidloop]

I obtain the same result with Beta 8.995, but this version prints the following exception after Ctrl+C

% python FlatCAM.py              
[INFO][MainThread] Starting the application...
[DEBUG][MainThread] Creating empty tools_db.FlatDB
[DEBUG][MainThread] Creating empty current_defaults.FlatConfig
[DEBUG][MainThread] FlatCAM factory defaults written to: /home/exp/.FlatCAM/factory_defaults_Unstable.FlatConfig
[DEBUG][MainThread] App.__init__() --> Applied English language.
[DEBUG][MainThread] MainGUI.__init__() --> UI layout restored from options. QSettings set to 'standard'
[DEBUG][MainThread] Stardate: 20220509_172031
[DEBUG][MainThread] TCL Shell has been initialized.
[DEBUG][MainThread] App.save_project_auto_update() --> updated the interval timeout.
[DEBUG][MainThread] App.PreferencesUIManager.save_defaults()
[DEBUG][MainThread] App.save_project_auto_update() --> updated the interval timeout.
[DEBUG][MainThread] Finished creating Object Collection.
[DEBUG][MainThread] Setting up canvas: 3D
[DEBUG][MainThread] Finished Canvas initialization in 0.5810956954956055 seconds.
[DEBUG][MainThread] Finished creating Workers crew.
[DEBUG][MainThread] Tools are installed.
[DEBUG][MainThread] Initialization of the Geometry Editor is finished ...
[DEBUG][MainThread] Initialization of the Excellon Editor is finished ...
[DEBUG][MainThread] Initialization of the Gerber Editor is finished ...
[DEBUG][MainThread] Initialization of the GCode Editor is finished ...
[DEBUG][MainThread] Finished adding FlatCAM Editor's.
[DEBUG][MainThread] + Adding Exclusion Areas
[DEBUG][MainThread] -> First Run: Setting up the first Layout
[DEBUG][MainThread]  ---> New Layout
[DEBUG][MainThread]  -> Remove Toolbars
[DEBUG][MainThread]  -> Add New Toolbars
[DEBUG][MainThread]  -> Add actions to new Toolbars
[DEBUG][MainThread]  -> Connecting Editors Toolbar Signals
[DEBUG][MainThread] -> First Run: Updating the Defaults file with Factory Defaults
[DEBUG][MainThread] App.PreferencesUIManager.save_defaults()
[DEBUG][MainThread] App.save_project_auto_update() --> updated the interval timeout.
[DEBUG][MainThread] Recent items list has been populated.
[DEBUG][MainThread]  -> Connecting Toolbar Signals
[DEBUG][MainThread]  -> Connecting Plugin Toolbar Signals
[DEBUG][MainThread] Finished connecting Signals.
[DEBUG][MainThread] END of constructor. Releasing control.
[DEBUG][MainThread] ... Resistance is futile. You will be assimilated ...
[DEBUG][MainThread] ... I disagree. While we live and breath, we can be free!

[DEBUG][Dummy-4] open_gerber()
[DEBUG][Dummy-4] AppObject.new_object()
[DEBUG][Dummy-4] Calling object constructor...
[DEBUG][Dummy-4] 0.068485 seconds before initialize().
[DEBUG][Dummy-4] Gerber format found. (%FSLAX46Y46*%) 
[DEBUG][Dummy-4] Gerber format found. Gerber zeros = L (L-omit leading zeros, T-omit trailing zeros, D-no zero supression)
[DEBUG][Dummy-4] Gerber format found. Coordinates type = Absolute (Absolute or Relative)
[DEBUG][Dummy-4] Gerber units found = MM
[DEBUG][Dummy-4] Starting macro. Line 15: %AMRoundRect*
[DEBUG][Dummy-4] Continuing macro. Line 16.
[DEBUG][Dummy-4] Continuing macro. Line 17.
[DEBUG][Dummy-4] Continuing macro. Line 18.
[DEBUG][Dummy-4] Continuing macro. Line 19.
[DEBUG][Dummy-4] Continuing macro. Line 20.
[DEBUG][Dummy-4] Continuing macro. Line 21.
[DEBUG][Dummy-4] Continuing macro. Line 22.
[DEBUG][Dummy-4] Continuing macro. Line 23.
[DEBUG][Dummy-4] Continuing macro. Line 24.
[DEBUG][Dummy-4] Continuing macro. Line 25.
[DEBUG][Dummy-4] Continuing macro. Line 26.
[DEBUG][Dummy-4] Continuing macro. Line 27.
[DEBUG][Dummy-4] Continuing macro. Line 28.
[DEBUG][Dummy-4] Continuing macro. Line 29.
[DEBUG][Dummy-4] Continuing macro. Line 30.
[DEBUG][Dummy-4] End of macro. Line 30.
[WARNING][Dummy-4] Joining 17 polygons.
[DEBUG][Dummy-4] Union by buffer...
[WARNING][Dummy-4] Union(buffer) done.
[DEBUG][Dummy-4] New object with name: cnc-test-F_Cu.gbr. 0.036174 seconds executing initialize().
[DEBUG][Dummy-4] parseGerber.Gerber.bounds()
[DEBUG][Dummy-4] Moving new object back to main thread.
[DEBUG][MainThread] on_object_created()
[DEBUG][MainThread] on_object_created --> OC.append()
[DEBUG][MainThread] GerberObject.set_ui()
[DEBUG][MainThread] set_ui --> FlatCAMObj.to_form()
[DEBUG][MainThread] build_ui--> FlatCAMObj.build_ui()
[DEBUG][MainThread] parseGerber.Gerber.bounds()
[DEBUG][MainThread] build_ui--> FlatCAMObj.build_ui()
[DEBUG][MainThread] register_recent()
[DEBUG][MainThread]    gerber
[DEBUG][MainThread]    /home/exp/workspace/playground/cnc-test/plots/cnc-test-F_Cu.gbr
^CProcess ForkPoolWorker-5:
Process ForkPoolWorker-7:
Process ForkPoolWorker-3:
Process ForkPoolWorker-4:
Traceback (most recent call last):
Traceback (most recent call last):
  File "/usr/lib64/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/usr/lib64/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib64/python3.8/multiprocessing/pool.py", line 114, in worker
    task = get()
  File "/usr/lib64/python3.8/multiprocessing/queues.py", line 356, in get
    res = self._reader.recv_bytes()
  File "/usr/lib64/python3.8/multiprocessing/connection.py", line 216, in recv_bytes
    buf = self._recv_bytes(maxlength)
Traceback (most recent call last):
  File "/usr/lib64/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/usr/lib64/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
Traceback (most recent call last):
  File "/usr/lib64/python3.8/multiprocessing/pool.py", line 114, in worker
    task = get()
  File "/usr/lib64/python3.8/multiprocessing/queues.py", line 355, in get
    with self._rlock:
  File "/usr/lib64/python3.8/multiprocessing/synchronize.py", line 95, in __enter__
    return self._semlock.__enter__()
KeyboardInterrupt
  File "/usr/lib64/python3.8/multiprocessing/connection.py", line 414, in _recv_bytes
    buf = self._recv(4)
  File "/usr/lib64/python3.8/multiprocessing/connection.py", line 379, in _recv
    chunk = read(handle, remaining)
KeyboardInterrupt
  File "/usr/lib64/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/usr/lib64/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib64/python3.8/multiprocessing/pool.py", line 114, in worker
    task = get()
  File "/usr/lib64/python3.8/multiprocessing/queues.py", line 355, in get
    with self._rlock:
  File "/usr/lib64/python3.8/multiprocessing/synchronize.py", line 95, in __enter__
    return self._semlock.__enter__()
KeyboardInterrupt
  File "/usr/lib64/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/usr/lib64/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib64/python3.8/multiprocessing/pool.py", line 114, in worker
    task = get()
  File "/usr/lib64/python3.8/multiprocessing/queues.py", line 355, in get
    with self._rlock:
  File "/usr/lib64/python3.8/multiprocessing/synchronize.py", line 95, in __enter__
    return self._semlock.__enter__()
KeyboardInterrupt
[DEBUG][MainThread] App.PreferencesUIManager.save_defaults()
[DEBUG][MainThread] App.save_project_auto_update() --> updated the interval timeout.
[DEBUG][MainThread] App.quit_application() --> App Defaults saved.
[DEBUG][MainThread] App.quit_application() --> App UI state saved.

‌

With this Python virtual env:

‌

% pip list
Package            Version
------------------ ---------
absl-py            1.0.0
affine             2.3.1
appdirs            1.4.4
attrs              21.4.0
certifi            2021.10.8
click              8.1.3
click-plugins      1.1.1
cligj              0.7.2
cssselect2         0.6.0
cycler             0.11.0
darkdetect         0.5.1
dill               0.3.4
ezdxf              0.17.2
fonttools          4.33.3
foronoi            1.0.3
freetype-py        2.3.0
GDAL               3.3.3
graphviz           0.20
hsluv              5.0.2
importlib-metadata 4.11.3
kiwisolver         1.4.2
lxml               4.8.0
matplotlib         3.5.2
numpy              1.22.3
ortools            9.3.10497
packaging          21.3
pikepdf            5.1.2
Pillow             9.1.0
pip                21.2.3
protobuf           3.20.1
pyee               8.2.2
PyOpenGL           3.1.6
pyparsing          3.0.8
pyppeteer          1.0.2
PyQt6              6.3.0
PyQt6-Qt6          6.3.0
PyQt6-sip          13.3.1
pyqtdarktheme      1.1.0
pyserial           3.5
python-dateutil    2.8.2
qrcode             7.3.1
rasterio           1.2.10
reportlab          3.6.9
Rtree              1.0.0
setuptools         57.4.0
Shapely            1.8.2
simplejson         3.17.6
six                1.16.0
snuggs             1.4.7
svg.path           6.0
svglib             1.2.1
svgtrace           2022
tinycss2           1.1.1
tqdm               4.64.0
typing_extensions  4.2.0
urllib3            1.26.9
vispy              0.10.0
webencodings       0.5.1
websockets         10.3
zipp               3.8.0

‌

Thanks for your patience.

[Marius Stanciu]

Have you tried to run the program in 2D Legacy mode?

‌

[voidloop]

Yes, I have tried. The problem is still there.

When I have more time I will try to debug the code deeper.

Thanks so far.

[Marius Stanciu]

What I would try in your place is to run a vanilla virtual machine (in your case the host is Fedora and the guest can be the same OS but clean, without any changes) and manually install only Python and the related packages for FlatCAM Evo. See if the issue is again seen.

You have to understand that what you see is not a bug on the side of FlatCAM Evo (beta) but the issue is on your system/OS. That make it hard to debug so you need to take out the variables, one by one, methodically.
I asked you to try to run the software in 2D mode because it uses a different graphic engine, based on Matplotlib, and I wanted to see if the issue is VisPy related or even video card related.
So it looks like it has nothing to do with VIsPy/video card since you say you have issues with 2D mode too.
Then you could try to use the TCL commands who does not automatically do the plot once launched. Then by doing the plot_all Tcl command you could follow and see where it crashes.
Anyway, remote debugging is not really possible so it is up to you to analyze and see where the problem is. Please report here if you have any findings! Thanks!

[Matti Eiden]

I’ve got a hang also at plotting in 3D on Gentoo (RX 480). However 2D rendering works fine.

I’ve done a few hours of debugging and while I wasn’t able to figure out WHY it breaks, the GLU triangulation appears to be key. If I skip glu triangulation in _update_shape_buffers everything seems to work.

Additional observation would be that if I convert from map_async to just regular map when calling _update_shape_buffers it seems I’m able to plot 1-n times, where n is some number between 2 and 20. This is with two worker threads. More threads makes things worse.

So my gut feeling is that it’s got something to do with threading and GLU+Linux (ayy).

I do see libffi segfaults in dmesg. So presumably the worker threads are segfaulting at various GLU calls.

As a bonus, I’ve also tested on another Gentoo computer with Intel 620 GPU - the observations are identical, so we can rule out that it’s hardware related.

‌

Package           Version
----------------- ---------
absl-py           1.2.0
affine            2.3.1
attrs             22.1.0
certifi           2022.9.14
click             8.1.3
click-plugins     1.1.1
cligj             0.7.2
contourpy         1.0.5
cssselect2        0.7.0
cycler            0.11.0
dill              0.3.5.1
ezdxf             0.18.1
fonttools         4.37.3
freetype-py       2.3.0
GDAL              3.5.2
hsluv             5.0.3
kiwisolver        1.4.4
lxml              4.9.1
matplotlib        3.6.0
numpy             1.23.3
ortools           9.4.1874
packaging         21.3
Pillow            9.2.0
pip               22.0.4
protobuf          4.21.6
PyOpenGL          3.1.6
pyparsing         3.0.9
PyQt5             5.15.7
PyQt5-Qt5         5.15.2
PyQt5-sip         12.11.0
pyserial          3.5
python-dateutil   2.8.2
qrcode            7.3.1
rasterio          1.3.2
reportlab         3.6.11
Rtree             1.0.0
setuptools        58.1.0
Shapely           1.8.4
simplejson        3.17.6
six               1.16.0
snuggs            1.4.7
svg.path          6.2
svglib            1.4.1
tinycss2          1.1.1
typing_extensions 4.3.0
vispy             0.7.3
webencodings      0.5.1

‌

‌

[Matti Eiden]

Update: If I skip using the pool when calling _update_shape_buffers everything appears to work fine ( self.data[key] = _update_shape_buffers(self.data[key]) ) with GLU tessellation.

[Marius Stanciu]

Hello Matti,

I use Windows, all works ok from this point of view, so I saw no issue. But I understand your problem so I’ve added a setting in the Preferences, a checkbox which when checked will disable multithreading usage when adding shapes for the 3D graphic engine (VisPy based), as per your observation.
The change is in the latest commit on my bitbuclet repository:
https://bitbucket.org/marius_stanciu/flatcam_beta/branch/gerber_editor_upgrade

Pay attention, this version is really unstable, it has unfinished features or broken ones (tools database, unit changing and some things in the Editors).

Thanks for the report,
Marius

[Matti Eiden]

Very nice, thanks. Yes I also briefly tested on Windows and saw no issue. I was suspecting the code could silently hit the except block, but that didn’t happen.

‌