Canvas performance
At this time there is no optimization whatsoever for the plotting canvas. The whole project is plotted every time, where the contents are in the visible area of the canvas or not, and on every zoom and pan actions.
This is not an issue with small number of polygons, but as they grow, the problem becomes more critical: The program takes a long time to plot and when zooming/panning, it becomes unresponsive.
Comments (78)
-
reporter -
2 & 3 variants are quite good, it is necessary to consider a little time.
Perhaps it is worth paying attention to Cairo backend for matplotlib.Fail. -
reporter We should attach a Gerber that really challenges FlatCAM as is, so we can test how much improvement we can get. I will look into my files, mut if you have some large Gerber, please attach it.
-
Single Gerber doesn't challenges FlatCAM even it is very complex. It's enough to take any "Gerber" file, add 3-4 toolpaths & create cnc-jobs for them.
I think it would be helpful to attach FlatCAM project.
-
- attached Mega_181.zip
Visualizer performance test project.
-
reporter The Matplotlib Cairo backend might still be a good idea as long as we can take the data and supply it to the Qt Cairo Canvas (There is no Matplotlib-Qt-Cairo backend). If we use the Cairo backend we can generate bitmaps and work with those in the GUI. We will need to draw the axes and keep track of data coordinates ourselves though if we don't use MPL directly in the application.
-
What about zooming, in case of use bitmaps? Render separate bitmap for each zoom step, downscaling prerendered image in high resolution? How about a case if function of zoom is not discrete and controlled by mouse drag? Any ideas?
Also, it is quite good to decide on the list of the functions which are required from a visualizer (I know not all functions which are used by the FlatCAM). Manage objects, draw polygons, paths and so on.
P.S.
I've played with mpgl backend (https://github.com/ChrisBeaumont/mplgl/), but it seems to be slow too. It uses "glDrawArrays" function to render paths. No shaders. No caches.
In my opinion, at the solution of a task "in a forehead", we need to store all lines, paths, ... in video memory, use shaders to render them, change only MVP matrix to transform view (pan, zoom,
rotate). -
Some statistics
"Mega_181" project:
Parameter Value Paths count 2892 Vertices 508226 Paths count by path vertices count:
Vertices Path count 1-9 79 10-99 1794 100-999 980 1000-9999 36 10000-99999 3 Of course, using geometry simplifying with tolerance = 0.01 mm, we can reduce vertices amount to about 100k.
-
reporter For a feature list, I would start with:
- Discrete pan
- Discrete zoom
- Continuous pan
- Auto-sizing when the parent changes size
- Click and move mouse signals with coordinates
- Data coordinates (Abstraction of pixel coordinates)
- Grid
- Object transparency
- Layers
Also note that the drawing tool uses an "animate" function in matplotlib that allows for fast repainting.
Now that I think of it, the "animate" functionality might be useful for panning somehow. It creates a bitmap background of the current plot. Maybe this can be used for rasterizing and manipulating in high speed.
-
reporter I just found out that there is no Cairo canvas in Qt. There is
QPainter
andQCanvas
which support OpenGL.I must say that I'm rather scared of implementing a canvas from scratch. We might end up with a worse solution than MPL after spending a lot of time developing it.
I'm not totally against it, but I think we should consider the option of optimizing the current solution, i.e. drawing only what is necessary on the screen. We could restrict the plotting only to what is being shown, and also use cached simplified geometry when a lot of data is within the view (No need to plot more data than what fits in a pixel). This would help plotting in general and not just panning and zooming.
-
How about optimized implementation of the MPL canvas/renderer backend based on QPainter or QCanvas engine? It is necessary to realize not too many functions.
-
reporter Could you please explain? Do you mean writing a new backend?
-
Yes, new backend.
-
reporter How do we get started with writing a new backend? Do you have a plan on how to build the optimizations?
-
It is worth beginning with the classes inherited from "FigureCanvasBase" & "RendererBase".
First class must implement "draw()" function at least. Second - "draw_path()", "draw_image()", "draw_gouraud_triangle()".
Optimization can be performed at the different levels. At higher level we can hide some geometry (for example, hide toolpaths if there is CNC-jobs for them), simplify polygons with "shapely" lib. At lower level, as you told above, it is possible to simplify details of figures which sizes don't exceed the pixel size, use the fastest of the possible rasterizers. As a last resort, create separate draw function for pan/zoom with very very simple graphics.
I at the moment have no exact and thorough plan.
-
reporter The only advantage I see with this is the OpenGL support (The bottleneck would be in the Python implementation of the backend). Apart from this what advantage does this have above the current Qt4Agg backend?
Perhaps what we want is to just show bitmaps on screen and use MPL to generate the bitmaps. We can still use a MPL backend to retain all other features like axes, labels, event handling, etc.
We can save bitmap versions to memory:
import cStringIO from matplotlib.pyplot import figure, plot, title, savefig, imshow figure() plot([1,2]) buffer = cStringIO.StringIO() pylab.savefig(buffer, format='png')
And to read in:
import matplotlib.image as mpimg img = mpimg.imread(buffer) imshow(img)
Of course, we need to get rid of the axes before saving and make sure the scales are correct and so on. Also, the plotting should not go to the screen, just done in memory, and only have the
imshow
going to the visible canvas.Thoughts?
-
reporter In fact we should use
print_raw
instead ofsavefig
as shown here: http://stackoverflow.com/questions/5391026/matplotlib-alternatives-to-savefig-to-improve-performance-when-saving-into-aNo need to waste time compressing and converting to a specific format.
-
The bottleneck would be in the Python implementation of the backend
It's possible to implement it not on Python.
But let's stop on showing bitmaps. As I see, we need to use two figures and two canvases:
1) First figure & canvas is "offscreen" on wich we plotting all graphics (separate thread i assume). On replot procedure we should set appropriate axes limits & figure size (limits calculated on objects bounding box in axes coordinates; size is the same limits but transformed to screen space pixels via second figure axes), draw all objects and store as image (savefig()). Canvas backend - "FigureCanvasAgg".
2) Second figure & canvas will have visual apperance on the application form. It will have only axes with grid/tick labels & previously stored image. Backend - "FigureCanvasQTAgg".
That's right?
-
Apparently, it is really quite good variant, but with some issues... Image size
& alphaproblems for now. I will be engaged in it a bit later. It's necessary to calculate limits of the image correctly (bounding box around all objects). -
reporter Great progress!
For saving the bitmap don't use
savefig()
. It takes for ever. This is what I did:import matplotlib.pyplot as plt import numpy as np import cStringIO ram = cStringIO.StringIO() fig = plt.figure() ax = plt.Axes(fig, [0., 0., 1., 1.]) ax.set_frame_on(False) ax.set_xticks([]) ax.set_yticks([]) fig.add_axes(ax) N = 50 x = np.random.rand(N) y = np.random.rand(N) colors = np.random.rand(N) area = np.pi * (15 * np.random.rand(N))**2 # 0 to 15 point radiuses ax.scatter(x, y, s=area, c=colors, alpha=0.5) fig.canvas.draw() buf = fig.canvas.tostring_rgb() ncols, nrows = fig.canvas.get_width_height() print ncols, nrows img = np.fromstring(buf, dtype=np.uint8).reshape(nrows, ncols, 3)
Then to plot it inside the axes:
fig = plt.figure() ax = plt.Axes(fig, [0., 0., 1., 1.]) fig.add_axes(ax) ax.imshow(img) print fig.canvas.get_width_height()
I think I'm off by 1 pixel in the width and the height. And the color look slightly different. May have to do with the background color.
-
reporter Yes, it makes sent to use FigureCanvasAgg for the "hidden" plots.
-
reporter Some ideas on the bitmap cache:
Of course
cacheW
andcacheH
can be much larger and we might want to store many more levels of zoom.Then the next challenge is to decide when to recompute the cache.
-
Or can update image "on the fly":
Buffer with alpha:
buf = self.offscreen_canvas.buffer_rgba() ncols, nrows = self.offscreen_canvas.get_width_height() image = np.frombuffer(buf, dtype=np.uint8).reshape(nrows, ncols, 4)
And one more moment: i think, size of buffered image should depend on overall objects size, not on main canvas axes size. Something like this:
# Get bounds x1, y1, x2, y2 = self.app.collection.get_bounds() # Calculate bounds in screen space points = self.axes.transData.transform([(x1, y1), (x2, y2)]); # Calculate width/height of image w, h = round(points[1][0] - points[0][0]), round(points[1][1] - points[0][1])
But in this case, we have max zoom limit (image will be to big to store in numpy array).
-
reporter I don't think we need the alpha. Unless we want the grid behind the image.
size of buffered image should depend on overall objects size, not on main canvas
I agree that we never want to cache beyond the project's limits, but when panning and zooming it's always around the current view, so the data that is needed is always the data neighboring the main canvas.
Or can update image "on the fly"
It's kind of cool, but it's preferable to get the smooth/final zoom level instantly.
-
Good, as soon as there is time, will be engaged in it. By the way, i have problem with drawing to "offscreen" canvas in separate thread now. In spite of the fact that the separate figure is used, drawing procedure blocks any actions with "visible" canvas. Any ideas?
-
reporter Qt threads are quite complicated. You have to "move" objects between threads to work on them in the different threads. Take a look at the code where FlatCAM Objects (Gerber, Geometry, etc.) are created in the background thread. Once they are created, they are "moved" back to the main thread and the main thread is notified with a signal.
What you are seeing might have to do with passing data back and forth.
Maybe the new "FlatCAM Canvas" needs to have it's own background thread and the OffscreenCanvas should reside there permanently. Then only the bitmap is passed back to the main thread when computations are complete.
It would be good if you started pushing small changes in the code to the main repo so I can experiment with them and add to them myself too.
-
Maybe the new "FlatCAM Canvas" needs to have it's own background thread and the OffscreenCanvas should reside there permanently. Then only the bitmap is passed back to the main thread when computations are complete.
I had approximately same idea.Think will move this way.
-
reporter For reference code, see
FlatCAMWorker.py
and this is how threads are set up in the main app (lines 395-401):self.worker = Worker(self) self.thr1 = QtCore.QThread() self.worker.moveToThread(self.thr1) self.connect(self.thr1, QtCore.SIGNAL("started()"), self.worker.run) self.thr1.start()
Then, an example of usage would be:
def mytask(*params): ... result.moveToThread(QtGui.QApplication.instance().thread()) # Emit some signal here to alert the main process that the task # has completed and new data is available self.worker_task.emit({'fcn': mytask, 'params':[param1, param2, ...]})
-
reporter On getting pixel-size of axes:
-
reporter Just added a skeleton for the threaded canvas cache in 7d63ce3.
-
reporter @Denvi I created a new branch and started some work in it. The whole system is probably going to be quite complicated but I implemented some basic functionality. There is a separate thread that does the plotting and generates a bitmap. The new thread now does the plotting of objects. I modified the
FlatCAMObject.plot
to receive anaxes
as parameter, and it is called from the thread when it receives thenew_object_available
signal.Right now there is no cache whatsoever, and only does something when an object is created. I also think that I've got the calculation off by 1 pixel.
I would really appreciate any help.
-
I forgot nothing, I will be engaged approximately in a week. Congratulate on Winter Vacation!
-
reporter Excellent! Enjoy your vacations! I hope you don't mind that I work on it in the mean time. It's much more complicated than I though it would be, so there is plenty for 2 developers!
-
I investigate in a new code. What help now is required?
-
reporter Welcome back!
Please let me know if I need to explain the code that I've added lately. Anyway, it's the CanvasExperimental branch.
Here is a quick summary of things that are not yet implemented or need fixing:
- The calculations of image size/scale/resolution/etc are really messy and seem to be a bit inaccurate. You can see as you zoom in and out that the edges are not always well defined. Also when you zoom in a lot, it seems that only a small portion of the full project has been rendered.
- The system is aware of when it needs to create extra zoom levels when the user zooms in and out, but it does not create a new cache when the user pans and approaches the edge of the rendered image.
- Extra rendered images (for different zoom levels) are not being removed.
- The code is not very clean in the sense that references of multiple things are passed into new objects. For example,
PlotCanvas
needs a reference to theApp
,RenderCache
needs a reference toPlotCanvas
, etc. We would like to avoid that. - The code is not very clean in general. Any improvement is welcome.
- Zooming still seems slow. Now it does not depend on how complicated the project is, but switching the whole bitmap still seems like heavy work. One way to address this could be that we only plot part of the bitmap. Cutting pieces of Numpy arrays is very fast. This can all happen in the foreground.
- There is no support for when the screen changes shape (window re-sizing).
- Several things around the whole program are broken because of the changes.
I will try to add some docs in the wiki. I've drawn some diagrams that might help understanding things and for communicating between developers.
Please lets keep a good dialogue because I feel that the cached canvas part of FlatCAM has become quite complicated, so it's important that we understand what each other is doing.
-
reporter More thoughts:
-
Here is the wiki (work in progress, additions and cahnges are welcome): https://bitbucket.org/jpcgt/flatcam/wiki/Canvas
-
One simple problem that needs to be solved soon is that
PlotCanvas.get_axes_pixelsize()
is returning a fractional number. Should be an integer for sure.
-
-
reporter @Denvi Just so we can work at the same time without overlapping, lets post a quick message indicating what we will be working on each time. Also, pushing small changes more often might be a good idea.
-
I try to deal with the following problem:
as you zoom in and out that the edges are not always well defined
Besides, I would like to check bitmaps swap time. It's very strange that switching bitmaps is slow.
And:
pushing small changes more often might be a good idea
For me it is almost impracticable against almost continuous employment by other projects. As a rule, i can make some "attacks" on project as if it wasn't bad.
-
reporter I meant pushing small changes more often instead of larger changes. That way we know what each other is working on. Of course this is second priority to employment.
-
Probably that the problem is almost solved with offsets of the image. I tested on my variant of offscreen canvas. Will adapt under your code.
Main idea is to correct image bounds in user space by rounded image bounds in pixels. There is also constant 1 px offset on x-axis in offscreen canvas (meanwhile not clearly what its nature).
Here is 1x1 rect in different zoom levels:
-
reporter My code for calculating sizes and densities is very dirty. Perhaps it would be cleaner using Matplotlibs transforms but I'm not familiar with them.
I will be working on:
...create a new cache when the user pans and approaches the edge of the rendered image.
-
I meant pushing small changes more often instead of larger changes.
Well, I understood.
Perhaps it would be cleaner using Matplotlibs transforms.
Yes, especially as they can be used to both directions (px -> units, units -> px). Simply by inverting transform.
-
As a programmer with absolutely no knowledge of Python, have you considered the option to avoid to have the grid in the rasterized image? The grid can then stay drawn on the fly and on panning you move the raster and the grid separately. This way the raster size can be limited (let's say maximum 3x screen size, or objects bounding box, whatever is the bigger) while still having a perfect grid.
Regarding the zooming I believe the "rebuild after zoom level has been selected" is a perfectly fine choice, may be with some user feedback to inform the user the raster is being rebuilt (a progress bar or something...): zoom and pan to whatever you want to llok at, then wait for the image to be updated.
-
reporter Yes, the grid is not part of the rasterized image.
Zooming is also supposed to be instantaneous. Several zoom levels are cached, and new ones are built in the background as the user gets "close" to a zoom level that has not been cached.
If you are curious, there is a lot of code written for this already: https://bitbucket.org/jpcgt/flatcam/branch/CanvasExperimental
Some explanation of how this works: https://bitbucket.org/jpcgt/flatcam/wiki/Canvas
If you need the performance now you can look at @Denvi 's fork of the project here: https://bitbucket.org/Denvi/flatcam
-
Maybe the new "FlatCAM Canvas" needs to have it's own background thread and the OffscreenCanvas should reside there permanently. Then only the bitmap is passed back to the main thread when computations are complete.
Bad news. We can't draw in background without blocking visual canvas drawing. Check 'RendererAgg' class of Mathplotlib:
class RendererAgg(RendererBase): """ The renderer handles all the drawing primitives using a graphics context instance that controls the colors/styles """ debug=1 # we want to cache the fonts at the class level so that when # multiple figures are created we can reuse them. This helps with # a bug on windows where the creation of too many figures leads to # too many open file handles. However, storing them at the class # level is not thread safe. The solution here is to let the # FigureCanvas acquire a lock on the fontd at the start of the # draw, and release it when it is done. This allows multiple # renderers to share the cached fonts, but only one figure can # draw at at time and so the font cache is used by only one # renderer at a time ...
only one figure can draw at at time
I work on transition to VisPy library.
-
reporter Hi @Denvi, welcome back.
I'm not sure if you are interpreting this right. And if you are, it is certainly not a reason to transition to a different graphics library. If threading does not work, then a separate process definitely will.
Please understand that cache-ing the canvas is not a simple project and you need to work with me if this is going to have any chance to succeed. Attempting to change the graphics library does not help to solve this problem and certainly will be a source of countless new problems.
At this point what is most important is designing the whole cache-ing systems, including how and what to cache, and inter-process/thread communication.
If you really want to make a case for VisPy, you need to understand all the challenges that FlatCAM has solved using Matplotlib, and explain how they would be addressed with VisPy, and why it would be better. And you need to convince me. Also, if and after you convince me, you need to make a commitment to finishing it and have a clear plan because too much is tied to the graphics and it would limit the development of other areas of the program until it is done.
If you want to work on your own, you can. But I cannot guarantee that your work is going to make it back into the project trunk.
-
Job almost done. It is necessary to hold testing.
I use develop version (0.5.0.dev0) of VisPy which works well, but have some minor issues. In general, the speed of plotting/redrawing has considerably grown.
The following additional libraries are necessary:
- VisPy 0.5.0
- PyOpenGL
- Polygon2 (used for fast polygon triangulation, not for commercial use)
Plot controls:
- Pan: Right or Middle mouse button
- Zoom (smoth) : <Shift> + Right or Middle mouse button
- Zoom (steps): Mouse scroll wheel
I will make the corresponding pull request.
-
reporter Create a branch. I will not take this into the master. You have not talked to me about the implementation at all.
And I definitely dislike the special licence on Polygon2. Many people use FlatCAM for commercial purposes.
You are making this really difficult by not working with me on this.
-
The 'Polygon2' library can (should) be replaced of cause.
New branch "VisPyExperimental"
-
reporter Okay. Will look at it during the weekend. Please also describe what you are doing.
-
Denis, can you provide vispy-0.5.0 as I can't find it? The latest available seems to be version 0.4.0 and without v. 0.5.0, FlatCAM throws erorrs like: ImportError: cannot import name CompoundVisual.
I need a wheel or something that I can use in Windows. I am interested to see what kind of improvements brings your mods. Thanks!
-
You can install package from latest source with "pip":
pip install https://github.com/vispy/vispy/archive/master.zip
-
Thank you! Wow!
It works great and it looks great. What's missing is a self adjusting mark on the origin (0,0)(with zoom stay always at the same size). As it is, when panning, it's a little confusing looking for the origin.
There is one bug: if I try to get the coordinates by left clicking in the canvas it will no longer be the right ones so I can't use them in the Offset field to get the Gerber positioned with the left lower corner on origin.
-
self adjusting mark on the origin (0,0)
Сan you describe in more detail (screenshots)?
if I try to get the coordinates by left clicking in the canvas it will no longer be the right ones
I have fixed this in latest commit. Thanks for the remark.
You can also try 'develop' & 'multiprocess' branches. Last one have fastest project loading time.
-
Latest commit solved the coordinates capture and I could offset a Gerber to origin. Of course it would have been nicer to have the option "Move to Origin", probable by creating a invisible bounding box and moving the object (GERBER, Excellon or geometry) so the lower left corner will be positioned on the origin (0,0 coordinates) :) Although for Excellon the Origin shifting should be depending on the shifting data from the Gerber file...
Regarding the Origin mark I was thinking of something like this:
and it stays the same dimension on the screen regardless of the zoom level.
-
Variants of the origin:
option "Move to Origin"
I think it isn't relative to current issue.
-
Well ... neither are ideal as they are actually making more difficult to align to origin the object. But from both I think that the square is a little better.
Almost all the CAD CAM programs that I've worked with, used a bolded small cross (compared with the grid lines) made from simple lines (no volume). Or they just change the color of the grid around the origin. This way it does not interfere with the design but offer a way to easily identify the Origin.
-
How about this?
-
Much better!
-
One other minor issue. With your latest commit, the one that added the origin lines, if I select the Excellon file and check the Plot Options: Solid then some of the holes are plotted solid and some are not.
Also it will be nice to have a possibility for a finer zoom level: like pressing SHIFT key while zooming to zoom in finer steps.
Also, at the first try to load this Gerber (the one in the picture), Flatcam freezed and in the command windows I could see a lot of lines rapidly passing. Seemed the same line. Trying a second time it loaded without issues.
-
A bug: if I create a blank geometry, I can't edit it anymore. I can't do any drawing. On your latest commit, on your repository.
-
Flatcam freezed and in the command windows I could see a lot of lines rapidly passing.
Need this "lines", or gerber file to test.
A bug: if I create a blank geometry, I can't edit it anymore
Repared.
Will look for solid holes.
-
Hi Denis,
The solid holes not being displayed seems to have a random factor. If you click repeatedly on the check button, sometimes some holes are displayed as solid, next time might not.
Also, I suggest setting a limit for the "zoom in" feature. Because if you zoom in too much, the numbers on the axis become mangled as they have too many decimals and also the grid start to not be displayed completely, some of the lines are missing. But also there is no reason to zoom under let's say, 0.1 um.
-
reporter I agree with fixing the display of the numbers, but not limiting the functionality. The number format should be set to something like "{:.3f}" if x >= 0.001, else use scientific notation. I think it's "{:.3e}".
-
reporter @marius_stanciu that is definitely unrelated to this issue. Please try to keep the issue tracker as tidy as possible. For what you are mentioning, I think using the shell is an option. If you think it's really worth having in the GUI, go ahead and create an issue for it. I would call it "Batch operations on objects" or something similar.
-
Sorry for that. I was not sure but in this case I am deleting the post... Deleted.
-
About what I mentioned before that a Gerber could not be loaded and in command window some lines repeated themselves over and over. Latest commit made this permanent.
This is a print screen with the fenomenon:
Here is the picture in full:
http://i64.tinypic.com/20py4ra.jpg
And the Gerber can be downloaded from here:
https://mega.nz/#!l8pwQBiJ!1WE9CbvXyn85gGJUtISukbJc4u4OGyh2dV9dNemtCec
-
- Have fixed holes (now they are on one layer above gerber objects)
- Triangulation based on OpenGL GLU library now
- Memory usage improvements
Latest commit made this permanent.
Good news to debug...
Update:
And the Gerber can be downloaded from here
Loaded without any problems for me:
-
reporter Does GLU replace the other library that had the non-commercial restriction?
-
Yes, new tesselator has all needed functionality.
-
Now everything is OK on my side, too.
-
- Zooming is limited now
- Axes has less labels
- Added lines labels for cnc jobs (disabled in 'plotcanvas.py' due to the lack of a possibility to turn them off from the gui)
-
Hi Denis, As JP said, it is better to fix the labels on the axes and leave the zoom unlimited. Or if you think that limiting zoom is good then at least make it able to zoom in until at least 0.1 microns per division. At that level of zoom the grid is still functional.
Thank you for your hard work.
-
reporter For the record: I installed additional requirements for ubuntu as follows:
sudo pip install https://github.com/vispy/vispy/archive/master.zip sudo pip install pyopengl #sudo apt-get install libqt4-opengl sudo apt-get install python-qt4-gl
This should be added to
setup_ubuntu.sh
.For repeatability, we need to find a release of VisPy instead of using the latest.
-
it is better to fix the labels on the axes and leave the zoom unlimited
I can't figure out how to fix labels by the formatting in this case for example: 123.000001 ... some labels ... 123.000002
Have set minimum level to 0.01 MM (Inch), grid fail otherwise.
Latest changes are in 'VisPyExperemental' (typo...) branch.
- Increased stability
- Decreased memory consumption
For repeatability, we need to find a release of VisPy instead of using the latest.
As I know, there is no version 0.5 released. Can bundle dev version to FlatCAM (https://github.com/glue-viz/glue-vispy-viewers/pull/143)
-
reporter Limiting the zoom is alright for now. It is not critical. Later on you can look at how Matplotlib solves that problem. In summary, if you have an axis from 123.000001 to 123.000002, then 123 is noted on the side like "123+" and the labels say 1.00e-6, 1.10e-6, ... 1.20e-6.
Denis, let's talk about the pull request (in the pull request page). It's getting extremely large and also harder to integrate.
-
reporter Issue
#249was marked as a duplicate of this issue. -
reporter Issue
#244was marked as a duplicate of this issue. - Log in to comment
@Denvi Following up from
#180, I agree that Matplotlib might be a bottleneck. Some ideas:Plot/Re-plot only what is visible on screen. Geometry objects (can be extended to Gerber, etc) have their geometry indexed spatially, i.e. we van very quickly search for geometry within a given rectangle (and nearest object to a point, etc.). If the whole project is within view this will not help.
Rasterize. You mentioned it. When panning there is no need to recompute anything, just move the image. So we can keep a "rastered cache" of a plot larger than the current view and show that while panning. Then recompute the rastered image in the background when necessary. We would need to have some combination of a Matplotlib canvas an raster image canvas.
Move to something different than Matplotlib. Cairo is the standard for drawing and Qt has a widget. But it's rather low-level, so we would need to re-implement many features that we are using in Matplotlib.