parallelize taks and keep GUI responsive

[BioBrillouin]

Hi,
I'm trying to sync multiple systems in a modular fashion. E.g. I have two docked windows in the itom main window each corresponding to different systems.

Currently while one is measuring the other one needs to wait until the measurement is done. Both systems need to be parallized bc otherwise the overall measurement time doubles. How can I achieve this?

Secondly bc I have this long integration times a Abort button would be a good idea. Same problem as above but within the same GUI. Otherwise I would like to have a global abort button as well.

best regards

[M. Gronle]

A parallel execution of several methods… in Python is a general Python issue. In general, Python does not provide a real parallel execution of different scripts or methods, Python rather provides a system similar to multitasking, hence, the different methods are always processed in one single thread, but Python executes a portion of the first method, then another portion of the 2nd method, an 2nd portion of the first method… Additionally, it might also be possible to use the itom.timer class to regularily call methods or class members in a certain time interval. Additionally all hardware plugins (motor stages and cameras) run in their own thread. For motor stages, you can set the async parameter to True. Then a setPosAbs or setPosRel command will return immediately while the motor stage is slowly moving to its target. The same holds for cameras. camera.acquire() returns immediately to only start the image acquisition. Then the camera acquires the image with a long integration time in the background. The call to camera.getVal / copyVal will wait until the image has been fully acquired or if this already happend returns the latest acquired image. All in all, your question is not so eays to answer. You should provide an example or more details about your requirements such that a help is easier to give.

About the abort: If you really want to abort a running exposure phase of a camera, I think that currently none of the camera plugins have implemented this. However, I also don’t know any camera SDK that provides this feature if new images are pulled and not obtained by a callback. In itom, most cameras are pulled by the plugins. In general, it is possible to add a special button in user interfaces of itom, that act like the script abort button of the general Python toolbar. Hence, a KeyboardInterrupt is emitted for Python that will stop the current script execution. Please see the demo_interrupt.py sample in the demo section of itom.

[BioBrillouin]

Hi,

this is linked to the activeX question prior. When running the COM-instance and hitting record phython is busy during integration time. I have to wait until it is done, even KeyboardTinterupt doesn’t work. So this is perhaps something more future related
The current problem ist to start the measurements in sync:
The other spectrometer just records the Serial port signal, so it wont need a trigger signal. It will recieve a line of numbers corresponding to different channels (Multichannel analyzer). Those need to be add up displayed as a histogram. So the main task is to keep the SerialIO recording to the memory during the COM-object is active and measuring. Then both methodes can be started together as long as the SerialIO ist started before the COM object (we are talking of integration times of several minutes, so a subsecond delay wouldnt impact this).

EDIT: The MCA has a rate about 2 Hz. But since COM-Object blocks python i cant use itom.timer

[BioBrillouin]

Still struggeling, when I try to start multiple processes by multiprocessing, threading or concurrent.future, then the script will open two new instances of itom. Would it be better to skip the parallelization on python and write two new plugins?

[M. Gronle]

The difference between multiprocessing and threading are listed in this stackoverflow article:

https://stackoverflow.com/questions/3044580/multiprocessing-vs-threading-python

Multiprocessing opens multiple completely independent instances of Python, calculates some subpart of the entire problem and each instance returns its part to the mother instance, where the subresults are merged. This is only partially possible in itom, since the newly opened instances can only be standard Python processes, where no itom specific methods are available. In order to configure this, please follow the hints given in this demo script:

https://itom.bitbucket.io/latest/docs/11_demos/python_packages/parallelization_threading/demoMultiProcessing.html#sphx-glr-11-demos-python-packages-parallelization-threading-demomultiprocessing-py

Threading is possible within itom like in any other python environment. Here is an example:
https://itom.bitbucket.io/latest/docs/11_demos/python_packages/parallelization_threading/demoworkerThread.html

However, python can only execute one thing at the same time. The thing that is currently executed holds the GIL (interpreter lock). From time to time, Python passes the GIL to another thread, that might then execute some lines of code. The benefit is, that many methods in Python do some complex calculation in some block of C-code or has to wait for any long operation (like the time.sleep command). During these operations, methods can also release the GIL and give the possibility to other threads to be executed. For instance any call to an itom algorithm plugin will also release the GIL and acquire it again if the operation is done. The threading module can for instance be used if a web request is done and waits for the content from any server. In this case, this operation will also release the GIL. A more modern concept for this are the new asyncio things of Python.

‌