Wiki

Clone wiki

quechua / flowcycle

Workflow lifecycle

This page describes data lifecycle in more detailed way.

Details

The state machine looks like below:

Quechua state machine diagram

  1. Initially every Workflow is in the IDLE state.
  2. When new data from channel comes, the state is changed to PREPROCESSING. For preprocessing stage a new thread is called and Processor::preprocess() method is called.
  3. When preprocessing is done,the new state will be POSTPROCESSED. The new thread is created (or reused) , state is changed to ALGRUNNING and Algorithm::work() method is called. When done, step now is ALGODNE.
  4. It's time to call (within separated thread) Processor::postprocess() method, step now is POSTPROCESSING and POSTPROCESSED when done.
  5. Logger comes to action, and logs the data. The state changes from POSTPROCESSED, through LOGGING to DONE.
  6. Check whether any new data was stored in the queue and return if not.

It is up tou you whether all the components will be used or not. In case a component is missing, particular steps will be skipped.

Let's say Processor is missing. In this case states like PREPROCESSING and POSTPROCESSING will be skipped and after IDLE will become PREPROCESSED and ALGDONE POSTPROCESSED.

Important When workflow is not in IDLE state it won't proceed any new data comming from channels. Otherwise it will store data in a queue and process them one by one. The reason is that synchronisation between components, even within single workflow) may be too expensive, besides, what is even more important, there might be situations where states (not machine states!) of Processor and Algorithm must be consistent throught the preprocess->algorithm->postprocess flow. So, again, when workflow f1 is in the ALGRUNNING state and any channel notifies about new data, no action will be carried out until f1 is in IDLE state again.

Updated