This is easy to fix actually. All the target value modifications need to happen in the main thread anyway (something that was not obvious in the beginning of the development). So we can defer code execution to the main thread a bit sooner in the processing chain. Should not have any performance penalty. On the contrary, it should improve performance and relieve the audio thread.
Issue #35 resolved