State of jitters
Issue #264
resolved
I have been asked by a client to evaluate if DWScript is suitable for use in a structural engineering application.
Specifically, the client is concerned about performance. Primarily with regard to floating point calculations but also flow control and host/script call overhead. For now, the scripting will be used to enable the end-user to define custom calculations and validation rules.
The client has done some tests with the Mandelbrot demo and while the performance is impressive (in particular the 32-bit jitter), there’s also some concern about the reliance on the jitter to achieve this performance. The client is focusing on 64-bit, Windows only.
So the questions here are:
- What is the current state of the 64-bit jitter?
(e.g. what’s jitted and what’s not) - Is the jitter considered a strategic feature of DWScript?
(Is it safe to rely on it, or something similar, being maintained in the future) - Are there any limitations or side effects when the jitter is used?
(e.g. I’ve noticed that single stepping in the debugger, understandably, doesn’t work on a jitted program)
Thanks in advance
Comments (6)
-
repo owner
-
reporter
Thanks for the prompt reply. This is excellent!
-
reporter
Btw, you’re welcome to close this issue as my questions have been answered. I can’t seem to do it myself.
Another thing, not related to this issue but to Bitbucket issues in general, is that I occasionally receive notification emails from Bitbucket when a new issue is created on this repo. But when I then try to view the issue here (e.g. in order to reply to a question) the issue is hidden… You might want to review the repo setting if this isn’t on purpose :-)
-
repo owner
I noticed the issue about issues under review on Bitbucket, as I have it as well… When going to the issue it says it’s under review and I need to login, even when already logged in. Doing nothing but waiting hours/days and the issue becomes accessible. I couldn’t find any setting related to that, so I assume it’s a bug/backlog in Bitbucket’s moderation algorithm...
Given that the repository is mirrored on GitHub, I am considering moving the issues there.
-
repo owner
- changed status to resolved
-
reporter
Given that the repository is mirrored on GitHub, I am considering moving the issues there.
Yes, please do.
I will be moving my own stuff there too - with mirrors on Gitlab or vice versa.
- Log in to comment
What’s jitted or not can be found in TdwsJITx86_64.Create, it’s where the JITter for expression tree nodes are found, the ‘InterpretedJITter’ references are for nodes that are currently not JITted, and fallback to Delphi code.
The JIT implementations are most complete on floats and integers. Boolean expressions are somewhat complete (incl. branching and loops), the rest is more on a case by case basis. Strings and interfaces are by and large interpreted.
Also while most calls to built-in math/integer functions are inlined, one weak point is calls to script functions, which take a detour through Delphi-side interpretation. This is because JIT calls still use the interpreted DWS stack rather than the CPU stack.
Calls from Delphi to script code aren’t particularly optimized, if you have script code to execute in a loop, it’s better to have the loop on the script side.
The 64bit JITTer is a key feature, it’s what I use in prod most of the time. Its testing coverage is a focus as well. The 32bit JITter on the other hand is unlikely to see any significant changes, and may face “performance rot” in the future (with previously JITted nodes becoming interpreted when features are added to those nodes).
You can make the JIT ‘steppable' with the jitoDoStep, this has performance implications but enables stepping into JIT (as well as breakpointing and stopping execution). Currently the JITted code can maintain CPU registers, but still always writes to the stack and other variables memory. This allows to evaluate as usual, but limits the performance, and may become optional in the future (ie. JIT would be allowed to keep some variable in CPU registers onlly, and they would no longer be “evaluatable” by the regular dws debugger). Also when jitDoStep isn’t used, a script with an infinite loop cannot be stopped: you have to kill the thread.
Another tool for Math JIT is the dwsTabularLibModule & dwsTabular, though it isn’t documented and very much experimental. It’s dedicated to performing math computations on tabular data, with an expression compiler to AVX2. Use case is quite narrow-ish, but when it applies, performance can be comparable to vectorizing C compilers.
The LLVM jitter hasn’t seen much love, it is rather complex and the JIT compilation is very costly, while the 64bit JIT is quite cheap. So it wasn’t investigated too much.