Call through delegate on object fails when call is dispatched from Delphi side

[Anders Melander]

The problem occurs sporadically[*] but the circumstances that cause the problem are fairly specific.

*) or at least it used to be sporadic before the recent Eval/VarCopy change. Now I consistently get a nice, clean AV @ 00000000 in my own application.

Test case, in the form of a unit test, attached.

Steps

1. The script instantiates an object.
   The object is declared and implemented Delphi side via a TdwsUnit.
   The object declares a delegate (an event).

2. The script sets up the delegate (an event handler) on the object.

3. The script calls a method on the object.

4. The Delphi side implementation of the method calls back to the script through the delegate.

5. The script side delegate implementation (event handler) calls a script side factory function.

6. The factory function instantiates and returns a script side object.

7. Error
   The kind of error varies depending on the script and Delphi side implementation but the attached unit test causes an assertion failure on my system:

Project LanguageTests.exe raised exception class EAssertionFailed with message 'Assertion failure (C:\Projects\Library\DWScript_am\Source\dwsExprs.pas, line 3783)'.

Call stack:

dwsExprs.TProgramExpr.EvalAsScriptObj($58F61C,Pointer($730025) as IScriptObj)
:00406bf8 @Assert + $3C
dwsExprs.TProgramExpr.EvalAsScriptObj($38AFE50,nil)
dwsCoreExprs.TObjCmpEqualExpr.EvalAsBoolean($38AFE50)
dwsCoreExprs.TIfThenExpr.EvalNoResult($38AFE50)
dwsCoreExprs.TBlockExpr.EvalNoResult($38AFE50)
dwsExprs.TdwsProcedure.Call($38AFE50,$216E2B0)
dwsExprs.TFuncExpr.DoEvalCall($38AFE50,$216E2B0)
dwsExprs.TFuncExpr.EvalAsVariant($38AFE50,Unassigned)
dwsExprs.TProgramExpr.EvalAsScriptObj($38AFE50,nil)
dwsMethodExprs.TMethodStaticExpr.PreCall($38AFE50)
dwsMethodExprs.TMethodExpr.EvalAsVariant($38AFE50,Unassigned)
dwsMethodExprs.TMethodStaticExpr.EvalAsVariant($38AFE50,Unassigned)
dwsSymbols.TExprBase.EvalNoResult($38AFE50)
dwsCoreExprs.TBlockExpr.EvalNoResult($38AFE50)
dwsExprs.TdwsProcedure.Call($38AFE50,$216E620)
dwsExprs.TFuncExpr.DoEvalCall($38AFE50,$216E620)
dwsExprs.TFuncExpr.EvalAsVariant($38AFE50,Unassigned)
dwsExprs.TFuncPointer.EvalFuncAsVariant($38AFE50,$217B0B0,Unassigned)
dwsExprs.TFuncPointer.EvalAsVariant($38AFE50,$217B0B0,Unassigned)
dwsExprs.TFuncPtrExpr.EvalAsVariant($38AFE50,Unassigned)
dwsExprs.TFuncPtrExpr.EvalNoResult($38AFE50)
dwsInfo.TInfoFunc.Call((...))
DelegateTestLibModule.TWrapper.OnTestHandler($38F35A8)
DelegateTestLibModule.TFooItem.Test
DelegateTestLibModule.TDelegateTestLib.dwsUnitDelegateTestClassesTFooMethodsTestEval($210A140,$38F3518)
dwsComp.TdwsMethodCallable.Call($38AFE50,$2193690)
dwsExprs.TFuncExpr.DoEvalCall($38AFE50,$2193690)
dwsMethodExprs.TMethodExpr.EvalAsVariant($38AFE50,Unassigned)
dwsMethodExprs.TMethodStaticExpr.EvalAsVariant($38AFE50,Unassigned)
dwsSymbols.TExprBase.EvalNoResult($38AFE50)
dwsCoreExprs.TBlockExpr.EvalNoResult($38AFE50)
dwsCoreExprs.TFinallyExpr.EvalNoResult($38AFE50)
dwsCoreExprs.TBlockExpr.EvalNoResult($38AFE50)
dwsCoreExprs.TBlockExpr.EvalNoResult($38AFE50)
dwsExprs.TdwsProgramExecution.RunProgramExpr($38B9480)
dwsExprs.TdwsProgramExecution.RunProgram(0)
dwsExprs.TdwsProgramExecution.Execute(0)
dwsExprs.TdwsMainProgram.Execute(0)
UDelegateTests.TdwsDelegateTests.Execution
UDelegateTests.TdwsDelegateTests.GoForthAndDelegate

About the attached unit test

• I have attached 4 scripts. Of these only method delegate calling factory.pas causes the error.

• The technique used to perform a Delphi-side delegate call on a script object is based on the one I described in this stackoverflow post: DWScript: Getting from IScriptObj to IInfo or TProgramInfo. It has worked fine so far, except for this particular pattern.

FWIW I have been chasing this problem, in various forms, for over a year (hence the priority). See also issue #3.

[Eric Grange]

Seems caused by leaving & re-entering the script within the same execution, the second-re-enter is treated as if it was a direct call from the script, which results in an incorrect stack assumption.

Currently experimenting with simpler test case version.

(on a side node, all the try/finally/free and extra logic in the test cases only come out as overhead in the script)

[Anders Melander]

re try..finally: Good habits die hard :-) but I'll try to simplify in the future.

[Eric Grange]

hmmm... it's more subtile than I though, been piling variants of the test case and unable to reproduce. :/

Even just consolidating to a single program bypasses the issue...

[Anders Melander]

Yes, I know :-)

It's taken me a good while to reduce my own scripts to a "smallish" test case that still reproduces the problem.

I have this problem in three different places in my own scripts but each is just a variation of the exact same pattern.

[Eric Grange]

Committed a probable fix, related to the stack level when acquiring a function pointer that was based on the level the func pointer was acquired in, rather than the one it would have when used.

[Anders Melander]

Thanks. I'll give it a spin immediately.

[Anders Melander]

I can confirm that your change resolves the problem.

Thank you very much, Eric. You just saved our release.

[Eric Grange]

I am going to suspend the delegate test from the suite temporarily until I have time to rewrite the Delphi side: the current implementation is too complicated and also leaks memory (because of a reference cycle outside of the script).

[Anders Melander]

After a brief moment of happiness with a4be4be everything has gone to hell again :-) I'm now getting AVs left and right with 83b5cc2 and later.

I'm guessing the recent changes aren't the cause of these problems. Rather I think the changes have caused the problems to become visible.

Even after a revert to a4be4be I'm still getting a few AVs, so it seems there's some memory corruption or stale pointers involved.

I'll try to narrow the problem down some more but as far as I can see the pattern that causes the problem is a bit different.

[Anders Melander]

It seems I can reproduce with script code that doesn't involve delegate calls from Delphi-side so I'll probably write a new ticket. Still working on producing a small test case.

[Anders Melander]

Got it: #32